devopsy.kinsta.cloud

Category: Blogs

New DevOps Book: Leading the Transformation

When I first started researching DevOps and was looking for early enterprises that were successful but not “Unicorns” I was introduced to Gary Gruver. At the time Gary had led the transformation at Macy’s and he was talking about it at several DevOps events. For me it was strong proof that DevOps could work beyond the native-cloud companies and start ups. Gary had made it work at Macy’s and at HP before that. In fact Gary had co-authored a book on his experience at HP, “A Practical Approach to Large-Scale Agile Development: How HP Transformed LaserJet FutureSmart Firmware (Agile Software Development Series)”

I was lucky enough to meet Gary around then and have stayed in touch since then. He left Macy’s and was consulting helping companies transform from waterfall to agile and DevOps. I knew that he was working on another book. Well finally I am happy to report that Gary’s new book is now available:

Leading the Transformation: Applying Agile and DevOps Principles at Scale. Gary is actually co-author of this along with Tommy Mouser. The forward is by Gene Kim and the book is published by Gene’s IT Revolution. I have just started reading it, but knowing Gary and from what I have read all ready, it should be a must read on your DevOps book list.

As a special treat for readers of DevOps.com, we are very happy and proud to offer our readers a sneak peek at this exciting new book. At the bottom of this article is a link to a PDF with the first two chapters of the book free for you to read! Of course we are sure that once you read the first two chapters you will head over to IT Revolution and buy the full copy.

BTW, don’t take our word about how good Gary’s book is. Here is what Jez Humble, VP Chef, author and DevOps leader said:

“If you want to understand how to lead a Continuous Delivery or DevOps transformation in your company, there’s no better book than this. Concise, practical, and based on hard-won executive experience, this book is essential reading for every IT executive.”

So click the link and enjoy!

Leading the Transformation

August 6, 2015
DevOpsDays MSP 2015: Minneapolis is a Tech Hub?

I stopped in the Minneapolis Airport to make a connection earlier this year. I never spent much time thinking about Minneapolis given its reputation for snow and ice. I hadn’t ever planned to go there nor had I explicitly made plans to avoid it. That was until the lineup for devopsdays MSP came out.
I remembered seeing videos from the previous year and being mildly surprised they had such a seasoned lineup. When the speakers were announced I wasn’t expecting to see such a solid group of presenters taking the stage again. I like going to events in cities with a thriving tech industry. It always makes for a good hallway track when a lot of sharp folks are in the area and can easily attend. So what were all these people doing in Minneapolis?
It ends up a lot of tech heavy companies are in the area. Target, US Bank, Cargill, General Mills, Best Buy, Land O’ Lakes, and 3M are among them. None of these are pure tech companies like I would expect to find around San Francisco, but any company that’s competitive these days is probably leveraging a lot of technology in an effort to better serve their customers and keep things running smoothly behind the scenes. If you know anything about what I’ve been doing with CookieOps it should be no surprise I was thrilled to see technology applied to traditional business models so enthusiastically.
Target not only sponsored the event but had employees show up in force, fully engaged. 2nd place highlight for this event was discussing devOps, agile, and lean principles with Mary and Tom Poppendieck, authors of Lean Software Development: An Agile Toolkit, along with a group of folks from Target. Mary’s talk, “The New New Software Game” focused on techniques that make it possible for small teams to deploy small changes to large systems very rapidly and very safely. More specifically, this was a talk on how to break monolithic systems into micro services. I really appreciated how her talk highlighted human interaction and how it incorporates with technology.
Getting to listen to Mary and Tom talk shop with Target was incredible. They mentioned the shelf life of software on more than one occasion, and I may have gotten a little light headed when Mary used a cookie company as an example of a manufacturing process that’s deeply concerned with time to market.
The top highlight of the event for me had to be Daniel Willis’s ignite talk, “Sports Stats 101: Putting the R in Sports”. As a mathematician and a father I think getting kids into technology early is one of the best things we can do for them. When I saw Daniel was giving a talk on baseball stats in R at the age of 12 I had no choice but to make plans for Minneapolis. Daniel didn’t disappoint either, cracking standard deviation jokes in his bow tie in front of a crowd of about 400 people.
Colleen Velo’s talk, “Rolling Your Own vs SaaS” covered the tradeoffs between internally coding solutions, using open source software, and running infrastructure in the public cloud. Bloom Health, another Minneapolis business, is in the high risk environment of healthcare. I found Colleen’s perspective and experience on this subject to be extremely relevant given the new security challenges we’re facing in our industry and the trouble that higher risk sectors are having with innovation as a result of it. I wasn’t surprised to hear Bloom is moving to SaltStack giving the risk associated with the industry where they operate.
I also encourage you to take a look at Josh Zimmerman’s presentation on “Why You Should Care about DevOps in the Public Sector”. Public sector IT employees face a different set of problems with respect to structure, motivation, and funding. DevOps has moved from startups towards the enterprise, but there’s still a lot of good these practices can do to solve problems in the public sector where much of the resistance can be attributed to the need for a cultural change. Josh and I had a chance to talk about the cultural perspective of libraries, and how they’re viewed as places to keep books instead of places to keep information. I try to check out a library when I’m in a new town and see if they have anything interesting in the archives. This trip landed me in the Minneapolis Central Library where I was able to look at an early edition of Walden that included signed letters from Henry David Thoreau. There’s a lot of good stuff at the Hennepin County Central Library including a Dunn Bros Coffee on the first floor, a Teen Tech Center sponsored by Best Buy, art exhibits, and even a sound proof piano room. I stayed in Special Collections the entire time I was there, but if you’re in Minneapolis it’s definitely worth stopping by.

I really appreciated Jason Clifford ignite on GameOps, volunteering it as an Open Space, and coaching a group of us through a few rounds of The Resistance at the evening reception, which was sponsored by Target. Games like The Resistance and Werewolf help us learn how to interact and reinforce the low trust commensalism environment I’ve spent a good bit of time promoting this year and will be presenting this October at Lean Agile Scotland. If table top gaming isn’t low trust commensalism I don’t know what is. You literally have to share a table.

I also introduced the group to The Drawing Game during the open space. You start with a random doodle on a blank page and pass the paper and pen around to each person so they can add something to the picture. We made it around our open space twice before stopping and ended up with the picture below. This game is a life saver for any group stuck waiting to eat, especially if there are kids with you. A spare place mat and a pen borrowed from the staff have warded off more toddler meltdowns for me than I can count. It’s still fun for older kids and adults too, even if you’re just watching the scene unfold.

So how did all this awesomeness end up in Minneapolis? As far as devopsdays MSP is concerned I’d give a lot of the credit to Bridget Kromhout. It’s obvious she puts a lot of time into the devOps community at large and hasn’t forgotten to help build a strong community in her own backyard. The food, shirts, event communications, AV, and even the slides and videos on the website for the event can all be traced back to her and the rest of the amazing staff. Bridget promised an awesome event while in New York but I was skeptical she might be partial when she started talking about the wonderful warm weather in Minneapolis. I can confirm first hand it doesn’t snow year round in Minneapolis, and devopsdays MSP happens during the perfect time of year for a visit.
We’re only two weeks out from devopsdays Pittsburgh, and I’m looking forward to attending the event, visiting this town for the first time, and meeting some new faces while visiting with some more familiar ones. My expectations are pretty high thanks to Minneapolis, Bridget, and the entire MSP crew. Is Pittsburgh a tech hub too?

August 6, 2015
The Art of DevOps
This is the third in a series of posts on DevOps. The first written by my colleague Lee Reid was titled The Simple Math of DevOps. The second The Calculus of DevOps was written by IBM client, and my friend Carmen DeArdo of Nationwide. Lee introduced mathematically improving delivery throughput by leveraging DevOps to improve Trust. And Carmen used Calculus to imply that the business value of IT can be improved by adopting DevOps to increase the frequency of releases. In my post I would like to visit the art of DevOps , or the art of adopting DevOps by picking up right where Carmen left off – by looking at the culture side of adopting DevOps.

Lee and Carmen have already alluded to some basic premises behind DevOps as an approach – reducing batch sizeand improving collaboration. These result in reducing complexity, increasing throughput, and improving trust. Let’s talk about how to get there.

As I have worked with customers adopting DevOps, across company sizes, across industries and countries, there are four common threads that make DevOps adoption successful, leveraging all three aspects of DevOps adoption – culture, process and automation. These are:
1. Reduce batch size: The most effective way to manage risk and quality, while increasing speed is to reduce the batch size in each iteration or ‘sprint’. This is a mind shift to release smaller, more frequent new versions. Where it is not possible to release new versions that frequently to customers, release small batches to a pre-production area. The deliverable in pre-production is tested and made customer-ready, and then released to the customer at formal release dates. As Lee and Carmen have both discussed, reducing batch size is a pre-requisite to improving both trust and business value.
2. Shift-left Operational engagement: Get operations engaged right from the beginning of a project/initiative. This has been a core principle of DevOps. Do not leave Ops in a silo that one throws code over the wall to, but have them engaged in the development process, right from requirement inception stage. This allows Ops to have visibility into what is coming from Dev; allowing them to deliver the right production-like environments, as and when needed to Dev and Test teams. It also allows Ops to shift operational concerns left, left into the minds of developers, making them more Ops conscious and engaged in what happens after they hand-off to Ops.
3. Continuous funding: This is a significant change in an organizations funding model to enable continuous availability and engagement of SMEs and business functions, previously not engaged on a continuous basis – like security, legal, marketing, etc. Carmen in his post mentioned several examples of ‘waiting for…’ work, someone else to do something and environments. Another ‘waiting for’ in enterprises is for funding. Funding is typically provided in a waterfall manner, with specific hard dates (Months, quarters, fiscal year) and gates, not suitable for a Continuous Delivery model. Funding too should be continuous.
4. Create a ‘Product Management’ team: This team includes a Designer, Development lead, operations lead and an Architect at the minimum (4-in-box). They own the ‘product’ through its entire lifecycle, and beyond transient projects. They are responsible for long-term thinking, design thinking, evolutionary architectural thinking, and overall ownership of the product, from concept to end-of-life.
5. Setting up a DevOps Center of Excellence (CoE): This center of excellence is not an administrative organization or a ‘tools/enablement group’, but a place where DevOps adoptees come to learn from each other and share expertise and lessons learnt. As organizations adopt and scale DevOps adoption, this CoE also becomes a source of DevOps coaches, helping teams and programs adopt DevOps, and own the organizations DevOps framework – their own flavor of DevOps.
Adopting DevOps is an art. It is not as simple as hiring a consultant who can show how to improve processes; it is not just buying and adopting tools to automate manual tasks; and it is not going through ‘fall-back-into-the-arms-of-the-person-behind-you’ exercises to build trust. It is a combination of all three – process improvement, organizational and cultural change, and automation with tools to replace manual processes. Adopting these in any enterprise, outside of startups, requires overcoming organizational inertia. By changing culture not by edict, but by showing the need and value of change. By building trust, not through meetings, but with improved communication and transparency. By improving business value, not by measuring mandated KPIs, but by improving organizational agility and throughput. Overcoming organizational inertia, by delivering towards a common goal for the business, irrespective of where in the organization one is, and what role one has. By developing a culture where everyone takes responsibility of delivering value to the business.

Watch the webcast series hosted by DevOps.com in partnership with IBM or download a copy of DevOps for Dummies book for more information.

About the Author/ Sanjeev Sharma

Sanjeev Sharma, CTO and Distinguished Engineer – DevOps Technical Sales and Adoption, IBM Cloud Unit, is a 20-year veteran of the software industry with expertise in DevOps, Mobile Development and UX, Lean and Agile Transformation, Application Lifecycle Management and Software Supply Chains. He is a DevOps Thought Leader at IBM and speaks regularly at conferences. He has written several papers and is the author of the DevOps for Dummies book.

Contact Sanjeev on
August 6, 2015
Benefits for the Product Team in Working Support Cases

An increasingly popular practice that’s being adopted by DevOps groups are developers working regular shifts in front line support. The thinking behind this exercise is pretty sound. When exposed to the real world problems and frustrations customers face when using their software, a developer’s work becomes more about supporting that customer experience and increasing customer satisfaction and less about “Get the tests passing and ship it fast!” The practice also increases an individual developer’s understanding of the larger solution their company provides. If a particular developer is master over a small part of the total application, working support will give them valuable insight into the bigger picture of the problems they’re solving and how well their corner of the world is supporting the larger solution.

We practice this at my company, Datical, but on a wider scale. In addition to the app dev engineers every member of the product team, myself included, regularly works support cases in direct contact with the customer. It has been a pretty eye opening experience for all of us and we are most certainly realizing the promised benefits of this practice.

Our product team operates with a single guiding principle: Well defined, customer driven requirements yield excellent solutions. When you schedule calls or take customers to lunch, you get a lot of great feedback on where your product should go and net new features you should consider implementing to improve the overall solution. The conversations we’ve had with customers in this way are generally very positive and supportive with a focus on future direction and growth.

Working a support case is an entirely different and much more crucial customer conversation. The conversation that takes place during a support interaction is centered on a pressing need. The customer may need clarification on usage that isn’t as intuitive or well documented as it should be. They may be dealing with a gap in previous requirements or a coding defect that is keeping them from completing a goal. Whatever prompted the call, it’s occurring because the customer is not delighted and we are getting a firsthand, real time account of what is problematic and what we must do better. These needs translate directly into our product plans and requirements.

But there’s more to glean from these interactions than the matter at hand. When the customer is amenable we like to use GoToMeeting to remotely look over their shoulders as they demonstrate for us the issue they are having. During their demonstration, we get additional insight into the mechanics of our solutions. How do customers interface with our products? Are there minor inefficiencies our users have to contend with that they aren’t reporting? A few of the most well received features we have delivered since adopting this practice have been very small stories that were pretty easily addressed, which removed a minor annoyance that was encountered frequently in our workflows. We’ll release these fixes with a minimum of fanfare, but they are almost always noticed, commented upon and appreciated by the customer.

The last class of lessons learned from our foray into front line support pertains to my new favorite non-word, troubleshootability. When something goes wrong, how easy is it to diagnose the problem and how long does it take? How detailed and centralized is your logging? Are users being updated on the progress of long running operations? If an error is encountered due to customer misuse, is the customer appropriately guided to correct the error? Anything that we can do to shorten the cycle and cut through the murk when something does go wrong improves the overall experience with the product and increases customer trust.

Working support in product management and engineering has been a really great exercise. It’s given us a deeper understanding of our solutions, our users, our strengths and our weaknesses. This understanding has led to requirements that are tightly focused on our customers’ needs, holistic development practices concentrated around comprehensive customer workflows, and a solution that clearly communicates what’s going on as it happens for fast diagnosis and remediation of issues in the field. I highly recommend it.

About the Author/Peter Pickerill

Pete Pickerill is Vice President of Products and Co-founder of Datical. Pete is a software industry veteran who has built his career in Austin’s technology sector. Prior to co-founding Datical, he was employee number one at Phurnace Software and helped lead the company to a high profile acquisition by BMC Software, Inc. His ability to understand product demands from a customer’s perspective and translate those demands into actionable product and development plans has led to expanded duties at every company for which he’s worked. In his time at Symantec, Pete worked directly with large financial services companies and online retailers to implement and improve online fraud prevention solutions that were used daily by millions of people worldwide. In addition to managing a development team and product schedule at BMC, Pete worked directly with large financial services institutions to provide the features they needed to achieve greater efficiency and cost savings in their application deployment processes.

Connect with Peter on twitter: @PetePickerill

August 5, 2015
Scaling Lean and Agile for the enterprise: All hands on keyboard!!!
Lean and Agile methodologies are nothing new, but in the last few years we have seen the emergence of several frameworks combining elements of both. The IBM DevOps framework is one clear example. Combining principles of Lean and Agile into a single framework enables scaling of the core benefits to the entire organization, not just pockets of experts dispersed across functional silos. The goal?
- faster release cycles, increased responsiveness, reduction of waste, lower cost base
- increased software quality, customer-centric feedback loops, cross-discipline collaboration
That said, these process transformation initiatives too often remain confined within the Engineering or IT side of the house. And given that more and more large scale software projects (mobile and web) are driven and managed out of the Marketing or LOB organizations, we need to get to a place where Engineering/IT and the Business share a common vision and framework for continuous software delivery. This blog will discuss key areas where the Business needs to engage to truly scale these benefits to the organization level.

Let’s take a look at the key principles behind these frameworks and highlight those which mostly impact the behavior of analysts, project/program managers and LoB executives on the business side:
- Optimize the whole: Development teams cannot get it right by themselves. The business must be actively engaged to ensure customer/user needs are properly understood, in-turn ensuring the intended outcome of the application/system is well communicated and understood, and eventually the overall effort is continuously steering toward the right business outcomes
- Eliminate waste: Identifying/removing wait time to reduce overall turn around and cycle times (active participation during needs/requirements analysis, timely inputs into work prioritization, immediate feedback on early prototyping/release). Discover the hidden factory!!!
- Empower people: Let the real user directly represent their needs, and build interaction systems and processes that make it easy for those with the needs to engage across the lifecycle
The most common symptom that engineering and the business are not in sync is a constant flow of “management urgencies” that are forced into the current sprint or iteration. These “must do now” urgencies are a massive source of distraction for the teams executing the work on both the business and the engineering side. Directly responsible for substantial loss in productivity across the whole chain, as well as a prominent source of friction in relationships between business and IT, these distractions and interruptions can be avoided with the right level of orchestration and collaboration.

So which are the crucial touch-points where the right level of engagement from the business can generate the most impact?
- Early definition of business needs: This is not about producing lengthy system requirements documents, it’s about interaction with software architects and designers early and often to properly describe the needs/expectations of the target users/customers, it’s about making sure the overall context is defined and understood. There will be several opportunities to get into detailed requirements as the work unfolds.
- Early involvement into definition of success/acceptance criteria: When I was a BA, little humbled me as much as thinking I had done a good job describing a business requirement only to then freeze when I was trying to define objective criteria that would serve to develop user acceptance test plans. Forcing the definition of acceptance criteria at the time requirements are defined often surfaces areas of the needs that could be better defined or drives important exchanges between engineering and the business.
- Early and on-going engagement into prioritization: Gone are the days when setting up priorities was the result of a monthly or quarterly exercise; customer/user behaviors change continuously, so should your alignment to their needs. Prioritizing monthly or quarterly is like driving your car on the freeway in a pitch dark night and turning on your headlights every 2 mins or so to get an idea of where you heading… a lot can happen in 2 mins driving at 100km/hour. Prioritizing is also no longer an exercise in faith/best-guesses/loud voice in the room…. this new world is about constantly adjusting based on validated learning; running A/B testing, driving early prototyping and experimentation, monitoring application usage, social media analytics all serve to drive the right backlogs/priorities of teams across the software delivery lifecycle.
So where to start?

This transformation is a journey, not a one-time call to action, different paths can be taken depending on your organization’s goals, culture and priorities. We’ve helped hundreds of organizations over the years plan their way forward, helping them structure teams and investments while respecting their constraints and objectives. Be bold, take this on-line DevOps Practice Self-Assessment! Or visit our blog on IBM’s support for the Scaled Agile Framework (SAFe).

Also, do visit us at Agile 2015 to experience IBM DevOps solutions for collaborative development, continuous testing and continuous release and deployment.

About the Author/Roger LeBlanc

Roger is the Strategic Offering Lead, DevOps Application Platform driving the Enterprise Scaled Agile solution. He joined IBM following the acquisition of Systemcorp ALG, serving as Managing Director, Professional Services and has since been involved in ensuring a smooth transformational journey for the clients. He has a credible perspective and proven track record for meeting executive stakeholders expectations and goals. Roger has contributed to revenue growth and customer successes as well as provided strategic oversight and guidance of customer engagements.

Connect with Roger on
August 5, 2015
Alert Logic lends more agile, cloud-native security to DevOps architects

Amid the avalanche of research and product news emerging from this year’s Black Hat USA 2015 conference, held in Las Vegas this week, at least one vendor is attempting to advance a new solution aimed at empowering DevOps architects to streamline and improve security workflow.

Houston-based Alert Logic announced the release of its Cloud Insight solution on Tuesday, positioning the offering as a cloud-native vulnerability and configuration management package designed for customers running on Amazon Web Services (AWS). Cloud Insight promises to integrate with AWS native security features, providing an integrated view of potential host and application-level vulnerabilities.

By empowering operational teams, including DevOps architects, to identify and move to remediate vulnerabilities before affected applications code is pushed into production, company officials claim, the solution promises to both improve the configuration of resulting infrastructure and remove obstacles related to traditional cloud security management.

“This solution is specifically aimed at helping AWS users who are not security professionals, such as operations and DevOps teams who deploy the infrastructure, address inherent security issues,” said Rahul Bakshi, Senior Director of Product Management at Alert Logic.

“When you think of rolling security into the existing model, typically it interrupts the notion of continuous deployment, as historically it hasn’t integrated via the application or plugged into automated orchestration systems, becoming a hindrance,” Bakshi said. “That’s where this approach introduces a significant advantage, allowing organizations to identify and fix high risk vulnerabilities before the code gets pushed to production.”

Unlike traditional vulnerability management specialists, such as Qualys or Tenable Network Security, which require users to manually configure and review data and provide only a “partial” view of cloud platforms such as AWS, Alert Logic claims that Cloud Insight lends critical automation to involved processes and allows ops teams to gain far deeper visibility into AWS services level security.

Alert Logic has already won some customers over with its ability to enable greater control over applications security in the cloud. Speaking at the annual AWS Summit, held in San Francisco in April, Colin Bodell, Chief Technology Officer at publishing giant Time Inc., highlighted some of the work his company has done leveraging the vendor’s existing Security-as-a-Service offerings.

“AWS is great for physical security, or for network security, but when you’re building an application you need to own security of the application yourself,” Bodell said. “AWS doesn’t know how you’re building things and you can build them in any weird and whacky way that you like; Alert Logic has been an amazing partner.”

In the recent report, “Market Overview: Cloud Workload Security Management Solutions – Automate or Die”, published in June, analysts at Forrester Research highlighted the specific need for new security methodologies that integrate and automate processes to stem potential DevOps bottlenecks.

“One of the key benefits of DevOps is to enable continuous integration of code and continuous delivery of apps and infrastructure,” the Forrester report notes. “To keep up, [security and risk management] pros must make security processes more embedded, faster, and more continuous – otherwise, security will slow down code release.”

In terms of bringing Cloud Insight – which exists as a solution, rather than a SaaS service offering – to market, Alert Logic claims that it is successfully appealing to organizations’ hopes to embed AWS vulnerability management further within operations, despite introducing an altogether new and unfamiliar approach.

“Beta customers seem very positive as we’re reducing the security workload and time related to remediation handoff for cloud architects,” Bakshi said. “There’s always a continuum for education, but with the culture of responsibility [for AWS security] evolving, we think that we’re meeting demand at just the right place.”

Dan Raywood, Analyst at 451 Research, said that Alert Logic could take advantage of related demand by leveraging the combination of its existing cloud security expertise along with capabilities added via its January 2015 acquisition of Critical Watch.

“I think it’s a very interesting move for them as even though they did have something offering this‎ type of scanning previously; the acquisition of Critical Watch added the engine to do continuous scanning and change management,” said Raywood. “The major bugs of the last year have driven business to assess and respond immediately, and having something which tells you if you are vulnerable or not a few clicks away is a really useful tool for businesses.”

About the Author/ Matt Hines

Matt Hines has over 18 years as an IT marketing/communications professional at companies ranging from ground-floor startups to established industry leaders with emphasis on advancing broader understanding of key technology industry trends, market drivers, customer requirements and innovative products to highlight value, increase awareness and drive revenue growth. Additionally Matt has a decade-plus writing in IT journalism covering leading trends for a wide array of industry-leading business technology publications including eWeek, InfoWorld, CNet and TechTarget. You can reach him on LinkedIn

August 4, 2015
Tailoring DevOps to your IT: A case study for SAP
I am frequently asked if DevOps principles can be applied to legacy environments such as SAP. They clearly can, with some tailoring. But you should be aware of certain constraints.

An agile transformation from conventional IT (where the SAP software delivery process is split between business analysts, development, the QA department and IT operations) towards an agile DevOps organization requires the following steps:
1. Organize development teams by business capabilities rather than technology – when defining the structure, ensure teams can work as autonomous and independent as possible.
2. Upskill business analysts to product owners.
3. Organize development output by agile principles using SCRUM or KANBAN.
4. Successively move QA into development teams – a separate QA team can take responsibility for automation of regression, security and performance tests, but should not be involved into day-to-day testing activities.
5. Successively augment development teams with job scheduling know-how.
6. Introduce automation tools to ease the promotion (transport) process for changes into upper environments and, ultimately, production.
7. Add unit test and static code analysis (e.g. for security and performance checks).
8. Automate regression, security and performance tests.
9. Focus responsibilities of the SAP platform team on managing the platform (monitoring, patching, upgrading, environment provisioning and refresh, performance testing) and leave the transport of changes (“deployments”) to development teams.
10. Use automation tools for environment clone and refresh.
11. Move the SAP platform team towards KANBAN.
While these steps are similar to agile DevOps transformations in other technology contexts, the specific automation tool chain is very different.

Instead of using Git, Jenkins, Ansible or Chef, you are dealing with tools from SAP and third-party vendors for code inspection and analysis, transport management, test automation, environment refresh, performance testing and more. This requires skills that can be hard to find. In addition, different tools may be used to cover different SAP platforms (the “ABAP-Stack” and the “Java-Stack”).

There is another peculiarity for SAP: Although new technology supports fast cloning of isolated SAP environments and their continuous refresh with current data, this is constrained to certain infrastructure. The costs and lead times to provision and manage a single SAP environment remain relatively high.

Therefore, several teams will often have to share a single SAP environment for development, testing or release purposes. This requires coordination and sets limits to continuous delivery. In the best case, daily production deployments can be achieved by opening the pre-production environment for transport and application tests over a couple of hours, then locking it for another couple of hours for regression, performance and security testing (including batch) and production delivery.

In summary, many principles of DevOps can be applied to SAP application delivery, if peculiarities are taken into account.

While continuous deployment for SAP environments remains an ambition, daily releases can be achieved today.

DevOps Dozen Awards! Automic is currently nominated for a DevOps Dozen Award in the Release Management and Continuous Delivery categories. Vote for us today!
August 4, 2015
How ADLM Gobbles Up DevOps

The Next Evolution of Software Delivery Automation

In the 1980’s and 90’s, the business software landscape was dominated by a diverse list of cutting-edge companies such as Best Software, i2, Brock Control Systems, Mapics, Ross Systems, Infinium, FBO Systems, Manugistics and MSA (of course I could go on and on). Now long gone, these and hundreds more really great companies have been gobbled up or rendered obsolete by the rising class of ERP giants. In this article I’ll explain why history will repeat itself leading to the extinction of most DevOps tools as leading ADLM platforms continue to assert their dominance across the diverse software development and delivery automation ecosystem.

History Informs Our Future and The Evolution of ERP

You already know that for the past twenty years, most companies have leveraged some form of ERP system to manage virtually every core business processes. One benefit of this tightly integrated solution is a powerful inter-functional data flow that enables corporate agility and provides the highest level of visibility. This super-integrated architectural model has become the standard adopted by virtually every enterprise around the globe. What you may not know is today’s “ERP model” is the result of four distinct evolutionary generations that I believe help predict the next major evolution of automated software delivery.

Phase 1 – Automation
Enterprise Information Systems (EIS) – In the 1960’s, early automation systems were developed to support important individual business functions such as general ledger, inventory management, billing, payroll, etc. These systems were architected completely independently of each other and added little value to the enterprise beyond their narrow scope.

Phase 2 – Core Data Model
Manufacturing Resource Planning (MRP) – Then in the 1970’s, the idea of a master production schedule was devised so that a few of these isolated systems could gain greater visibility into future inventory and product requirements of the organization. The big idea behind the master production schedule was the creation of an open data model that could be leveraged by other systems impacted by the manufacturing production schedule.

Phase 3 – Expansion
Manufacturing Resource Planning II (MRP II) – In the 1980’s, software vendors began to build and sell off-the-shelf packages that promised “best of breed” process design. These solutions provided tightly integrated versions of the key manufacturing processes required to produce products.

Phase 4 – Business Process Domination
Enterprise Resource Planning (ERP) – Finally, in the 1990’s, business software vendors expanded well beyond the manufacturing scope by creating highly-integrated solutions that now cover just about every standard business function imaginable. These systems leverage a unified data model to dramatically improve visibility, consistency, accuracy and planning capabilities enterprise-wide.

What Does ERP Have To Do With Automated Software Delivery?

The evolution of ERP has taught us how natural pressures forced the creation of a unified and comprehensive “business data model” spanning the entire enterprise. Those software vendors with enough influence to dictate that data model were the ultimate winners in the ERP space.

In the very first generation of ERP (EIS), software was leveraged to deliver a high degree of automation to many business processes that were previously manual. Over time, it became clear that these newly automated processes were interconnected and the evolution towards a tightly integrated and unified data model was underway. The business objective that fueled each successive generation outlined above was the need to design more efficient business processes that increased organizational visibility and agility.

As Marc Andreessen famously said in 2011, “…more and more major businesses and industries are being run on software and delivered as online services—from movies to agriculture to national defense.” (Why Software Is Eating The World). That statement resonates even stronger four years later. Today, virtually every corporate organization is seeing the familiar pressure to deliver software more efficiently and reliably. If history is indeed our guide, any highly fragmented and/or isolated process required to deliver incremental software change will face mounting pressure to be merged into an integrated end-to-end enterprise-grade platform that can deliver improved cross-functional visibility with even greater efficiency.

Why Application Delivery Lifecycle Management Will Win

In my view, there are really only two broad solution categories in the realm of automated software design and delivery – Application Development Lifecycle Management (ADLM) and the catch-all term DevOps (which I’ve hijacked here to describe any other type of process automation that assists the software delivery process).

DevOps tools are often narrow point solutions that have sprouted from open source projects, in-house development or commercial vendors. Organizations rely heavily upon a diverse collection of these DevOps tools to help document, validate and automate a steady flow of software change along its path to end-users.

Here’s the problem DevOps tools are beginning to face: Like the EIS systems of the 60’s, fragmented DevOps tools have little or no visibility into the overall end-to-end process; however, they do generate lots of important data that is often locked away and isolated. This isolation creates a clear barrier to efficiency, visibility and agility across the software delivery process. Because of the limited function each individual DevOps tool performs, none have the gravitational pull required to define the larger data model. As comprehensive enterprise software delivery platforms emerge elsewhere, standard DevOps tools will face ever-increasing pressure to fold inside them.

Currently, Application Development Lifecycle Management solutions provide a platform to manage development projects, team resources and all manner of development activities. ADLM platforms also contain the “master production schedule” for every development initiative – past, present and future. The data contained within ADLM is now at the core of a quickly emerging software delivery data model and leading ADLM vendors are expanding their footprint pwell beyond traditional use cases. When it comes to ownership of this software delivery data model, I see no other solution category across the entire ecosystem with enough enterprise clout to pose a serious challenge to leading ADLM vendors.

Unified Software Delivery Platforms Are Already Emerging

Each of the five current ADLM leaders (according to Gartner’s most recent Magic Quadrant) are now racing to bring to market an enterprise software delivery platform that integrates many key DevOps capabilities.

Here’s my two cents on each…

The Giants in the space – IBM and Microsoft both have plenty of muscle and IP today. Clearly both are moving down the path toward a comprehensive software delivery platform. IBM acquired DevOps vender urban{code} several years ago and is hard at work building their developerWorks platform. Seemingly everyday, Microsoft is adding some kind of DevOps capability into its Visual Studio product suite. Still, I don’t see either vendor gaining much traction outside of their traditional (albeit very large) customer base. Perhaps more importantly, neither seem to have bonafide credentials within the super-influential agile development community and I believe this kind of street-cred (at least for now) is a must-have to dominate this space.

Atlassian does enjoy wide support among the agile community and no doubt has the broadest adoption footprint of any of the current ADLM leaders. Atlassian is in a strong position to mount a serious threat. However, Atlassian’s core product (JIRA), is widely believed to lack heavy-weight depth in the ADLM feature spectrum and it is often implemented as a departmental or “team tool”. They’ll have to develop deeper strategic planning and multi-team project capabilities to beat the rivals.

This May, software giant Computer Associates announced a definitive agreement to purchase ADLM heavyweight Rally and its agile development platform. In its announcement, CA said it intends to leverage Rally’s capabilities to “complement and expand CA’s strengths in the areas of DevOps and cloud management”. With the crucial addition of Rally, CA is now in a strong position to assemble its diverse capabilities into a single unified and enterprise-caliber software delivery platform. Now… can they seamlessly integrate all of the pieces-parts into a cohesive solution with a unified data model? If so, how long will it take?

Finally, I believe VersionOne may have a slight edge over the other ADLM vendors in the race toward a unified software delivery platform. I may be a bit biased because of my direct involvement in a joint project currently underway – none the less, here are four reasons why they will absolutely be a dominant force to recon with:

Vision: Robert Holler, VersionOne CEO, is clearly buying into the “enterprise software delivery platform” vision. He and his team have a well thought out strategy and they are actively executing against that strategy.
DevOps Automation: VersionOne has partnered with ClearCode Labs and both teams have been hard at work integrating ClearCode’s Continuous Delivery Automation framework into the VersionOne core product. This integration provides VersionOne the ability to orchestrate virtually any DevOps tool or platform and (just as importantly) incorporate all related data across VersionOne’s product suite to fee its quickly expanding data model.
JIRA Integration: VersionOne has just announced a tight integration into the JIRA platform. This integration will give them the ability to fold fragmented JIRA installations across the enterprise into the unified VersionOne platform providing a more strategic and enterprise-grade solution.
Availability: VersionOne’s automated delivery platform is available now and they are demonstrating their comprehensive solution to the eager agile community this week at the sold-out Agile2015 conference in Washington, DC.

Summary
The top 5 ADLM vendors are already well on their way toward developing enterprise-grade software delivery platforms that will consume many of the current “DevOps” automation solutions. Soon, development organizations will benefit from a comprehensive platform that can deliver increased efficiency, visibility and agility when compared to the heterogeneous solutions that have been cobbled together today.

About the Author/Dennis Ehle

Dennis Ehle, is a pioneer and thought leader in continuous delivery automation and agile delivery methodologies. Dennis is passionate about helping agile teams dramatically reduce the transaction cost associated with delivering incremental change. His company, ClearCode Labs, does just that by helping organizations continuously deliver high-quality software releases more frequently merging proven methodologies with empowering tools and technology. Twitter: @DennisEhle

August 4, 2015
Essential Ingredients for Continuous Deployment

To build the system to automate the build and deployment (continuous deployment) process there are few main areas which need to be covered. Following are our prerequisites.

– Source code
– Builds System
– Deployments
– QA Automation

To describe above here are the main ingredients
– Configuration Management
– continuous integration/ Automated Build
– Automated Deployment
– Automated testing

Configuration management
Very fist step to align your development practice is to select the right source control tool. Once you have the tool then you need to think of branching policy to make sure the starting point of the workflow is well defined and accepted by developers during their development phase.
There could be lot of process around merging, branching code review etc which needs to be address and policy driven. Some key points could be
– Developments and release branch should be separate.
– Each change goes qualifies for next stage only if reviewed.
– Code quality should not go down.

Following are a few options
Source control tool
– SVN (Free)
– CVS (Free)
– Git (Free)
-Perforce (Paid)
-Clearcase (Paid)

Build system
Team needs to have robust and automated build system which can take care of all our needs (build, analysis, deployment etc).
– Need to have automated build system which can keep building the source code from various branches to make sure we catch the broken piece fast.
– Build can run the unit test and provide report
– Build can check the health of the code by running findbugs and checkstyle (if any).

Above checks can verify the heath and quality of the code (static analysis).
Nothing should go beyond this point if any of the above is not matching with our criteria.

Till this point we have reached to qualify the minimum criteria for continuous Integration.
Here are few tools which comes under Build system
Build Tools
– Ant
– Maven
– Gradle
– Make
CI tool
– Jenkins
– Cruise control
– Luntbuild
-Bamboo
Repository management
– Nexus
– Artifactory
(Above are not open source but they are free to use with limited features).
Code quality check
– Findbugs
– jacoco
– cobertura

**Deployments **

Deployments are the next big bottleneck for continuous deployment.
We need to have a deployment tool to move ahead,
– configuration management tool like puppet or chef
– Private cloud management (Openstack)
– Third party tool like xl deploy, Udeploy, Opsworks etc..

Deployment strategy can be chosen on the basis of our need and the tool we are using, most of the plugins are available in jenkins (or bamboo) to make tight integration of build and deployment.

(At this stage I am not considering the dynamic provisioning of hardware in this section).

Automated testing

Once deployment phase is over QA regression suite can be triggered via jenkins (or any other tool) and entire reports can be published to central QA metrics board.
This this point we can even use jacoco/cobertura to provide functional code coverage(for Java code) to demonstrate how much % of code QA regression/automation is covering and what is the level of automation.
Similar way job can be setup to raise jira/bug ticket to file issues found during any of the above step.

Finally we can have last step which can perform the final release build and push to staging /production in similar fashion.

Resource Provisioning

Developers/QA can just get the hardware either from infra team or if auto provisioning is enabled they can setup the environment and use the jenkins to perform build+deployment+QA
If we talk about automatic scaling of environment, we can use AWS for same.
Jenkins has AWS plugin to launch instance, This can be setup to launch instance dynamically and run the application and perform the QA Automation. Once need is over aws instance can be terminated automatically.

Reference-1

Reference-2

August 3, 2015
DevOps in a Hybrid Model
As enterprises have begun to demystify the value proposition around cloud adoption, several best practices, optimization techniques and delivery models have been described in the technical community around cloud enablement and large scale adoption. While the debate between choosing private cloud vs. public cloud model has been longstanding, enterprises have been looking to mitigate the level of software disruption caused by such large transformation. A Hybrid model is thus a very appealing prospect since enterprises can still technically control the security policies and governance models, while also taking advantage of capacity bursting and other benefits of the cloud.

As with most technologies, adopting a hybrid cloud solution doesn’t buy you a free lunch either. There are several complications that are being discovered as enterprises mature into this model. One of the biggest challenges is to keep the software development lifecycle in sync across multiple target environments that are the basic backbone of a hybrid cloud model. Thus, having a robust DevOps solution is one of the key necessities for making a hybrid model work.

The Business Case for a Hybrid Cloud Model

In today’s complex business world, enterprises either by choice, or by circumstances, are choosing a hybrid model of IT solution. Enterprises are best served by implementing a “best of breed” solution, which means a combination of several domain specific SaaS (Software as a Service) and PaaS (Platform as a Service) solutions. This also eliminates any risks of a “vendor-lock in”.

So by design, a hybrid model will have a combination of the following characteristics:
- Multiple platforms within an enterprise architecture
- Both traditional as well as virtual platforms
- Domain specific SaaS/PaaS solutions, typically in a multi-vendor scenario
- Lack of standardization across cloud platforms (hypervisors, hardware types supported, so on and so forth)
Now, the challenge is to make IT delivery work seamlessly around these characteristics. This is only possible by a high degree of automation in the delivery process, operations process as well as the business planning processes. Those are the very elements a DevOps platform attempts to solve.

The Challenges of Cloud Adoption

Let’s take a quick look at some of the core existing IT capabilities built into a large enterprise.
- Security model
One of the biggest challenges towards cloud adoption is the notion of extended or revamping existing enterprise security model for cloud (and as a consequence, into DevOps solutions). How can one make sure the same security policies for data classifications and user management are applied especially on a public cloud model? Will existing audit processes suffice in a cloud model? Can existing security policies be leveraged as-is or with minor modifications and still be able to take advantage of a cloud model? These challenges will drive enterprises into a hybrid solution.
- Support and Governance models
Many enterprises have built up a history and matured best practices around IT governance and support structure (think ITIL). The cloud model in many ways disrupts those practices. One of the ways to mitigate the disruption is to adopt some form of a hybrid model so some best practices and core support capabilities can still be leveraged while taking advantage of the cloud.

One of the emerging business models for enterprises is to avail a more “managed services” model so they can focus more on their core business capabilities rather than on IT. In a managed services model, one or more service providers will be responsible for portions of the overall cloud solution while the enterprise would retain responsibilities over centralized capabilities like security. So the “hybrid” nature is not just limited to physical environments, but also around support models.

Implementing a DevOps strategy could help mitigate a lot of these issues.

Hybrid DevOps Scenario

A typical SDLC (Software Development Life cycle) DevOps looks like the following.

Figure 1 – A Typical DevOps Pipeline

The aforementioned actors in figure 1 may or may not actually be “people” since in a true DevOps world, those actors are replaced with some degree of automation. But it sets the context on what types of roles & responsibilities are expected in a typical pipeline.

Now, imagine this pipeline that is multi-folds, across several application or components. This is the real nightmare of deployment and release management issues that one has to account for since in a hybrid model. Each of these applications/components may technically reside in a totally different environment developed and managed by totally separate groups with different service level agreements.

A centralized DevOps platform, standardized across these application environment boundaries (for the most part) to provide a single pane of control over code, release and deployment management is what is required for a successful adoption of hybrid solutions.

Here is an indicative centralized architecture to solve the deployment and release management issue described above:

Figure 2 – DevOps Platform Architecture Example

The components listed in figure 2 is purely indicative and shouldn’t be misconstrued as any product marketing or messaging. The main challenge will be to standardize components to the extent possible (this is easier said than done) and stitch together a meaningful platform that works within an enterprise (based on existing tooling & processes, skills, funding etc.). Bear in mind I’m merely touching the surface in this article with regards to a DevOps platform, and limiting this to a tooling discussion. Bigger challenges do exist in the form of process and organizational transformation, which are also necessary to make a DevOps implementation successful. I’ll pen those topics in a separate article 🙂

Summary

DevOps for Hybrid cloud implementations will soon become the de facto model for process transformation in large enterprises. There is simply no other way to go about such large transformation in a purist way. There is a wide verity of use cases for a hybrid model, and most enterprise situations would fall under one or more of those categories. A robust DevOps platform is absolutely critical to making a Hybrid cloud adoption successful.

About the Author/ Sunil Joshi

Sunil is an Executive Architect with IBM. He is currently the Application Development & Innovation (AD&I) DevOps offering owner for Global Business Services. He is IBM Senior Certified and Open group certified Distinguished Architect. His specialization are in the areas of hybrid cloud solutions, platform as a service and DevOps strategy. He’s created complex hybrid solutions on cloud and DevOps for large enterprises. Sunil has authored several IBM Redbooks, blogs and also a co-author of the IBM Cloud Computing Reference Architecture (CCRA).

Sunil is passionate about international music, multi-cultural cuisine, active sports and mentoring school & college students on career path and technology.

Contact Sunil on
July 31, 2015