devopsy.kinsta.cloud

Category: Blogs

DevOps Leadership Series: Gov Does DevOps (Part 2)

During my second day at DevOpsDays DC, I had the opportunity to catch up with a couple more industry thought leaders. First up, John Willis, (@botchagalupe) DevOps Days core organizer and co-author the upcoming “DevOps Cookbook”.

In this episode of DevOps Leadership Series, John highlights his first introduction to DevOps and the journey it has taken him on, and what about the current state of DevOps makes him excited. He says DevOps is really happening everywhere. Today large organizations and CIO’s of government agencies are now experimenting with DevOps at all stages. He references legacy companies that do not necessarily have “aggressive IT”, and how they now want to share their DevOps experiences.

https://www.youtube.com/watch?v=jU1O2bCU_hc

After that, I had the chance to chat with Leon Fayer, (@papa_fire) Vice President at OmniTI, following his DevOps Days DC presentation. Leon illustrates the things he found most interesting from the DevOpsDays Conference. He noted that a lot of people have started to pay attention to the monetary strategies, and how to monitor from top-to-bottom in a DevOps fashion. Leon was happy to see that people are now more generally interested in how to generate and collect data more accurately to solve their problems.

https://www.youtube.com/watch?v=kx4Y3P3Njro

June 17, 2015
Do Regulatory Auditors Hate Continuous Delivery?

In a world where superior customer engagement and other market imperatives depend on having the right code in production sooner rather than later, continuous software delivery is a competitive necessity

But you can’t get code into production unless you properly test it. And you can’t properly test code without appropriate test data.

And that’s where the regulators come in.

Solving for Personally Identifiable Information
Production data is rife with personally identifiable information (PII) and other compliance-sensitive content. That data is typically well-protected by various security measures and access controls in the production environment itself. But putting that data (or, more precisely, a sample set of that data) in a test environment exposes it to significant security and compliance risks.

For one thing, test/QA engineers aren’t account managers or clinicians. So, from a regulatory perspective, there are certain kinds of data they shouldn’t be allowed to see. For another, test/QA environments are typically not secured in the same way as production. Worse yet, test/QA is often performed by contractors — and may even occur offshore. This creates unacceptable security risks and throws up big red flags for regulatory auditors.

To protect PII while enabling thorough and effective testing of new code, companies use synthetic data and other masking/obfuscation techniques. That’s what regulators demand and expect.

Yet another roadblock?
Unfortunately, most companies generate test data on a very ad hoc basis. At some point in each project, someone realizes they’re going to need some test data — and they set about creating it. But without a global mechanism for determining what needs to be masked and actually performing that masking, the undertaking becomes iffy and time-consuming. Not all the data that needs to be masked gets masked. A couple of developers may independently decide that the best thing to do is write a one-off script. And if the script doesn’t work well, the team may just cross their fingers and opt to forego masking altogether.

Hence the title of this blog. Of course regulators don’t actually hate continuous delivery. Most of them probably don’t even know what it is. But their mission — enforcing the diligent protection of sensitive data — can be a serious obstacle to continuous delivery for companies that don’t get their act together when it comes to test data management.

The solution: Continuous compliant delivery
The only way to reconcile the requirements of regulatory auditors with the need of the business for continuous delivery is to adopt a well-managed enterprise-wide mechanism for test data creation. Such a mechanism accelerates every code promotion workflow across every project, while also providing an auditable means of enforcing PII masking/obfuscation policies. Enterprise test data management also reduces devops costs by eliminating the need to re-invent the data-masking wheel on every project.

So, no, regulators don’t hate continuous delivery. But they could — if companies sacrifice compliance for speed. The best way to avoid that sacrifice is to embrace enterprise-class masking discipline for test data.

About the Author/Aruna Ravichandran

Aruna Ravichandran is vice president of DevOps Solution Marketing and Management at CA Technologies, responsible for cross-portfolio solution marketing for DevOps across all product lines. Before joining CA Technologies, Aruna held several management positions in product marketing and engineering at Juniper Networks and HP. Aruna earned an MBA and master’s degree in Computer Engineering from Santa Clara University and holds a bachelor of science degree in computer science from Bangalore Institute of Technology. Ravichandran is on twitter @aruna13 and on Linked in

June 17, 2015
Continuous Delivery and The Proof of Quality Assurance

Quality Assurance is critical in Continuous Delivery. Every change we commit to the common source repository is a release candidate. That’s why we test our code before we release it. We find out whether objectives has been met or not. Unfortunately, testing is a discipline that many try to avoid and it’s one of the main reasons for having specialized testers in the teams who know how to break the code best. But shouldn’t be the developers of the code know best how to break it?

(more…)

June 16, 2015
GE Capital Releases Mobile App Twice Weekly, Automating Compliance at Velocity, Thanks to DevOps/CD/CI

GE Capital offers equipment leases, mobile apps, and financial products for corporate customers. GE Capital’s Matt Merchant, Global Managing Director—DevOps, spoke with DevOps.com about development initiatives and processes at the financial services giant.

According to Merchant, his team leads the charge in driving

(A) the DevOps cultural transformation throughout IT,
(B) the adoption and expansion of DevOps tooling, and
(C) the change in governing structures in order to operate at a much higher velocity while continuing to meet regulatory requirements.

That last mouthful speaks to a key pain point that GE Capital’s DevOps processes relieve: how to speed development while remaining compliant. GE Capital does it by automating security and compliance. You’ll read how shortly, but first, the mobile app.

The Mobile App for Fleet Drivers

GE Capital provides vehicle fleet leases globally for enterprises with delivery drivers, sales people, and others who travel on company business. The mobile apps that GE Capital provides as part of the leasing platform are many and vary from country to country. For this discussion, Merchant takes an app deployed in Sweden as the example.

The web-based app enables drivers to locate nearby gas stations, tire shops, repair shops, and windshield replacement services. Drivers also use the app to download routes, to collect refueling, mileage, and oil change / service data, and to log their hours. The app creates a number of efficiencies.

Drivers used to get shop data from a physical driver’s manual that they had to pore through to find the closest service provider. “The instant you hand them that catalog it’s out of date. Now they get real-time updates on their phones,” says Merchant. They also used to hand log their hours with pad and pen and then return to the office at day’s end to type this data into a computer. “Now they do it in real-time, and are done at the end of the day,” says Merchant.

Mobile App Development Before & After DevOps

In GE Capital’s old Waterfall mobile app development process, releases took six to eight weeks. Challenges included an inconsistent code repository, manual scripting and testing, and sitting before change, architectural, and security review boards every time developers evolved the code. “Because all that is so painful, you don’t want to repeat the process very often, and so you bundle more changes into a single release, which increases the risk of error,” says Merchant.

With DevOps, GE Capital added Chef configuration management and created an “over-arching umbrella” development process that the company calls ArchOps. ArchOps covers CD/CI and virtualized cloud platform management while leveraging a Lean Six Sigma approach to swift Agile application development, explains Merchant. This enables fast iterations, consistent code, automated testing and, using Chef, fast VM spin-ups. And there are no more change management reviews.

Achieving “Compliance at Velocity”

To achieve what Chef calls “Compliance at Velocity” and deliver compliant, auditable code at a high speed, GE Capital reconsidered its toll gate processes and artifacts. “We applied a value stream map to our governance model, removing 60-percent of our toll gates and 70-percent of our artifacts,” says Merchant. This shrunk mobile app release delivery to days.

GE Capital automated compliance by automating test output analysis. When the analysis shows that a portion of the code failed a test, the code goes back to development, which must fix it before retesting, further analysis, and eventual promotion to production. “That removed the need for a change advisory board,” says Merchant.

And when an auditor requests test results, release dates, or copies of specific code releases, Merchant simply pulls the given code and all the test results, design artifacts, and governance data that are bundled with it from the Git repository and hands it over.

Once Through Development, Please

Now with a DevOps mindset in command at GE Capital, all developers use the Eclipse IDE, which transfers the code directly into the company’s Git repository. Once the code enters Git, developers can run jobs using tools such as Jenkins, do Fortify application scanning, and perform UI testing using GE Capital’s home grown tool called Rainbow.

Once the code passes all tests, the system sends it into a pre-production staging environment. “GE Capital built the staging environment using Chef, which calls out to servers, spins up the environment, and places the code inside it,” says Merchant. User acceptance and remaining tests happen there. Then the Chef code that created the internal environment creates the production environment, ensuring application stability upon release.

June 16, 2015
2015 State of the Software Supply Chain Report
In April of this year, I embarked on a six-week journey diving deep into an analysis of the world’s software supply chains. I evaluated the practices of 106,000 organizations, the 100,000+ suppliers they relied on, and the billions of software components that fueled their agile, continuous delivery and DevOps practices.

The facts I discovered and share in the 2015 State of the Software Supply Chain Report: Hidden Speed Bumps on the Road to Continuous, (pre-register for the full release: Tuesday, June 16th) fundamentally changed the way I thought about software (and about DevOps).

The volume and velocity of consumption, the variety of parts and suppliers, and the impact on innovation and quality astounded me. Early reviewers of the report including Gene Kim (co-author of the Phoenix project), Gareth Rushgrove (Puppet Labs and DevOps Weekly newsletter), Nick Galbreath (Signal Sciences), and Nigel Simpson (Fortune 100 Entertainment and Media company) agreed.

My aim for this research is not simply to present facts about the global software ecosystem. I’m aiming to point a spotlight on software supply chain best practices within across a variety of industries that could be used as new benchmarks for software supply chain automation. Similar to manufacturing of auto, pharmaceutical, healthcare, or defense systems, the effective management of supply chains will create winners and losers. I’ll share evidence that the best, high-performance software development organizations are benefiting from:
- Working with fewer and better suppliers
- Relying on the highest quality supplies from those suppliers
- Maintaining traceability and visibility throughout the software supply chain for prompt and agile recall
Key Points from the Study

In the best organizations, the research revealed developer net productivity increasing by up to 40%. Just imagine applying that time to more innovation, rather than to rework and maintenance efforts.

At the same time, the report touches on inefficiencies and complexities that are creating a huge drag on the velocity software development teams are aiming to achieve. A lack of discipline, focus, and visibility around the software supply chain has resulted in mountains of technical debt, unnecessarily context switching, and outdated sourcing methods that wasted over 3.3 million build days last year alone.

Hightlight: Automation Across the Software Supply Chain

The other key insight from this research is the clear need for further automation across software supply chains. With individual organizations consuming hundreds of thousands and sometimes millions of software components annually, it became obvious that waterfall-centric approaches to identifying the most functional parts, checking quality, validating appropriate licenses, or evaluating security vulnerabilities could not keep pace. Sourcing practices that regularly go unchecked have also resulted in the use of severely outdated software components, even numerous versions of the same component part.

Download the Full Report

Over the next few weeks, I will publish excerpts of findings and best practices identified in the 2015 State of the Software Supply Chain Report and invite you to read along. Or you can access the full report now through this link.
June 12, 2015
DevOps and QA: What’s the real cost of quality?

As continuous delivery and DevOps speeds the cadence and volume of development, quality assurance professionals and the groups they work in must find ways to transform their roles. Because in order for businesses to truly get value out of the applications they’re delivering through DevOps automation and iteration, they don’t just need to attain speed from the process but also quality. And if they don’t, the costs to the business may be more than people think.

“Software is transforming from a business enabler to a business differentiator,” says Wayne Ariola, chief strategy officer for Parasoft. “The costs and risks associated with failure has shifted dramatically over the years.”

Ariola recently presented some cost figures at the STAREAST conference to show how dramatic the costs of quality failures can be. Over the past few years he’s made it a pet project to follow headline news on notable public company software failures and track them to stock prices to do a quick back-of-the-napkin equity analysis. According to his analysis, the typical net decline in the value of a company associated with the announcement of a failure is $2.3 billion. These types of events can be anything from something like American Airlines’ announcement of that a glitch in iPad software used by pilots in the cockpit grounded a number of flights this spring to Chrysler’s admission that software incompatibilities between the electric control unit and the battery pack control module in Fiat electric cars could suddenly shut cars down. Ariola says that in 2015, the average net decline of publicly-traded companies that made headline news for software failures was 4.18 percent.

“When you’re about to release something, and it’s about to go out the door, do you hang out with the developers and say, ‘Hey, if this fails our stock price is going to decline by 4.18 percent? ‘” he says. “These are conversations we’ve got to start having (as QA professionals).”

Market valuation is only one indicator of the costs, too. There’s also the more difficult to track issue of customer churn, which is only going to increase as the software-driven world evolves and newer generations’ expectations of software quality ramp up. Whereas Boomers, GenX and GenY consumers simply got used to things like the blue screen of death and troubleshooting wonky software, Millennials have been less desensitized to quality issues. What’s more, they have more alternatives than ever, so when problems come up, they’re more likely to ‘nuke’ a product or service and choose a competitor—say switching between one app to another on their phone—rather than even bothering with failed software.

“Their tolerance for faulty software is much, much lower than ours is, and it’s going to change the way people consume software,” Ariola says. “We need to be aware of that as a team of testers, because we need to drive the agenda much differently than we are today.”

One of the key elements of that is switching the mentality of testing as a time-boxed event into a set of practices that “infiltrate” development and deployment processes and “escalate the process maturity.”

“You’ve got to go from manual to continuous tests. You’ve got to get beyond this idea of just pure automation, and strategically look at this application so you understand the risks associated with it today, and the concept’s going to change,” Ariola says. “So with this speed comes a totally different paradigm.”

In the past, the question that gated releases was whether testing was done or not.

“I’m going to propose that the real question is does the release candidate have an acceptable level of risk?” he says. “I have to understand the business expectations associated with the risk of that application, and if it’s in an acceptable boundary, we’re going for it, we’re releasing it, and hopefully we can roll back really fast if we need to.”

June 11, 2015
ProfitBricks shakes things up with new Docker hosting platform

Organizations and developers that want to leverage Docker containers have a variety of cloud platforms to choose from. Rackspace, Amazon, Azure and others all support Docker containers. There’s also a relative newcomer in the mix looking to shake things up.

ProfitBricks is an infrastructure-as-a-service (IaaS) company that believes it has a unique value proposition for companies that want to take advantage of cloud infrastructures and Docker container technology. ProfitBricks was co-founded by its CEO, Achim Weiss, and its CMO, Andreas Gauger. The two previously founded the Web hosting service 1&1 Internet.

Weiss and Gauger launched ProfitBricks four years ago to solve a perceived problem. They saw the value of cloud infrastructure and cloud services, but felt that the existing platforms suffered in the areas of performance, flexibility, usability and cost. So ProfitBricks was born.

ProfitBricks claims to have more speed and provide better value compared with rival cloud platforms. It calls its platform Cloud Computing 2.0.

ProfitBricks Docker Hosting

Docker has become a household name virtually overnight—quickly establishing itself as the de facto standard for container technology. David Bartletti, a principal analyst for Forrester, recently wrote in a research report, “Docker helps developers package code for quick assembly and hand-off to quality assurance, testing, and production release teams with greater confidence that their app will run consistently as it moves through the release process…The Docker open source community is growing quickly, with strong representation from nearly every major cloud vendor…As Docker containers proliferate, developers and operations teams alike will demand management tools that help organize and orchestrate hundreds or thousands of containers.”

With so much momentum behind Docker and container technologies, it makes sense for a cloud infrastructure provider to integrate Docker into its ecosystem. Last week ProfitBricks announced its new Docker hosting platform. The platform features dedicated resources, redundant block storage, and 80 Gbps networking. It allows developers to choose images, link containers, and expose ports. The free tier offers customers 2,500 CPU Core hours as part of the Early Access Preview with up to 4 CPU cores, 8 GB of RAM, and 100 GB of storage.

“When we first embarked on our Docker product architecture, we knew that it would not only need to meet the standards of existing Docker platforms, but also incorporate new, industry-leading features that addressed existing problems encountered with Docker,” said Achim Weiss, co-founder and CEO of ProfitBricks, in a press release. “Docker has been taking the DevOps world by storm since its widespread introduction to the community in 2014, and our platform will combine its impressive features with our flexible and painless cloud computing infrastructure.”

ProfitBricks says that one thing that sets its Docker hosting apart from the competition is autoscaling. The ProfitBricks cloud infrastructure will ensure that resources are automatically managed to meet the demands of the Docker containers running on it. Developers don’t need to rely on standalone applications or custom scripts to scale resources because those capabilities are integrated seamlessly into ProfitBricks.

ProfitBricks claims to be the best value in cloud computing. It offers a bold promise for the price / performance ratio on its site. “We guarantee that any workload deployed on ProfitBricks will cost less than the same workload running at the same performance level on the IaaS platforms of Amazon, Google or Microsoft.”

You can check out the new ProfitBricks Docker Hosting for yourself. Visit this link for an Early Access Invite and find out if ProfitBricks can really deliver.

June 11, 2015
DevOps Leadership Series: Monitoring Containers and Microservices

Trevor Parsons (@trevparsons) is a Co-Founder and Chief Scientist at Logentries, a leading SaaS-delivered log management and analytics service. I caught up with Trevor at the Velocity Conference in Santa Clara and asked him what themes were resonating with attendees this year.

For this episode of the DevOps Leadership Series, Trevor briefly illustrates some current problems with state of microservices and container-based architectures, while sharing a positive outlook for the future.

https://www.youtube.com/watch?v=zHRE4011WVk

Trevor says there’s been a lot of development in how to log and monitor containerized systems and he sees better support from Docker for logging, showing an increasing focus on this topic in 2015 and beyond. He anticipates the next year to have huge improvements in managing and monitoring these new systems and architectures.

Next up in the DevOps Leadership Series, I headed to London for the DevOps Connect: Rugged DevOps event, where we’ll discuss more about Rugged DevOps practices. In London, I was able to catch up with Helen Beal (Ranger4), Justin Arbuckle (Chef), and Gareth Rushgrove (Puppet Labs).

If you have missed any of the other videos from this series, you can find them here. (We’re up to 15 so far).

June 11, 2015
DevOps Leadership Series: Compliance, Testing, and Rugged

This past week, I headed to London for the Rugged DevOps event, where I had the chance to catch up with a few more industry thought leaders.

First, I caught up with Gareth Rushgrove from Puppet Labs, who also runs the DevOps Weekly newsletter. In this episode of the DevOps Leadership Series, Gareth explains why the importance of security tests within DevOps practices is going to be a big topic for years to come. Gareth anticipates that “In five years time we will look back on this and ask ‘why were we not always doing this’?”

https://www.youtube.com/watch?v=ktpkmGv5gX4

I then caught up with Helen Beal, Head of DevOps at Ranger4, where she discussed the importance of DevOps and security. While the two ultimately need to work hand and hand, she voiced concerns about DevOps practices sometimes circumventing controls that are essential to a business’s safety. She also said that DevOps supports security in a number of ways: from making things consistent to relying on more automation.

https://www.youtube.com/watch?v=W4Zs1SSjGFs

Finally, I caught up with Justin Arbuckle, Vice President, EMEA & Chief Enterprise Architect at Chef. He discusses key trends he expects to see in over the next 9-12 months regarding compliance and security. Justin explains that compliance as a core driver of DevOps is something that will start to shape our conversations over the next year, whereas the two have previously been seen as contradictory. He believes that we will see that high velocity organizations are able to improve compliance continuously, and tells us to look out for the changing role of the security officer in 2015.

https://www.youtube.com/watch?v=ojEOqRJey2o

Next up in the series, I head to the United States Capitol, Washington DC, for DevOps Days DC.

NOTE: If you have missed any of the other videos from this series, you can find them here. (We’re up to 15 so far).

June 10, 2015
Follow the Unicorn: Transform your Enterprise Applications

For those of us that work on Enterprise Systems such as the mainframe, the claims that the Unicorns⁽¹⁾ make, can be pretty astounding. Dozens and sometimes hundreds of releases a day from big companies like Netflix ⁽²⁾ and Etsy ⁽³⁾ are hard to comprehend. But look a little closer and you can see that the key difference is in the way changes are released to customers, not so much the total volume of changes. A traditional enterprise application may have thousands of changes made over the course of 3 to 6 months, but all of those changes are released at once. The Unicorns are typically able to release a very small number of changes at a time because they can develop, build, test, and deploy these changes very efficiently. Small batch releases to customers has significant value. It enables you to get feedback very quickly, and it is significantly easier to verify because the change is not mixed in with other changes.

So how can you transform your development environment to deliver small batch of changes? I like to tackle the problem like any other computer science problem, no doubt because of my background. Break the large problem into a set of smaller problems and then solve those smaller, less daunting problems:

Step 0: Start with a Proof of Concept

Pick a Proof of Concept (PoC) to begin with that is important, but not mission critical. The PoC can be used as a learning ground and hopefully as a template for subsequent PoC’s. Make sure you have a team that culturally is willing to transform. Often the hardest issues to tackle have nothing to do with technology, but are cultural.

Step 1: Understand your Application

You need to have a good understanding of what your application consists of. Not only does the source code make up the application, but what source files are used, what the middleware configuration might be, what scripts you may have to build and deploy the application, and what data files (e.g. flat files, databases) that you use are also important. Some of these components may be shared with other applications, so you need to understand those dependencies as well. Once you have a good handle on the application, you can then automate the application build and deployment process so that all members of the team can work efficiently.

Step 2: Understand your test cases

If you can build and deploy the application quickly but it still takes days or weeks to test, then testing prevents you from delivering to your clients. An important shift for testing needs to be from testing the entire application to testing the change. You need to focus on running the right tests to ensure the change is correct and causes no regressions. Tests that do not exercise the code change are irrelevant and can be skipped. For many existing applications, the test suites may consist of hundreds if not thousands of test cases. Many of those test cases may be duplicates or testing areas of the code that are not being changed.

Step 3: Understand your processes

How fast can you get your updates to your customer? If your bug fixes and enhancements have to sit on the shelf for days, weeks or months before the customer gets them, they will not be satisfied. Streamline not only the development processes but the operations processes in your organization.

To know more about the topic with the help of a real life case, watch the replay of a recent webcast which IBM in partnership with DevOps.com hosted with Canada Mortgage Housing Corporation’s Jeff Blackadar on how breaking down IT silos helped speed up mainframe development for them.

Also, do let me know your thoughts on the blog and on the topic.

¹ the born on the web technology teams

² http ://www.slideshare.net/jedberg/devops-at-netflix-reinvent

³ http ://www.slideshare.net/beamrider9/continuous-deployment-at-etsy-a-tale-of-two-approaches

About the Author/ Mike Fulton

A IBM Distinguished Engineer, CTO Enterprise DevOps at IBM, Mike is a software development professional specializing in compilers, debuggers, profilers and virtual machines. At IBM, Mike works towards understanding and influencing trends and directions in Enterprise IT delivering new technologies into the DevOps Portfolio. He also overlooks integration of the DevOps Portfolio into IBM and Enterprise Software Solutions. Mike is an IBM Master Inventor with several patents to his credit.

He is an avid blogger and regularly writes on his blog site – makingdevelopersliveseasier.wordpress.com

June 10, 2015