Tag: devops

When Good Code Goes Bad

Milk spoils. Iron rusts. And in software, good code goes bad. Yet the difference is, with the first two, you know the change has occurred. With software, those changes are not always obvious.

Your 5,100 Binaries Went Bad

There is no way to prevent software from “going bad”. As with all products, bugs and defects,are bound to happen at some point. No one and no code is immune from these issues. But who’s looking for the 5,100 software components in your organization that went bad last year (meaning new security vulnerabilities were discovered in them)? It’s all too likely, no one.

Earlier this year, I took a deep dive into the analysis of software supply chains that fuel high velocity development practices and IT operations. The analysis revealed that some of the largest development organizations were consuming an average of 240,000 open source components to expedite development, accelerate innovation, and meet time to release deadlines. While reaping huge benefits from this consumption, these same organizations were unintentionally growing their technical and security debt. How you ask?

Here is a summary of the their average debt:

Source: 2015 State of the Software Supply Chain Report

The analysis showed that over 15,000 (7.5%) of the open source components being consumed by these organizations in 2014 had known security vulnerabilities. Of those 15,000 components, an average of 66% (9,900) had known vulnerabilities dated 2013 or older. That means, they were known vulnerable components (“bad”) before they were downloaded.

The remaining 34% (approx. 5,100) might have actually been “good” components at the time they were downloaded by development teams from public open source repositories, but at some time during 2014 a new security vulnerability was discovered and a CVE identifier was assigned.

Even with the best controls in place to protect developers from downloading open source components with known vulnerabilities, the fact that new vulnerabilities are continuously discovered — about 50 new CVEs are announced daily — means that continuous integration and delivery practices also need to be complemented by continuous governance and visibility practices.

We’re Not Alone

Software development is not the only industry facing such issues of things good things going bad. I bet all of us have received notification from our car’s manufacturer that a certain part is being recalled. While the part was known to be safe when the manufacturer sourced it from suppliers, placed it on the car during assembly and shipped it, discoveries of defects are often found years after the vehicle has been on the road. The same scenario has been played over and over in the food, pharmaceutical, defense, toy, electronics, and other industries.

The difference is those other industries are known to have more mature supply chain practices where they can track and trace every part or ingredient ever used in every product they have shipped. And when a part goes bad, they can recall and repair that defective component.

In software, track and trace practices are still in the earliest stages of maturity. In a 2014 survey of 3,300 development professionals, 63% stated that their organizations have incomplete or no records of the components used in their applications. This lack of traceability means that even when good components go bad, development and IT operations teams have no way to quickly find the defective component and replace it with a better, safer version — dramatically extending mean times to repair, or in the case of no action, growing their technical and security debt. And when applications with those defective components are in the hands of customers, the ability to “recall” the impacted software or distribute newer versions of the application are hindered.

Did he recommend a “Bomb”?

A Bill of Materials (BOM; pronounced “bomb”) is used in traditional manufacturing supply chains to list the suppliers and parts used in a product. According to Wikipedia, “The concept of a BOM is well-established in traditional manufacturing as part of supply chain management. A manufacturer uses a BOM to track the parts it uses to create a product. If defects are later found in a specific part, the BOM makes it easy to locate affected products.”

In Part 2 of this blog, “The Software BOM Squad“, I will introduce the importance of automating the creation of a “software bill of materials“ (Software BOM) to take inventory of the third party and open source components used to build an application. When software components age more like milk than wine, a software bill of materials can help you quickly identify where your 5,100 “bad” component were used.

July 22, 2015
The Whole Nine Yards with DevOps and Agile Operations
Growing up as a boy in Manchester, UK, football (ok, soccer to US readers) was everything to me. I vividly remember being herded into the stadium stands on cold rainy Saturday afternoons to watch my team (#MCFC) win, draw or lose. Back then, football was for the die-hards – no frills or fancy tech, just hot tea and a suspect meat pie at half-time.

How things have changed. Now sporting events have undergone a complete transformation – all digitally-driven.

High Tech Sports

Take Levi Stadium, home of the San Francisco 49ers for example; considered by many to be the most high-tech stadium in the world. There’s too much tech to describe in detail, but you can be sure that all the Wi-Fi beacons, 400 miles of cable, photo voltaic energy and a funky mobile app have been designed with one thing in mind – enhancing the fan experience. So no ‘cattle-herding’ at a 49ers game – nope, a personalized digital experience that’ll find you your seat, food concessions, the closest bathroom — and of course the shortest beer line (#1 feature in my opinion).

Levi Stadium isn’t just another example of technical wizardry. It’s more an exercise in an organization deeply connecting with its customers through the delivery of highly experiential services. Understanding the needs, motivations and desires of their loyal fan base and servicing them accordingly.

It’s a pity this same thinking isn’t always applied in IT Operations – that is, deeply understanding what Ops ‘fans’ need if they’re to ensure a good business user and customer experience across new highly dynamic agile environments.

High Stakes IT Operations

Similar to the Manchester game day experience years ago, IT Operations has often done it tough. They’ve worked in isolated NOC’s and service centers, been placed on 24/7 call; working weekends and missing their kid’s birthdays. They have been served specialized ‘team merchandise’ in the shape of complex tools and processes – always counting on their loyalty to save the day when the next mega release comes over the wall. No great surprise then that Operations in many organizations is seen as uncommunicative and uncompromising – almost, well – tribal.

But IT tribalism doesn’t win matches in the software-driven economy game. Now, digital business realities means IT Operations must supplement its traditional fixation on reliability and resilience with a willingness to drive change – Operations must itself become more Agile. But how do you start? Thinking back to the innovation at Levi Stadium, here are three good ways.
1. Gaining Loyal Supporters – Development and Operations aren’t enemies, but like rival UK football team supporters the teams are often segregated. Dev and their tools to the left, Ops with theirs to the right. From a business perspective this is plain dumb – especially when it comes to getting the most value from technologies like application performance management. Rather than frantically monitor applications as problems manifest in production and then call Dev for help, wouldn’t it be better to incorporate operational intelligence much earlier in the software lifecycle? So when code is released, monitoring could immediately provide development with an early warning on operational issues. Conversely, Ops practitioners gain accurate and timely insights into production performance requirements and KPI’s – that’s not only a DevOps style win-win, it’s an agile like best practice to increase quality and avoid technical debt.
1. Filling the Business Stadium – The goal of Operations has always been keeping the business lights on, but Agile Operations now means making them shine brighter. Cloud, Microservices architecture and containers all give businesses the technical clout needed to support speed at scale, but it comes at a cost. Today’s sophisticated applications rely on specialized technologies, so it’s perhaps understandable that Operations chase the technology down the proverbial rabbit hole with more specialized tools. But the real truth is that any monitoring system which doesn’t deliver true business value by providing unified insight into how customers are actually experiencing IT services is of no use to anyone. Remember too that technology scale is about future proofing your business, so look for scalable monitoring to match it.
1. Delivering a Complete Experience – The digital experience for a 49ers fan doesn’t start and end on game day. It is delivered by a linked series of services – a digital journey comprising seat selection, ticketing, parking, entry, action replays, and player stats reviews – every one of which can positively or negatively impact the fans experience. Whatever your industry, IT Operations will be called upon to support similar services, meaning deep and uninterrupted visibility across the myriad of front and back technologies underpinning every customer interaction, engagement and transaction will become essential.
Successful digital transformation needs more DevOps and agile operations, agile development, faster release cycles and higher quality software. It’s time for IT Operations to stop fixing production problems and become an active participant in supporting these goals. Without Agile Operations, your role in the software-driven business game ferociously being played out will be like that meat pie I ate many years ago – suspect.
July 15, 2015
Top 5 Anti-ESB Arguments for DevOps Teams.

Unix and Mac. Developers are always in combat. My last verbal combat was a “discussions” around ESB and Microservices. They are fun because both supporters tend to have very strong opinions about what they think is best and both want to fight each other. I recently got involved in one of those discussion about ESB vs. Microservices and the ESB guy throw lots of arguments at me. “ESB is magic” got almost burnt into my mind until I started to investigate a bit: “How to fight ESB discussions in combat?”. Here are my top 5 findings that may help in your next verbal ESB combat situation.

(more…)

July 6, 2015
SaaS: Building Tools for the DevOps-Minded

DevOps, Continuous Delivery, Continuous Integration and Continuous Testing are no longer just methodologies and development strategies — they’re all a part of a growing economy within the tech industry.

Great Software-as-a-Service (SaaS) companies such as Chef, Docker and New Relic have done an amazing job of spotting the trends early and building their tools and services around a new wave of development processes. Chef and Docker alone have reportedly raised nearly $250 million, and New Relic went public in 2014.

In fact, this very website is another example of how impactful these new development practices are — as several specialized news outlets have been born out of these revolutionary development strategies. So, not only has the DevOps methodology helped developers create better products, faster; but the process has opened opportunities for technical experts to share their experiences on sites like this, e-books, webinars, conferences and any other way people communicate.

On devopsy.kinsta.cloud, you’ll see the best articles on the web discussing how DevOps is defined in different organizations, why Fortune-500 companies are shifting old practices toward CD and the best practices of CI from industry experts.

In the race to be the most DevOps-friendly solution — no matter the niche — hundreds of startups have been founded and funded, which has only enhanced the toolboxes of thousands of developers. You’d be hard-pressed to find a good SaaS company that doesn’t offer integration with Jenkins CI, TeamCity or Chef, and that’s because not only should a SaaS product solve an important problem, but it should have minimum disruption of your current development process.

With all that in mind, Continuous Testing is one of the most exciting spaces in technology today. Continuous Testing is a process that aims to synchronize testing and QA with dev and ops, which ultimately yields better products more quickly.

There are several factors that make Continuous Testing successful in a team’s software development lifecycle — such as open-source collaboration solutions, test automation APIs and tools to track the historic performance of applications .

With the release of Load Impact Version 3.0, the global performance and load testing SaaS company has announced its latest feature that fits right in with the DevOps mindset: Performance trending.

Performance trending allows users to plot runs of the same test over time. That gives companies a clear view into data that locates patterns of performance degradation or improvement in order to easily validate code and infrastructure changes.

In the spirit of Continuous Testing, DevOps, CD and CI — performance trending improves users’ ability to gain both a low-level and high-level view into the performance of their systems. And with the increased number of tests run by DevOps professionals, performance trending yields valuable information for companies concerned with their application performance — which should be everyone.

Between devopsy.kinsta.cloud and plenty of great conferences and meetups around the world, it’s safe to say this economy of DevOps-minded tools will only continue to grow in the coming years.

July 1, 2015
Rework is Choking Software

Rework is Hell

“Software may be eating the world, but rework is choking software”, tweeted John Jeremiah (@j_jeremiah). To shed more light on what is choking software, new data was released last week in the 2015 State of the Software Supply Chain Report.

In its discussion of application quality and integrity, the report revealed that the average application includes 106 open source components. It is clear that the use of these components has benefited development tremendously in helping to speed time to market and improve innovation. While the benefits are undeniable, development teams are also delivering applications that are “insecure by design”. Of the 106 components per application, the report’s analysis revealed an average of 24 (i.e., 23%) have known critical or severe security vulnerabilities. Those same apps also showed an average of 9 restrictive license types (e.g., GPL, AGPL, LGPL).

By electively sourcing components with known vulnerabilities and potential license risks, software development teams are building up higher levels of technical and security debt. And in order to reduce some of that debt, unplanned and unscheduled rework is often required.

No Developer Intentionally Uses Bad Parts

I have never met a developer that intentionally wanted to use “bad” components in their applications. That said, I have also never met a developer that wanted to spend four hours researching the latest versions, open source licenses and known security vulnerabilities for components (including dependencies) they wanted to use in their applications. While 57% of development organizations have internal policies in place that direct “thou shall not use components with poor quality and integrity”, many have stated that the policies are not followed or are simply not enforced.

The problem is one of volume and velocity, while people clutch to old processes that simply can’t keep pace. The analysis in the 2015 State of the Software Supply Chain report revealed that billions of open source downloads occurred in 2014, of which 6.2% had known vulnerabilities (1 in 16 downloads). For some large companies, the volume of downloads exceeded 240,000 — where 15,000 (7.5%) had known security vulnerabilities. At these volumes, manual processes and paper-based policies to prevent poor quality or risky components from getting into your software have no possibility of keeping pace. While software development practices aim for higher velocity, through component-based development as well as Agile, continuous and DevOps practices, the processes to keep up with it have not changed enough.

The only way to keep pace with this velocity is through automation. And if we want to avoid unplanned rework associated with remediating known security vulnerabilities or selecting alternative components with approved license types, automation has to become the norm rather than the exception within development.

At Speed

“If I look at the mass, I will never act. If I look at the one, I will.” — Mother Teresa

Looking at the massive volume and velocity of open source and other artifact consumption can be discouraging and overwhelming. However, the volume and velocity within our software supply chains will not diminish—and without a new approach, the volume of unchecked quality and integrity of parts being consumed will continue to build up as technical debt.

Automation in areas of testing, build, and deployment has provided significant performance benefits. Likewise, investments in software supply chain automation have shown markedly improved efficiency and controlled risk, as the best practices in the 2015 Report illustrate. Automation can unleash the potential of an organization’s development capacity. Rarely is there such an opportunity to simultaneously increase speed, efficiency, and quality.

In order to improve the quality and integrity of components used in our applications, we need to release our clutch on old practices. We can further embrace automation across our software supply chains — and improve the quality of our applications — by considering these three steps:

1. Start with creating a software bill of materials for one application. Visibility into one application can help you better understand your current component usage. A number of free and paid services are available to help you create a software bill of materials within a few minutes. The bill of materials will help you to identify the unique component parts used within your application and the suppliers who contributed them. These reports list all components used, and several services also identify component age, popularity, version numbers, licenses, and known vulnerabilities.

2. Design approval processes to be frictionless, scalable, and automated. Manual reviews of components and suppliers cannot keep pace with the current volume and velocity of consumption. Your organization must not only define your policies for supplier and part selection but also find practical ways to enforce them without slowing down development or inadvertently encouraging workarounds. Policies must be agile enough to keep pace with modern development. Strive to automate policy enforcement and minimize drag on developers.

3. Enable developer decision support. Developers are primary consumers of components in software supply chains. They initiate every component request. Help developers by automating the availability of information on component versions, age, popularity, licenses, and known vulnerabilities within their existing development tools so it is easy to pick the best components from the start. By selecting the highest-quality components from the highest-quality suppliers early, you will improve developer productivity and reduce costs.

Here is the complete 2015 State of the Software Supply Chain Report. I will also be discussing more of the Report findings and best practices on a webinar scheduled for Wednesday, June 24, 12pm ET (register or view it on-demand).

The previous blog in this series was, Fewer and Better Suppliers. Next up, I will be reviewing Visibility and Traceability practices across the software supply chain with more commentary from the 2015 State of the Software Supply Chain Report.

June 24, 2015
Three Ways Dev and Ops Teams Can Manage the Integration Lifecycle
Many enterprise organizations did not invest enough money into integration as they built their cloud or hybrid infrastructures. As if integrating on-premise systems and applications wasn’t tough enough, in today’s cloud era with the rise of hybrid environments, connectivity has only gotten more complicated – especially when combined with the increased speed of deployment. Puppet Labs found that high-performing organizations deploy code 30 times more frequently than their lower-performing counterparts, meaning dev and ops teams are struggling to break down data silos while simultaneously managing constant technology upgrades for line-of-business users.

There’s no doubt that, whether public or private or hybrid, the cloud is now part of most companies’ infrastructures, and its share will only increase. Goldman Sachs estimates that spending on cloud computing infrastructure and platforms is expected to grow at a 30 percent CAGR from 2013 through 2018 compared with 5 percent growth for overall enterprise IT, while Forrester predicts that 25 percent of all software applications purchased will be cloud-based by 2020.

With the speed and agility the cloud imposes on businesses, dev and ops need to come together to look at data integration throughout its entire lifecycle, from developing new apps or services on top of existing apps, all the way to the ongoing service, maintenance and upgrades of those applications. The two teams must join forces to manage this process as one comprehensive integration lifecycle, where dev and ops play equal roles solving for new data integration challenges brought about by hybrid deployments and siloes created by line of business users’ cloud purchasing habits.

Today’s dev and ops teams have to look at data integration as a service, not just a one-off implementation they can hand off. Here are three strategies to help bring the two teams together to jointly manage the data integration lifecycle.
1. Assess Current Integration Challenges Collaboratively
  Without alignment between dev and ops teams, integration lifecycle can’t be managed successfully. Organizations need to align on the people and process portion first, asking questions like: what systems does each business unit need data from? How likely is it that their needs will change, and how can you plan for a flexible integration as a service? What capabilities will ops need after dev implements the integration?
After reaching agreement, teams will need to jointly assess and select a data integration as a service technology or platform that allows for joint requirement setting, collaboration, testing and ongoing adjustments throughout the lifecycle of the applications. Selecting the right integration platform as a service at the onset is critical to minimize cost, time and wasted dev and ops cycles down the line as business and technology needs change. The most agile integration option typically consists of a plug-and-play, flexible platform that makes use of APIs and also allows for deeper customization down the road.

2.Understand the Four Key Stages of Integration Lifecycle Management
Dev and ops teams are now part of the same cycle dealing with frequent updates and agile product development. It’s no longer about developing, testing and then having ops maintaining, but about a seamless and transparent handoff from dev, to ops and back throughout the entire application and data integration lifecycle.

To capture the most value from disparate organizational data and to minimize ongoing data and application synchronization costs, dev and ops teams need to adopt a joint integration management strategy that accounts for the four distinct phases of data and application connectivity. These include:

3. Design phase. After understanding business use cases and selecting a platform, dev and ops must collaborate on priorities – what near-term functionality can they design for based on the plug-and-play layout, and how can they tackle that low-hanging fruit while maintaining the flexibility to meet future requirements?

By starting with the right technology and a common integration framework, dev and ops teams can design connected core systems and data early on with plug-and-play API connectivity for solutions like CRM and ERP, while maintaining the ability to easily expand functionality with custom/deeper integration capabilities with marketing automation, business intelligence and other apps.

In addition to functional capabilities, the design phase should address performance and scalability by taking into account throughput and latency. This process involves planning to meet performance expectations, even with the customization and extensions users will likely want down the road. Dev and ops should ask themselves questions like: which company systems will be on premise, in the cloud or hybrid? What are the performance, scalability and security tradeoffs of each approach?
1. Develop phase. Dev and ops need to work together in the development phase to carefully document key aspects of the integration while building connectivity for specific use cases. This documentation will be critical to the seamless handoff between the two departments, so they both need to be involved in the process. Strategic collaboration and thorough documentation can improve the efficacy of maintenance and debugging activities, help companies adjust to staffing changes and more easily transfer integrations to and from the two teams during the ongoing development and roll-out process.
2. Deploy phase. Given that you’ll likely deploy in a hybrid environment, make sure you test both on-premise and cloud integrations to ensure that data is flowing freely. To help test integrations and move them into a production environment, look into staging and production environment tools that streamline the process. Infrastructure costs should also be considered, as they may increase unexpectedly if you don’t watch them throughout the deploy phase.
3. Run phase. Continuously monitor to confirm that the data is flowing as designed. If dev and ops selected the right technology platform, the teams won’t have to worry about ongoing API management, because the platform takes care of it.
The right platform will also lower downtime and maintenance costs with a multi-tenant management environment that allows the expert for any given situation to dive into the integration backend or runtime. The platform should also provide ongoing diagnostic reports and proactive alerting to allow ops to home in on potential performance issues.
1. Build a Responsive IT model
  With dev and ops coming together to take advantage of the speed and agility offered by the cloud, the resulting just-in-time services for businesses are turning IT from a cost center into an innovation center. The data integration lifecycle is another result of the new cloud-shaped world, helping dev and ops teams create a more responsive IT model and allowing them to better meet business needs while driving innovation. While shared environments bring with them an additional element of risk, a responsive IT model is an important step towards both short-term and long-term success in the modern connected enterprise.
These three steps allow dev and ops teams to offer the value, speed, efficiency and opportunity that today’s enterprises require. By adding collaboration and agility at every stage of the integration process to plan, implement, track, debug and update continually, dev and ops teams can scale to respond to the data-driven needs of the business more precisely.

How have you overcome cloud and hybrid application and data connectivity challenges in your organization? Connect with me in the comments below, or on Twitter at @scribesoft.

About the Author/John Joseph

John Joseph, VP of marketing for Scribe Software, earned a master’s degree in business administration from the MIT Sloan School of Management, a master’s degree in electrical engineering from the University of Southern California and a bachelor’s degree in electrical engineering from MIT.
June 23, 2015
Bringing the Database into the DevOps Fold

In my last post on devopsy.kinsta.cloud I asked the question, “Where is the DBA in the DevOps Conversation?” I have wonderful news. I’ve found them! Having not been offered a seat at the table with everyone else, the DBAs have decided to have their own party in the form of a survey and corresponding report by Unisphere Research for the Independent Oracle Users’ Group.

Until now so much of the research, literature, and web content on the topic of DevOps has been completely devoid of talk about the database and DBAs. The oversight was significant and troubling given the immense strategic business value of any organization’s ever growing store of data. The IOUG report does an excellent job of framing the case for DevOps in terms of the special considerations required when trying to adapt to the lightning fast pace of change in modern IT.

For me, the biggest take away from the report is that DBAs and their managers are feeling the pain of increased release velocity acutely.
Interestingly, while the age of automation and cloud may be upon us, an increasing number of database managers and professionals report that they are being mired in manual processes. More than one-third cited this as a challenge, up a dramatic 162% from four years ago.
There’s no question that companies large and small understand that releasing quality services and features based on market demand at a faster rate than your competition keeps your organization on top. It’s also apparent that product development and most of IT operations have made the necessary investments in process modification and the automation necessary to truly achieve faster delivery. Unfortunately the report confirms what the good folks at my company, Datical, have been saying all along. The lack of safe, intelligent deployment automation options for the database is starting to limit the potential gains of DevOps implementation…and the data team knows it. But it’s not the actual script execution that takes so much time. It’s all the time spent before that maintenance window making sure that security isn’t being compromised, that the change is compliant with external regulations and corporate standards, and that it won’t negatively affect your data.

Today’s manual database operations, processes, and approval time are among the causes of delays holding up the real-time enterprise, many database managers and professionals admit. Frantic development cycles and the accelerated pace of business innovation requires that data and insights be available at a moment’s notice. Close to 41% of respondents report that it takes a week or more to approve change requests, as well as to configure new databases for key purposes, even for testing.

To safely reduce these cycle times a DevOps solution for database deployments has to harness the expertise and experience of an organization’s DBAs to fuel intelligent, environment-aware automated validation of these change requests. The automation can’t simply parse the SQL script looking for patterns that may be problematic. It must be able to evaluate all 5 W’s associated with the change and make the correct determination as to whether it is acceptable or needs further review. By focusing the valuable finite resource that is your organization’s DBAs on only those changes that need their special attention, you’ll reap even greater rewards from your DevOps investments.

I’ve just scratched the surface of what’s available in this report and I highly recommend you read and share it with the DBAs at your company. The survey covers responses from folks in every size company and at every level of the database team. It is full of facts and observations that will be readily familiar to any DevOps enthusiast or practitioner. DBAs, IT, Developers, and the business are now all speaking the same language when it comes to DevOps. Do what you can in your organization to make sure they are speaking that language to each other.

About the Author/Pete Pickerill

Pete Pickerill is Vice President of Products and Co-founder of Datical. Pete is a software industry veteran who has built his career in Austin’s technology sector. Prior to co-founding Datical, he was employee number one at Phurnace Software and helped lead the company to a high profile acquisition by BMC Software, Inc. His ability to understand product demands from a customer’s perspective and translate those demands into actionable product and development plans has led to expanded duties at every company for which he’s worked.

June 23, 2015
Choosing a Mobile Test Automation Solution: What You Need to Know

Organizations today rely more and more heavily on mobile apps to drive their business. In fact, many companies are already seeing more than half of their web traffic coming in via mobile devices. In some cases, their mobile app is the only way customers interact with their products.

It’s no surprise, then, that developing, testing, and releasing quality mobile apps quickly and efficiently is a high priority for most organizations—as it should be! Fortunately, mobile development tools are evolving to meet these needs, with modern dev teams having access to better test automation than ever before.

Of course, the effectiveness of a mobile testing process is only as good as the technology behind it. This makes it even more crucial for organizations to know what factors are most important to take into account when choosing or building a robust mobile testing solution. Below, we outline some key things to consider:

Native, Hybrid, and Mobile Web Support
Modern mobile applications come in three different flavors: native, hybrid and web. Native apps are built using the vendor-provided SDKs. Hybrid apps run some or all of their functionality inside a webview embedded in a native app. Finally, mobile web apps are simply mobile-optimized website that users access through their mobile browsers. While there is a growing trend towards mobile web and hybrid applications (rather than native), to ensure maximum flexibility and impact of your QA solution, it’s important that you choose a tool that supports all three kinds of apps.

On-Demand Cloud Testing
With QA teams becoming more and more distributed, it makes sense to ensure that your mobile application testing solution includes a cloud component. Leveraging an automation cloud enables teams to use organize around a single implementation of test architecture, with devices and test result data accessible from anywhere in the world. You can spend your time worrying about your testsuite, not the cost and maintenance headaches associated with keeping up multiple home-grown device grids and architectural patterns.

Automation
When it comes to mobile testing, automation is essential. Choosing the right automation frameworks (Selenium, Appium, etc…) and test tools (languages, test runners, etc…) can be a challenge, and it’s important to think about all the factors involved in getting a solid mobile build process going, relative to the expertise your QA team possesses. What kinds of user flows will you need to automate, and can the automation tool support them? Do you want automation code mixed in with your app code or cleanly separate? Is automation support available for the languages and tools you know best? Think about building a testsuite with the same level of architectural foresight as you would building an app.

Integration with Key Support Systems
It’s also important to consider what level of integration is needed with support systems in your environment – whether it’s a test management system, a CI server, or defect tracking system. A strong solution will be able to easily integrate with all of these other systems, perhaps all the way down to the IDEs your developers are using.

Cross Platform Support
With the majority of applications being developed for iOS and Android, it’s a minimum requirement for your mobile application testing solution to support automation on both platforms. Of course, this will no doubt expand as you consider other devices and operating systems such as Windows, FirefoxOS, etc…, based on your customer base.

Scalability
As the number of device types and operating systems being tested increases, it becomes increasingly important that your test architecture is able to scale automatically based on the number of tests you want to run and the amount of time you want your build to take. Building your automation on a framework like the open source Appium project is a good architectural investment; Appium’s client/server architecture makes it easy to build a testsuite than can automatically utilize a cloud service as your need for more emulators, simulators, and real devices grows.

The proliferation of web and mobile applications across a myriad of devices and operating systems is fueling a demand for faster and faster delivery of software experiences, without any sacrifice of quality. Today’s leading software companies know that successful apps are built on the foundation of their mobile application testing processes and tools. By keeping in mind the key features mentioned above, organizations can work towards building and identifying the best testing solution for them. The QA process should be a breeze, rather than a hassle.

About the Author/Jonathan Lipps

Jonathan Lipps serves as the director of ecosystem and integrations at Sauce Labs, where he leads a team of developers to ensure that the open source ecosystem surrounding the company’s core products is always maintained. He is a frequent speaker on automated web and mobile app testing, and holds advanced degrees in Linguistics and Philosophy. Reach Jonathan on twitter.

June 23, 2015
7,600 Open Source Projects Per Company (and how it impacts DevOps)

That Supplier is Better For You

Since releasing the 2015 State of the Software Supply Chain Report, there has been a lot of great discussion across the industry on best practices for managing the complexity introduced by the volume and velocity of the components used across your software supply chain.

Today I want to focus on the huge ecosystem of open source projects (“suppliers”) that feed a steady stream of innovative components into our software supply chains. In the Java ecosystem alone, there are now over 108,000 suppliers of open source components. Across all component types available to developers (e.g., RubyGems, NuGet, npm, Bower, PyPI, etc.), estimates now reach over 650,000 suppliers of open source projects.

Source: ModuleCounts.com

However, like in traditional manufacturing, not all suppliers deliver parts of comparable quality and integrity. My latest research, the 2015 State of the Software Supply Chain Report, shows that some open source projects use restrictive licenses and vulnerable sub-components, while other projects are far more diligent at updating the overall quality of their components. Choosing the best and fewest suppliers can improve the quality and integrity of the applications we deliver to our customers.

While I am hosting a webinar next week to share many of the detailed report findings, I wanted to share a few of the more meaningful stats here.

Your 7,600 Suppliers

My research for the report revealed many new perspectives on “suppliers” across the software supply chains. First of all, I saw that the average large development organization consumed over 240,000 open source components last year — sourced from over 7,600 open source projects.

Average number of open source component downloads per organization in calendar 2014

On the surface, the huge reliance on open source projects is a great thing. Development teams have chosen to not write those pieces themselves, but have sourced the needed components from outside suppliers. This practice speeds development, enables more innovation, and ensures time-to-release goals are achieved. The use of open source is so prolific today, few of us could ever imagine reducing the use of those components and their suppliers in the future.

At the same time that we benefit from open source, our high paced, high volume consumption practices don’t allow us the time needed to do the due diligence on the suppliers or open source projects where we source our component parts from.

Average number of known vulnerabilities downloaded by organization in calendar 2014.

For example, of the 240,000 average component downloads in 2014, the same businesses sourced an average of 15,000 components that included known security vulnerabilities. In many cases, developers were downloading vulnerable component versions, when safer versions of those same components were available from the open source projects. While no one intends to download components with known vulnerabilities, the problem is exacerbated due to the lack the visibility into a better recommended version.

Fewer Suppliers, Less Context Switching

Choosing an open source project supplier should be considered an important strategic decision in organizations because changing a supplier (“open source project”) used is far more effort than swapping out a specific component. Like traditional suppliers, open source projects have good and bad practices impacting the overall quality of their component parts.

Traditional manufacturing supply chains intentionally select specific parts from approved suppliers. They also rely on formalized sourcing and procurement practices. This practice also focuses the organization on using the best and fewest suppliers — an effort that improves quality, reduces context switching, and also accelerates mean time to repair when defects are discovered. One industry example from the report describes how Toyota manages 125 suppliers for their Prius to help sustain competitive advantages over GM who manages over 800 suppliers for the Chevy Volt.

By contrast, development teams working with software supply chains often rely on an unchecked variety of supply, where each developer or development team can make their own sourcing and procurement decisions. The effort of managing over 7,600 suppliers introduces a drag on development and is contrary to their need to develop faster as part of agile, continuous delivery and devops practices.

Coming to Terms

When you come to terms with the volume of consumption and the massive ecosystem of suppliers you can source your components from, you quickly realize it is impossible to address this issue with a manual review process. Any organizations clutching to these outdated manual practices are will continue to be outgunned by the velocity by their software supply chains.

Just as traditional manufacturing supply chains have turned to automation, software development teams need to take the same approach by further automating their software supply chains. Information about suppliers and the quality of their projects needs to be made available to developers at the time they are selecting components. Information about the latest versions, features, licenses, known vulnerabilities, popularity of versions being used, and the cadence of new releases should be made available to developers in an automated way. Automating the availability of this information about suppliers can lead to better and fewer suppliers being used.

Be sure to read the full 2015 State of the Software Supply Chain Report for more information about open source suppliers and organizations sourcing practices. The report also highlights current and best practices being used in organizations that are managing their use of suppliers that feed their software supply chains.

To hear more about the overall report findings and industry best practices, please join me on Wednesday, June 24th (1pm ET) for our webinar.

June 19, 2015

Two paths to metal devops: cloud-like API driven & cluster building

I’ve been seeing a rising interest in metal DevOps fueled by containers and scale-out data center platforms (like Hadoop, Ceph & OpenStack) that run at the metal level. While I see this is a growing general trend (Packet, Internap , RackSpace, OpenStack Ironic, MaaS), I’m going to stay firmly within my wheelhouse and use OpenCrowbar as my reference here.

Building on the API-driven metal features of OpenCrowbar, this has translated into two paths for workloads to run on metal:

1) “Cloudify” the metal using APIs from tools like Chef Provision, SaltStack Libcloud, Docker Machine, Cloud Foundry BOSH. These tools have clients that target cloud APIs like OpenStack and Amazon. These same clients work against cloud are easily ported to Crowbar’s APIs. Five years ago, conventional wisdom was that we’d need a universal cloud API; however, practice has shown it’s not very difficult to wrap APIs in a way that does not reduce every cloud to a least common denominator.

2) DevOps deploy the workload using hand-offs to tools like Chef, Saltstack, Puppet or Ansible. This approach leverages the community scripts (Cookbooks, Modules, Playbooks) for the workload with the critical ability to create a tuned environment and inject the needed parameters directly into the scripts. A critical lesson we learned going from Crowbar v1 to v2 was for our scripts to have crisp attribute input/output boundary to avoid embedding environmental knowledge into the code.

While I’m casting this in Crowbar terms, I see this approach to metal as coming into the market by force fuels by a desire for containers-on-metal and devops-on-metal.

Let’s look at some of the unique and shared use-cases for each approach:

Metal API	Both	Metal Cluster
Easy Cloud to Metal Migration Minimal Tool Customization	Portability of DevOps Scripts Take advantage of power cycling Enables constant refresh cycles	Leverage Hardware features Advanced Network topologies

In either case, you have to handle bespoke (hipster word for custom) steps in the provisioning flow that are unique to the your operational needs. Our experience is that each site (even each server!) is unique in some incremental way. For example, one site may require teamed networks with VLANs while another requires flat networks with an SDN layer.

These differences are not mistakes or errors: the reality of physical ops and individual operational choices mean that there are a lot of valid configurations. Rather than attempt the Sisyphean task of enforced conformity, we work to abstract differences so that they can be ignored when they are not material.

In the end, the choices are not mutually exclusive. Metal APIs are often faster but harder to optimize. You can use them to get started quickly and then invest time to optimize a cluster for long term operations. The underlying physical orchestration can support both.

Are you looking at getting closer to metal? Which of the options above makes the most sense to you? I’d love to hear about your use-cases, architecture and configuration requirements.

June 19, 2015

Tag: devops

Your 5,100 Binaries Went Bad

We’re Not Alone

Did he recommend a “Bomb”?

Rework is Hell

No Developer Intentionally Uses Bad Parts

At Speed

About the Author/John Joseph

About the Author/Pete Pickerill

Let’s look at some of the unique and shared use-cases for each approach: