devopsy.kinsta.cloud

Tag: QA

TechWell Announces the Full Program for STAREAST 2016

The Premier Event for Software Testing and QA Professionals to be Held in Orlando, Florida

Jacksonville, Florida (PRWEB) February 03, 2016

TechWell Corporation, an industry leader in software development conferences and training, has announced the full program for STAREAST, one of the longest-running, and most respected conferences on software testing and quality assurance, to be held May 1–6, 2016 at the Renaissance Orlando at SeaWorld® in Orlando, FL.

The week-long conference will feature over 100 learning and networking opportunities and covers a wide variety of some of the most in-demand topics: Test Management, Test Techniques, Test Automation, Agile Testing, DevOps & Testing, Testing the Internet of Things, Mobile Testing, Testing Metrics, Cloud Testing, and Performance Testing to name a few.

Attendees can build a customized week of learning with a variety of topics available through pre-conference training classes, full- and half-day tutorials, testing-oriented concurrent sessions, and keynote presentations.

One of the most popular features of STAREAST remains the top-notch keynote presentations.

This year’s keynote speakers include renowned testing professionals: Keith Klain, David Dang, Lee Copeland, Isabel Evans, and Dawn Haynes. “The keynotes will allow attendees to gain insight into the evolution of the software testing profession and lessons learned along the way,” said Allison Wade, President, TechWell Corp.

Keynote presentations include compelling topics like Lessons Learned in (Selling) Software Testing, Open Source Test Automation: Riding the Second Wave, The Evolution of a Software Tester: From Novice to Practitioner, Telling Our Testing Stories, and Lightning Strikes the Keynotes.

STAREAST will also feature a Women Who Test full-day event on Friday, May 6 during the conference. Women Who Test was introduced at last year’s STARWEST conference in Anaheim and received rave reviews by the participants. This program will cover testing topics and will support both women’s personal and career journeys. The Women Who Test event covers prominent topics for female test/qa professionals, including: reengineering your life, growing your personal brand, setting goals, discovering your career superpowers, and how to problem-solve as a female in the tech industry where the ratio of men to women is drastically outnumbered at 7:3.

For more information on STAREAST and access to the full conference program, visithttps://stareast.techwell.com.

ABOUT TECHWELL CORPORATION:

TechWell—through its conferences, training, consulting, community websites, and online resources—helps people and organizations develop and deliver great software. For more information, visithttp://www.techwell.com.

For the original version on PRWeb visit:http://www.prweb.com/releases/2016/01/prweb13186748.htm

February 3, 2016
What it’s Like Working as Quality Assurance in Consultancy Companies

Quality Assurance (QA) engineers working at consultancy companies should embrace trying new technologies and solving problems. This role is best defined by two words: variety and challenges. QA engineers at consultancy companies must pay great attention to detail and check all of their actions meticulously to make sure everything is being done correctly. The position also requires equal focus on the quality of the software application and client expectations, and constant communication between all individuals and teams involved in the process.

Thinking about a job in QA? Here are some more details about this demanding yet rewarding position.

Expectations

Working as a QA engineer in a consultancy company demands very strong communication skills, flexibility and prioritization skills to help deliver a high-quality product in a timely manner. This role requires continuous improvement of both the process and working methods, which are not limited to testing or preventing defects but also consider the application’s lifecycle and ways to improve the product.

Challenges can arise from working with myriad intricate software applications and environments. As the complexity of the software and the multitude of targets that can be addressed increase, so do the technical and communication challenges. A good QA engineer should always plan for this complexity in each project and be prepared to handle these challenges.

Variety

QA challenges and projects become even more complex and varied at consultancy companies. In a consultancy company engineers work with multiple companies from different cultural backgrounds and different levels of technical experience. Sometimes there are significant cultural differences across projects and within a team, which can have a big impact on the overall quality assurance process. To overcome any obstacles, QA engineers must communicate frequently with each company to answer any questions that may occur during the project lifecycle and to keep it on track.

For each project, the QA engineer is expected to contribute to various parts of the process, which can include everything from client and internal team communications to delivering technical requirements. In addition, QA engineers must ensure technical requirements are communicated, tracked, planned, scheduled and executed correctly for each team. This is where attention to detail and prioritization skills are key, in making sure the correct communication is reaching the correct teams so the project runs smoothly and successfully.

In addition to the innate variety that exists within the QA role, there is a fair amount of diversity, with multiple clients of different sizes spread across numerous industries. While some clients write out the project specifications and define clearly the expected outcome of the software application, others don’t, creating another opportunity for a QA engineer to contribute to a new project.

Finally, QA engineers in a consultancy company constantly work with a variety of platforms and applications, from desktop to mobile projects and web-based to server-side. I see this as a positive—it keeps you on your toes and you constantly are gaining additional technical and architectural experience.

As a Quality Assurance consultant, you will be expected to work on a wide variety of technologies and communicate effectively under a variety of circumstances. While the position can be challenging, it’s never boring. And knowing the project is successful because of your work is extremely rewarding.

January 4, 2016
How do you identify gaps in your testing?
Every piece of software ships with some defects in it, but major gaps in your testing coverage can be disastrous, and they can occur for a variety of different reasons. Part of the problem with a typical root cause analysis is that it comes after the fact. Perhaps the customer has complained, there’s been a big outage, or the software quality simply isn’t where it should be. But reactive firefighting like this isn’t the most effective use of your resources.

What if you could identify those gaps earlier in the testing process and dynamically shift strategy to close them before they cause major issues?

The best possible start

Lack of domain knowledge and an incomplete understanding of the purpose and intent of a piece of software is enemy number one. There are two ways you might consider mitigating this risk in order to get a good start:
1. Automated Testing – Identify critical core functions and automate mundane repetitive tests in order to free up your testers to leverage their experience and domain knowledge to break the system.
2. Exploratory Testing – Context driven testing encourages critical thinking and collaboration. Testers have to understand what the application is trying to do, they have to learn about the intent and direct their efforts at the most important areas.
Making sure you build a solid foundation, understand the software, and define your scope at the outset can take time, but it will be time well spent.

Communication and assessment

The traditional approach with gap analysis is to start with a problem and trace it back. In theory, you should learn from your mistakes and avoid repeating them in the future, but it doesn’t always work that way. It’s also a great deal more time-consuming and expensive to identify and fix problems when they’re already released. There’s no reason you can’t apply the same logic from root cause analysis earlier in the process.

The earlier testers are involved, the better. There should be wide-ranging discussions during the planning phase that include the development team, the business team, and the test team. This is an excellent opportunity to ensure that everyone understands the workflow of the new software, what it’s supposed to deliver for the end user, and where the priorities lie for validation and testing.

By including testers from day one, you reduce the chance of misinterpretation and give yourself an opportunity to spot potential gaps or misunderstandings. But the communication and assessment has to be continuous throughout the project, the second it slips, you’re running the risk of a gap opening up. You need to make sure that communication, collaboration, and mitigation are an integral part of your process for each new sprint or cycle.

Developing process oversight

In order to ensure that your coverage is tight, and to immediately flag possible gaps, you need to have a good test data management strategy in place from the beginning. You need to be using analytical tools, like Jira, QMap, or VersionOne. Regardless of the tool in play, what you want is a complete record of all your manual testing activities. The ability to generate reports or visualizations of your coverage, and to analyze it from different business perspectives, is absolutely vital.

Where are most of the defects coming from? How did you find them? How critical are they? Where are your efforts being concentrated? What topics are trending in tester’s comments? Are you seeing the same questions or recommendations popping up over and over again?

There’s a good chance that you’re collecting this data anyway, but keeping an eye on all of it in real-time should enable you to strategize effectively about where to focus your resources. The best approach is never carved in stone, it should continually evolve throughout the development lifecycle. Rigorous and regular analysis presents an opportunity to identify gaps as soon as they begin to form, instead of waiting until they cause a problem and then tracing it back. It’s about taking a proactive approach instead of a reactive one.

About the Author/Mush Honda

Mush Honda is QA Director at KMS Technology, a provider of IT services across the software development lifecycle with offices in Atlanta and Ho Chi Minh City. He was previously a tester at Ernst & Young, Nexidia, Colibrium Partners and Connecture. KMS services include application management, testing, support, professional services and staff augmentation.
December 8, 2015
BoomTown No Bust with VM Supported Parallel Testing Environment

An Introduction to BoomTown DevOps Processes

BoomTown uses a specific set of DevOps processes in order to deliver websites with the latest real estate listings and housing data as well as a CRM solution and analytics tools for real estate professionals. “Our development, operations, and QA teams automate everything in sight while offloading to cloud providers and other services the tasks that those providers are best suited to,” says Brian Baumgartner, QA Manager, BoomTown.

As only one example, BoomTown recently adopted Amazon Web Services (AWS) for hosting various aspects of its infrastructure. “It enables us to provide high availability and resilient service while allowing our development team to focus on revolutionizing the Real Estate software industry,” says Baumgartner.

Docker containers support regular deployment processes at BoomTown, containerizing the environment setup for a more reliable, orderly deployment experience, whether the firm is pushing code to QA or to production, explains Baumgartner. TeamCity, the increasingly popular and frequently appearing name in build management tools and CI servers courtesy of JetBrains is BoomTown’s go to build platform. New Relic is BoomTown’s performance management solution of choice, enabling live transparency into the production application and server performance that keeps BoomTown citizens (customers) content.

BoomTown Before & After Using Sauce Labs VMs to Scale Testing

Prior to applying Sauce Labs, BoomTown’s QA analyst team processed all software tests exclusively in a manual fashion. Every test run crossed their laps / machines / laptops. “Before a new feature release, an analyst would execute a full set of regression tests against the product or feature, often through several different browsers and versions to verify the new release and its function. This process alone took up to two weeks for large projects,” says Baumgartner.

BoomTown now uses a test automation framework constructed using several popular DevOps tools. The custom built testing automation leverages Ruby, Selenium WebDriver, Cucumber, MySQL, Sinatra, and Angular JS, says Joey Gryder, Senior Software Engineer in Test, BoomTown. The framework offers BoomTown views into historical and real-time test results as well as the capability to run manual tests through its dashboard front end software connection to its backend systems.

BoomTown applies a single web service to assert control and manipulation of test system automation features. A “test runner” library enables QA to run Cucumber features in a parallel execution approach. BoomTown uses MySQL to store test results. “When we combine all this with Sauce Labs’ VMs, we can schedule nightly test runs using our full testing suite on a variety of browser configurations,” says Baumgartner. This has enabled BoomTown to run up to 100 tests on software nightly.

Parallelization

Where time, resource, and person hours’ savings are concerned, the secret sauce in BoomTown’s software testing framework is its use of Sauce Labs VMs to support many automated tests in parallel. This has cut BoomTown’s longest test run time of an hour and a half down to less than 15 minutes.

“With Sauce Labs, it is trivial to spin up any number of parallel VMs, configured against your desired browsers to run through a set of Selenium WebDriver test steps. Without such a solution, BoomTown’s test team would have had to either execute its entire test suite locally through a single machine and browser, or build an in-house testing lab to achieve parallel browser testing,” explains Baumgartner. (Sauce Labs is obviously not the only player with competitors including BrowserStack and TestingBot; let your due diligence be your guide in selecting the best service for your needs.)

BoomTown saw offloading the work as the best option in order to accelerate otherwise time intensive testing tasks, preventing QA from becoming a sort of time-related bottleneck on CD. “We did this simply by increasing our VM count and running more tests in parallel,” says Baumgartner.

Adding VMs and increasing the number of parallel testing instances is an easily scaled approach that continues to speed tests and drive CD even as BoomTown adds tests and lengthens total test runtime.

Using automated parallel tests supported by Sauce Labs VMs frees up QA analysts to be more creative in their testing, finding more and more interesting ways to break the application under testing loads, says Baumgartner; meanwhile the automated testing ‘machine’ verifies the mundane and repetitive tests that the team has already established.

“We recently incorporated automation as a quality gate in our CI process. Before a pull request is accepted, it must pass a set of critical path UI tests that run up in Sauce Labs. This sets a certain level of quality expectations during our daily development process, which saves us time in testing,” explains Gryder.

November 19, 2015
Meeting Challenges in incorporating automated testing throughout the DevOps cycle

Enterprise development teams find it difficult to perform QA tests on software in short DevOps cycles where they cannot squeeze in lengthy regression tests. DevOps teams must insert automated testing approaches to resolve this predicament. “DevOps teams need to be more selective in prioritizing tests that occur at each stage of the integration to delivery pipeline,” says Marc Hornbeek, Senior Solutions Architect of DevOps, Spirent.

At each point between stages of the development lifecycle, there is an opportunity to insert the most applicable tests to determine whether the software has incurred errors. It is crucial to test early. The earlier in the development process that team members discover their errors, the sooner they can fix them, ensuring fewer error-related issues in releases. The earlier in the development lifecycle that errors occur and go unchecked, the more opportunity they have to grow and become increasingly debilitating to the software.

With best practices, DevOps shops can use the right tests in the right stages of development and achieve quality assurance without depending on large test suites that run at the end of the cycle, according to Hornbeek.

The best time to create automated tests is together with or prior to developing the code for the given software feature itself. This will enable development shops to keep pace with the speed of DevOps; the old approach of automating testing after the feature is developed has truly become ineffective, according to Hornbeek.

DevOps Targets Quality / Requires Testing

DevOps has the goal of creating better software faster. Apart from effective, continuous, automated testing at keenly selected intervals, DevOps is only faster. This would achieve only a form of dev similar to what we had before DevOps, i.e., the kind where coders believe they have done well even though operations staff sees a lack of fitness of the software for production use.

Enterprises must embed a culture of QA inside the culture of DevOps rather than separate out QA testing from DevOps using organizational silos; communicating only through inefficient bug reporting methods is no longer the state-of-the-art, let alone practical with the increasing industry expectations of DevOps.

Testing: Why Must I Be Misunderstood?

The impact of testing is often overlooked. DevOps shops that take testing for granted may not invest in or improve upon testing as testing needs change and grow. “This was true before DevOps and continues to be true with DevOps. Developers focus on feature creation, while the business leaders focus on delivery,” says Hornbeek. As a result, testing is often not a focus of DevOps.

Further, there is a common misconception that testing just the part that is done after the builds are created is all that is necessary; but enterprises must integrate continuous automated testing with DevOps throughout the development-to-delivery pipeline, according to Hornbeek.

“When testing is conducted a long time after the feature is developed, there are many inefficiencies. The developers lose context of the problem and have a harder time recreating, diagnosing, and fixing problems,” says Hornbeek. Automated continuous testing eliminates this delay, improves software, and shrinks time to quality software releases.

Slothful Testing Gone Awry

There are high-profile examples of slothful software testing where projects grew out of scope and way off target. A chief example is the original rollout of Healthcare.gov, says Hornbeek. “In another example, a network equipment manufacturer tried to implement DevOps twice without success. They finally realized that they had underestimated continuous testing,” says Hornbeek of a Spirent customer.

To prevent these kinds of outcomes, says Hornbeek, follow these best practices:

• Create automated tests as you develop the features.
• Do not accept features into the integration branch without automated tests.
• Ensure that the developer test environment is a good representation of the production environment.
• Select DevOps-ready tools that have Rest APIs.
• Test priority features so you can easily integrate them into the DevOps orchestration framework.

Follow these steps, according to Hornbeek, to incorporate automated testing for QA:

• Establish a clear leader with responsibility over the entire dev-to-deliver pipeline who is visible and will champion continuous testing.
• Assign an architect for continuous testing tools selection and to oversee the technical implementations.
• Determine best practices for Continuous Testing and decide what practices best fit the organization.
• Address the culture, roles, and responsibilities for continuous testing.
• Put some automated metrics in place, such as a goal to automate 85% of all tests.

Editors Note: I had a chance to catch up with Mark Hornbeek at the Enterprise Summit 2015. Here is my interview with Mark.

November 11, 2015
Testing Nightmares: Real-Life Frightening QA Experiences

In the spirit of Halloween, this past month testing experts from around the globe answered the call to join in a “crowd-sourced” article highlighting their most frightening testing QA experiences. These are real-life examples of testing gone wrong—leading to all sorts of mayhem in some cases. We allowed these testers to hide their identity, and to keep their organizations anonymous to solicit the most behind-the-scenes responses.

In no particular order here are the top five Testing Nightmares, as told by the testers themselves:

The Importance Of Making Sure Your Hosting Company Gets Paid

By Ivar (from a medium sized company/ now acquired)

The company I was working for at the time had bet its existence on signing an enterprise contract that it had already been in negotiations for more than year with. We were building a digital asset management system that controlled the licensing and delivery of assets to corporate devices. (Think ‘private app store’.) I had written backup tooling that wasn’t ready for use but was included, unused, in the branch that was being deployed. Ten minutes after the final deployment (in preparation for client review!) all of our assets disappeared. Nothing new in the deployment was related to asset storage except my code. There was an eight-figure contract on the line and we had no backups. Fueled by the mother of adrenaline rushes, I reviewed my work and remained convinced that my code was not the cause of the loss. I was already into investigating other possible causes when another developer informed our team that after calling our asset host, he discovered that we had neglected to update our billing credit card. The host had turned off access to all of our assets ‘to get our attention’. They succeeded, but at oh what a price…

Not The Best Way To Start As The New Test Automation Engineer

By James Eisenhauer

This happened my first week as the new and only Quality Engineer at a health and wellness startup. I was on a mission to get test automation created and show them what I could do for them! My first test was to invite all the members of a test company to a new company wellness challenge. It just so happened the company President’s test account was a member of that test company. I scheduled my tests to run at 5 a.m. At 5:38 a.m. we all received an email from the President that read, “Emails every few minutes telling me that QA wants me to walk by the damn river. Make it stop!” “Walk by the River” was a copy of a real wellness challenge we had in our test system. Something in my auto-retry logic had gone horribly wrong. First impressions are a bitch!

Gaming Fail

By Abe Chiu

While working for a gaming app company, we pushed out a build with an engine update that reverted all of our users back to levels that were saved on their device. This as opposed to the one that was saved on our server side. (Example: If logged in to create an ID and was at level 50 on your device, that was the save that popped up even though on our server you had previously reached level 125). All of our users were affected. That’s not the worst part. Some of our users reverted back to level 1 when they were above level 100 and up. This all happened on a Friday and the build was pushed on Thursday night. On Friday we were only able to fix some issues with a small percentage of our user base but the ones that were reverted back to level 1, those were not able to be helped until the following Monday.

The White Screen of Death

By Anonymous

I work as a Software Engineer in test at a media company. This company has its own video player that is embedded either as a single standalone video or a playlist. The player was re-done completely, and I was responsible for testing it before it went out to production. I tested the player as a single video and found a number of issues so the release to production was pushed out. After the issues had been fixed, I tested it again as a video and then gave my “go-ahead” to release it to production. The problem was I had COMPLETELY FORGOTTEN to test the playlist! We had figured that everything looked good and since the release had already been pushed off once, we made a decision to release it to production at the end of the day, around 5:30 p.m. I double-checked it on production as a single video and it looked fine. An hour later, when I got home, we started receiving urgent messages and emails that the playlist wasn’t working!!! You could only watch the first video and then it became a blank white screen! Since the playlists are embedded so deeply, this was a major problem. At 6:30 pm that night only one engineer was around and online but he wasn’t sure how to revert it. Finally after a very stressful hour and a half, he figured out how to do it and I verified that everything was back and working again with the old player. I still can’t believe that I had forgotten to test the playlists, which is such an important part of the feature.

Global Communication Breakdown

By Joe Nolan

We were trying to create a mobile on-line conferencing feature to include in our education app, much like Google Hangouts. Testing had gone well in controlled environments. Then we pulled all of the teams together to have a large group call using and testing the feature. We immediately had reports of crashes and all sorts of issues. It was impossible to talk over the people trying to diagnose the problems, while at the same time others were randomly pushing buttons to try the feature. Even worse, participants were located around the world with different networks and connectivity issues. It was worse than herding cats!

About the Author/Sauce Labs

Sauce Labs is the leading cloud-based web and mobile application automated testing platform. Its secure and reliable testing infrastructure enables users to run JavaScript unit and functional tests written with Selenium and Appium, eliminating the time and expense of maintaining a test grid. With Sauce Labs, organizations can achieve success with continuous integration and delivery, increase developer productivity, and reduce infrastructure costs for software teams of all sizes.

October 29, 2015
Continuous test and monitoring essentials for SDN and NFV
The prior blog Continuous Test Monitoring DevOps Healthbeat discussed the importance and suggested practices for testing and monitoring test trends. This need is amplified when the system that is being tested and monitored is a network of co-operating applications, infrastructures and devices such as networked components of next generation Software Defined Networks (SDN) and Network Function Virtualization (NFV) systems. In networked systems a problem with any network component such as end-user applications, network interface devices, network control plane, network data plane, load balancers, firewalls, network administration, orchestration, and network virtual infrastructures can seriously impact user and network services performance. The number, variety, topology and versions of components and their host environments present special challenges for DevOps continuous testing and monitoring.
In an SDN and NFV world all production topologies and network variations must be tested and results must be monitored or the results will be uncertain. Despite the complexity, or perhaps because of it, continuous testing and monitoring must be optimized to ensure that testing and results analysis for each CI/CT cycle keep pace with accelerated continuous testing and longer term continuous test monitoring is necessary to assure overall delivery confidence for each release and longer term product performance trend in a positive direction. Individual test results or a fixed number of test campaigns can only tell you so much.

Unless the results of testing and test analysis are continuously monitored and results aggregated over multiple test and release cycles for all network nodes and all topologies then there is no way to build sufficient sustainable confidence in the long term health of the SDN network and NFV components. The combination of topology-aware continuous testing solutions together with continuous test monitoring tools that can aggregate results over the entire set of network resource can provide a longer term strategic view of test results that is necessary to collect, aggregate and organize test results data to gain confidence in network components and services for each release and evolution over multiple releases.

Below are some suggestions in a checklist format that have proven useful for continuous testing and monitoring for sustainable high performance SDN and NFV systems and services.
1. Determine continuous testing and monitoring priorities: Some examples of problems that continuous testing and monitoring can help with include intermittent failures caused by the virtual network infrastructures, virtual network functions, network administration, orchestration and management, 3rd party network and end-user systems compatibility, network performance, network fault-tolerance features. The best practice for continuous testing and monitoring indicates that the problems of most concern to a specific network system or service will be tested and result trends will be monitored.
2. Regression test network systems and services even though there were no expected changes: Unintended consequences of indirect changes may impact performance so automated regression suites should audit all production network test topologies and version variations areas occasionally just to be sure. Typical examples that are often caught by this are features important to audit functions, system backwards compatibility features and upgrade/downgrade administration features.
3. Select continuous test and monitoring tools that collect and report trends:
  Tools that can correlate and report test results across multiple network nodes and topology variations can find intermittent bugs or problem trends. Thresholds and email alerts and dashboards that highlight short term results views from long term results views are especially valuable.
4. Use continuous test monitoring results to diagnose and resolve problems:
  Intermittent failures and problems that only become apparent when test results are presented as trends over a long term trend are easier to diagnose when a large data set is accumulated and filtered by most probable cause tags. Once a diagnosis is determined, the root cause can be verified through targeted retest cases that set all the conditions in accordance with the diagnosis. Once confirmed then the offending design can be refactored to handle or avoid the failure condition.
The above is a partial list of suggestions for continuous test monitoring that have been proven to support sustainable SDN and NFV network systems and services. At Spirent and Zephyr we think continuous testing and monitoring is critical for successful DevOps. You can hear more about our views on this topic by reviewing our joint webinar at:
Overcoming DevOps Challenges

What do you think of these suggestions and do you have others that should be mentioned?

About The Authors ⁄ Marc Hornbeek & Sanjay Zalavadia

Marc Hornbeek is Sr. Solutions Architect of DevOps continuous test solutions at Spirent Communications, Infrastructure Test Optimization (ITO) BU. He recently managed DevOps at Spirent. He has performed as the primary architect of test automation tools and champion of test automation for firms ranging from start-ups to large multi-national companies. He published more than 30 articles and has been a speaker at numerous conferences and user forums primarily regarding topics related to continuous automated testing and DevOps.

Sanjay Zalavadia is responsible for driving custom success at Zephyr. This includes training, consulting, customer support and client management. Most recently, as the Associate Vice President for Patni Computers Telecoms IT Managed Services Practice, he established IT Operations teams supporting mobile content providers. Sanjay brings more than 15 years of leadership experience in IT and Technical Support Services teams across multiple geographies for both large and small companies. Sanjay has a graduate degree from the Manipal Institute of Technology in India
October 28, 2015
DevOps Leadership Series 2015: Women & Diversity

Many of the leading minds and practitioners in the DevOps movement are women. I hope that comes as no surprise to you. DevOps has always embraced a diversity of thought and contribution. Case in point, close to 50% of the presenters at this year’s DevOps Days Boston event were women.

In this episode of the DevOps Leadership Series, we caught up with Jennifer Davis, co-author of Effective DevOps (O’Reilly 2015, $31) and software engineer at Chef. Jennifer discusses a greater diversification of what DevOps means. She sees more explicit efforts to expand DevOps into QA, security, DBAs, and other functional teams across the organization. Jennifer also discuss the benefit of re-setting one’s expectations on the meaning of certain words (e.g., agile, test driven development, audit driven development) to accelerate progress and remove roadblocks:

https://www.youtube.com/watch?v=B-zONUI5KQ4

We then sat down with James Meickle, Site Reliability Engineer at Harvard and member of the organizing committee for DevOps Days Boston. James witnessed distinct growth over last year in the diversity of companies in attendance — both from a size and industry perspective. James also discusses the importance of diversity that the organizing committee insisted upon as they planned for the event. If you are organizing your own DevOps Days event, be sure to have a listen:

https://www.youtube.com/watch?v=BZZCjw9YEps

If you missed part one of our series from DevOps Days Boston, be sure to watch these interviews from Beno Chapman of Constant Contact, Anthony Spring of Yieldbot, and Jeremiah Sullivan of Puppet Labs.

Want learn more from the leading minds in DevOps? You can find the entire library of 21 videos from this series on YouTube.

Special thanks to my colleague, Mark Miller for capturing these videos for us at the event.

October 1, 2015
DevOps Leadership Series 2015: Women & Diversity – a diversity of thought and contribution

Many of the leading minds and practitioners in the DevOps movement are women. I hope that comes as no surprise to you. DevOps has always embraced a diversity of thought and contribution. Case in point, close to 50% of the presenters at this year’s DevOps Days Boston event were women.

In this episode of the DevOps Leadership Series, we caught up with Jennifer Davis, co-author of Effective DevOps (O’Reilly 2015, $31) and software engineer at Chef. Jennifer discusses a greater diversification of what DevOps means. She sees more explicit efforts to expand DevOps into QA, security, DBAs, and other functional teams across the organization. Jennifer also discuss the benefit of re-setting one’s expectations on the meaning of certain words (e.g., agile, test driven development, audit driven development) to accelerate progress and remove roadblocks:

https://www.youtube.com/watch?v=B-zONUI5KQ4

We then sat down with James Meickle, Site Reliability Engineer at Harvard and member of the organizing committee for DevOps Days Boston. James witnessed distinct growth over last year in the diversity of companies in attendance — both from a size and industry perspective. James also discusses the importance of diversity that the organizing committee insisted upon as they planned for the event. If you are organizing your own DevOps Days event, be sure to have a listen:

https://www.youtube.com/watch?v=BZZCjw9YEps

If you missed part one of our series from DevOps Days Boston, be sure to watch these interviews from Beno Chapman of Constant Contact, Anthony Spring of Yieldbot, and Jeremiah Sullivan of Puppet Labs.

Want learn more from the leading minds in DevOps? You can find the entire library of 21 videos from this series on YouTube.

Special thanks to my colleague, Mark Miller for capturing these videos for us at the event.

October 1, 2015
Impacts of DevOps on Testing
I always come across a standard question from testers in any DevOps related presentation. “Will DevOps remove the need for Testers?” A simple answer is “No”. DevOps is not going to replace Testing as a discipline. Firms implementing DevOps have a vision for better product quality workable by brilliant Testing.

DevOps is an enabler for the following.
- Reduction in IT Budgets
- Efficient end-to-end delivery
- Lean delivery framework
- Working end-to-end automation framework
Successful businesses today are design thinking driven. Businesses have a strong sense of urgency, aided by innovative software engineering practices. Businesses today need to have a working continuous delivery practice in place. Continuous delivery also mandates better product quality and zero touch end-to-end delivery. Automaton is key to success across DevOps processes. Delivery by co-located, integrated teams using innovative tools prevents collaboration and communication issues.

Manual errors, Stove-piped communication issues get sorted with a good DevOps Governance model.

DevOps mandates “Fail Fast, Fail Often” driven by “Test First” concept. This means that testing is a continuous process and more extensive in nature. Testing allows for cohesiveness between various teams, automation of everything possible. Testing also allows for continuous improvement in test process delivered via iterations. Intelligent test automation, improved regression, continuous test augmentation are some benefits delivered by DevOps.
- Key benefit delivered byDevOps is the cost reduction in end-to-end delivery.Businesses achieve cost reduction by Automation, manual error reduction and reduction of duplicate effort.
  - Continuous delivery enables streamlined governance and reduces time-to-market.
  - With fail-first, fail-fast, fail-often philosophy, shift-left testing is a reality. With early defect identification, product quality increases. With increased automation, continuous testing is a feasibility.
  - With co-location, productivity of DevOps teams (design, test, operations) improves.
Implementing DevOps could be tricky in organizations having monolithic systems and processes. Culture change is critical for successful initiation of DevOps and Continuous Delivery.

From a testing standpoint, DevOps addresses the following key testing challenges.
- Test Environment – Virtualization, Automated Deployment, Automated build and configuration management, data refresh and maintenance becomes simple
- Continuous Integration, Automation and Verification – Test First centric test driven development, behavior driven development concepts allow test processes to improve in a DevOps Model
As per a report published in 2002 by NIST – Cost of Poor Quality (in US) was $59.5 Billion per year. Cost of poor quality is higher in large organizations running legacy infrastructure. By some industry guesstimates average cost of a production bug is about $7000. Customer experience and agility for quick delivery takes a hit if the organizations aren’t quicker. DevOps becomes a positive catalyst here.

Some skeptics may say “DevOps looks easy on paper but difficult to put in place”. Any innovative change is difficult to implement at first. It takes discipline and drive to run the operations watertight to achieve the goals.

Some key factors to implement for Successful DevOps are
- Cohesive, One Team mentality between teams
- Implementation and Use of Tools that take away manual effort
- Continuous Delivery Phases
  - Superior, Automated Configuration Management Approach
  - Continuous Improvements across phases
  - Continuous Automation
  - Continuous Integration
  - Continuous Testing
  - Continuous Monitoring
Use of tools is key to success of a DevOps program. Plethora of tools are available to support automation. Tools are available for Environment Build, System Configuration, Application Build, Database Setup, Artifact/Document Generation, Testing, Requirement Management, Collaboration, Production Monitoring, and Reporting across DevOps activities.

Like Agile Delivery Programs DevOps needs a strong Team Cohesion to be successful. A DevOps team need to have a Product Life-cycle Management mindset as well. This also means that team needs to unlearn standard SDLC processes. This means, DevOps team should have a clear vision. Vision to Ideate, Implement, and improvise and retire the product. An integrated DevOps team with clear vision will make DevOps projects successful.

Conclusion:

DevOps testing needs to address the need of Business and Technology needs. Automation plays a major role to achieve this aim. DevOps in principle has a goal for better product quality albeit the need for faster delivery. Terms such as “Shift-Left”, “Fail-First, Fail-Fast, Fail-Often” will be regular vocabulary in DevOps. DevOps Test team comprises of Cross Functional Specialists and Testing Purists. A Performing Testing Practice for DevOps will take some time. Testing as a practice has matured over the past two decades and is well equipped to support DevOps. Use of automation and productivity enhancement tools helps testing move faster.

Successful DevOps implementation in large firms with legacy processes and technology is a challenge. A big culture change is the only way to achieve success in such firms. It will be a hybrid DevOps Model as purist implementation will be a challenge.
August 21, 2015