devopsy.kinsta.cloud

Category: Continuous Delivery

Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.

  • The Top 10 Reasons to Use Cassandra

    The Top 10 Reasons to Use Cassandra

    The Apache Cassandra database is the right choice of database if you are looking for scalability and high availability without compromising performance for your mission-critical applications. Additionally, Cassandra’s support for replicating across multiple data centers is best-in-class, providing lower latency for users and the peace of mind of knowing that you can survive regional outages. To that end, we’ve compiled a list of the Top 10 reasons why enterprises onboarding and deploying mission-critical applications should use Cassandra (Apache Cassandra, DataStax Enterprise).

    1) Requirement for fast writes: Easily deals with data velocity, data variety and data complexity issues

    Many of the challenges associated with next-generation cloud applications center around data volume and data velocity. Is Cassandra able to handle the speed of data coming into the system? The answer is “yes” based on the amount of data and cluster size. Not only does Cassandra come with this ability out of the box, but there are systems of data pipeline architectures being built around ingestion speed. And to top it off, it scales linearly, making it easy to determine the right amount of capacity based on data flow.

    But, there are also two often overlooked components it provides: 1) data variety and 2) data complexity. Data variety is an alternate way of saying that data coming into one database can come in different forms. An example of this would be sensor input from a heart monitor and sensor input from an IV both for the same patient. The second component, data complexity, extends the previous example. The heart monitor might report 100 metrics twice per second under normal operating circumstances and up to 125 metrics once per second while the wearer is sleeping. This means the write patterns, locations, and frequencies can vary. Cassandra handles these situations gracefully.

    2) Can handle massive data sets

    If there were any questions about whether Cassandra is capable of handling large data sets, there is no need to look any further than the companies using it. They operate at massive scale—Netflix, Hulu, Instagram, eBay, Apple, and Spotify all have Cassandra working in interesting ways as part of their offerings.

    The other way you know Cassandra is up to the challenge is in use case examples. Many organizations use it for applications where data grows in an unbounded way very quickly. These include Twitter clones, a web log analytics data warehouse and telemetry or sensor data.

    3) Homogeneous environment

    Unlike some of the legacy distributed systems, Cassandra does not require outside support for synchronization. All of the required components for basic operation are built in. Since Cassandra also operates in a peer-to-peer fashion, this means that there is no master-slave or sharding setup and that all nodes in the ring are equal. Additionally, there is only one machine type that an administrator needs to worry about.

    4) Highly fault-tolerant

    Cassandra employs many mechanisms for fault tolerance. Since it is masterless, there is no single point of failure. There is also the potential for zero downtime rolling upgrades. This is because Cassandra can support the temporary loss of multiple nodes (depending on cluster size) with negligible impact to the overall performance of the cluster.

    The safety net Cassandra offers extends outside of your data center as well. Cassandra allows you to replicate your data to other data centers and keep multiple copies in multiple locations. This helps satisfy many regulatory requirements in addition to being a part of an strong disaster recovery and business continuity plan.

    5) Proven success across enterprise applications and in many use cases already

    There are already many examples where Cassandra is being used effectively. Banks and other financial institutions are storing large quantities of financial data in Cassandra. Analytics companies are using Cassandra to store web analytics data. Medical companies are using Cassandra to store sensor data and other time series inputs. There are also many companies making use of Cassandra for storing internet of things (IoT) data.

    6) Ease of administration

    Cassandra is a straightforward system to administer. With Cassandra being a masterless system, all nodes in the ring are the same—a homogenous system. It’s fault-tolerant and can support the temporary loss of nodes with minimal impact to production performance. This means that nodes are easy to replace and the requirement to replace downed nodes immediately isn’t as strict.

    7) Custom tuning

    There are a lot of knobs and levers that can be turned to get Cassandra to perform optimally for your workload and environment. You can set it up to operate in a way that is consistent with your workload. For example, if you write lots of log data and read infrequently, then there are configuration tweaks to be made for write-heavy systems. If you write heavily to one data center and then do all your reading from another data center, then you can adjust the settings on a data center-by-data center basis.  This idea of tuning isn’t just available at the application level; you can also tune the JVM and Java settings, including GC and logging levels. Changes can even be made by the drivers at connection time to aid in the performance of your system.

    8) Easy to integrate core applications

    A lot of work has been done on data manipulation and parsing systems to integrate with Cassandra. For instance, the full text search engine Apache Solr has been packaged to work with Cassandra to provide full featured search capabilities to an existing Cassandra database. Apache Spark, a big data analytics engine, also has been plugged in to work on an existing database. There are entire suites of tools that can be integrated or bolted on to increase its capabilities. These include Apache Mahout, Kafka and Zipkin, to name a few. This is important because the more tools you have available to you, the more powerful your data becomes. You also have the ability to gain more insight about your data without having to build and maintain the application systems that were previously required.

    9) Excellent Monitoring Options

    Included in the system of tools referenced in No. 8 are monitoring packages. If you are a user of automated monitoring platforms such as Datadog or Netuitive, you’ll find examples of prepackaged agents to monitor the important parts of Cassandra. You can then tack on your own additions of other metrics that are important to you. This is made possible by Cassandra taking advantage of Java MBeans and exposing them to the client. You can use these to get at much of the internal information Cassandra uses to make its own decisions and decide on its own health. Datastax also offers its own monitoring and control application called Opscenter.

    10) Amazing community

    One of the best things about any piece of software is having a great community of developers and experts available to you for help or guidance. There is a huge yearly database summit put on by Datastax, the primary backer and largest contributor of code to Cassandra. It also sponsors community events all over the world so you can meet and interact with other developers around you.

    For open-source software to be successful, there needs to be an ecosystem that develops around it. In the case of Cassandra, there are consultancies, monitoring and troubleshooting systems, plugins, instrumentation systems and backup systems. That is all a set a competencies that your organization no longer needs to own and can use what the greater community has already created. There are even PaaS companies that will completely take over the management for you, leaving you just the development of your application to focus on.

    Given the sizable number of organizations and people that are a part of the ecosystem and the Cassandra community as a whole, there is no shortage of articles, documentation and people willing to help. A welcoming and helpful community isn’t always a given, but in the case of Cassandra, it’s alive and thriving. This is important because software is always about people.  The more of them that you can interact with that have shared your experiences, the better. It also will be easier to find solutions to your problems having a network of people who might have faced them before.

    There are many reasons why Cassandra could be the right tool for your application. Knowing your systems requirements, workloads and future will help you make the right choice. As you can see, if you do choose Cassandra, you are bound to be in good company.

    — Michael Colby

  • Boots and Sandals: Know the Market

    Boots and Sandals: Know the Market

    Boots and sandals are basically the same thing—something to protect your feet while you walk about. The thing is that they are intended for slightly different purposes. You wouldn’t wear sandals outside in the middle of a Wisconsin winter*, and you wouldn’t wear boots to your day off, hanging at the beach. The thing is that boots and sandals have been around forever, and it is relatively simple to gauge the merits of each in a given setting. The same is not true of DevOps tools.

    Cowboy Boot Sandals.

    * Every few years there is a news report about someone found only in sandals in sub-zero weather, they’re not the norm, and there’s a reason they’re in the news.

    Rate of Change

    The rate at which DevOps markets are maturing is a pace that few enterprises can keep up with. When both development processes and operations processes are changing, new tools are being introduced and the two groups are increasingly working in tandem to support the business, yet another market space is not what they want to study and yet another tool in the tool chain is not something that they want to just “pop in.”

    But the DevOps space is young, and there is a lot coming that can help achieve the goals. I have had reason to look into application release automation (ARA), cloud security auditing and global file systems (GFS) in the last few months, and all three bring something to the table in terms of helping IT adapt and automate. The problem is that in each case, the market is a bit muddied. ARA must explain why it does more than Jenkins alone, cloud security auditing must explain why it is better than calling the cloud vendor APIs to do the same task, and GFS must explain why it is more than a cloud storage gateway and how that helps the enterprise.

    And they can. But when considering them, you need to do the research and figure out the market and how that market applies to your organization. Which is a bit unfortunate. devopsy.kinsta.cloud and other sites can help by talking about the tools over time, and the vendors’ own promotional pages can help you understand what gap they’re filling, but it’s legwork (though, in many cases, the legwork is worth it). ARA marries app provisioning tools with CI tools, giving a unified workflow, for example. And it makes the deployment process more agile. If you are running many projects that deploy on a regular (not hourly or even weekly, but regular) basis that you may one day want to deploy to a different environment, ARA can help, along with the wealth of other functionality it provides, bridging the two sides of the DevOps coin is probably its largest driver. The same type of thing is possible with the other two toolsets I mentioned, but trying to keep this short enough to read, so will leave it to those interested in GFS or Cloud Security Auditing to research them.

    The Key is Knowing Why and Which

    In the end, if you’re building out DevOps, you want to automate the process as much as possible. That means as the markets mature, you will want to explore what’s new. These tools can help many organizations provide a smoother or more secure process, but knowing the reason the market exists is the first step, and only then can you decide if you want to evaluate products. They do not all help all customers, so seeing which are right for you—there is no need for either GFS or Cloud Security Auditing if you are a single office with no cloud presence, for example—will help you narrow which toolsets might improve your DevOps efforts.

    And whatever you do, don’t buy the “sandal cowboy boot” of IT. It might look good, but won’t be very useful in either of the environments it’s intended for. 😉

    — Don Macvittie

  • Metrics Only Matter When They Matter

    Metrics Only Matter When They Matter

    Many studies have shown the significant value that DevOps can bring to a software organization. However, organizations on their own DevOps journey need more than general industry studies or case studies. Any practitioner needs to be able to show the value—specific to their organization—that the changes to people, process and tooling made in the name of DevOps are having meaningful results.

    Measuring the success of a DevOps practice or project is critical to securing and maintaining support for similar projects in the future.

    In a previous blog, I talked about some of the pitfalls that come from focusing on the wrong DevOps metrics and outlined a few general guidelines around what makes a good metric. Here I’ll expand on that theme and discuss more specifically what types of metrics can help you show progress within your organization.

    To review, in the previous blog we looked at the characteristics of a good metric. Specifically, good metrics should be obtainable, reviewable, incorruptible and actionable. With those as general guidelines, it’s helpful to consider the three core components of successful DevOps implementation to help us determine where to look at measurements.

    DevOps is built around people, process and technology. But it doesn’t make sense to measure the components of DevOps; instead, we need to think about the goal, or outcome that each of these achieves. For example:

    • People changes are focused on improving culture, collaboration and sharing
    • Process changes are largely implemented with the goal of driving efficiency & effectiveness
    • Technology changes contribute to efficiency and effectiveness, but primarily drive quality and velocity

    Together, all three of these should lead to significant customer and business value.

    Notice how certain words above are in bold? That’s not an accident. These four key areas are the foundation of a successful DevOps practice—and therefore an excellent place to focus your measurement efforts. The figure below illustrates how these four areas fit together, and provides a few examples of the types of things that can be measure in each area.

    Of course, it’s important to consider your own organization when considering what metrics to focus on, as well as the guidelines mentioned previously. For example, in some organizations people-related metrics could be difficult to obtain on a regular basis. That’s not to say they’re not important—it’s been shown that DevOps can have a meaningful impact on employee satisfaction—but these metrics may not be the primary focus for some groups.

    DevOps has a wide range of benefits and I’ve seen organizations often have their own primary motivation, whether it is costs, velocity, competition or something else. It’s best to focus on metrics that are aligned to your organization’s primary objectives.

    Metrics matter only if what you’re measuring matters to your organization.

    If you are interested in learning more about DevOps metrics, the book “DevOps for Digital Leaders” explores this topic in more detail.

    In my next blog, I’ll delve into what to do with the metrics, once you’ve gathered and analyzed them. Because even the best metrics are only as good as what you—and your organization—can learn from them.

    — Aruna Ravichandran

  • Where Does the Database Fit into Continuous Delivery?

    Where Does the Database Fit into Continuous Delivery?

    Continuous delivery (CD) is increasingly becoming the norm in software development, with companies releasing updates more and more frequently. Those that aren’t are exploring how they can catch up. But in all of this push to DevOps and CD, it’s vital not to forget the database. Changes to applications or web services often require changes to the database, which means companies need an end-to-end process that spans both application and database development if they want to successfully embrace CD.

    However, many companies still work in silos, with applications and databases developed separately. So while the ability to deploy application changes across environments is relatively common, supporting databases in a truly elastic environment remains a major challenge for organizations.

    To find out the current state of database DevOps, and how it can be included in CD, Redgate surveyed 1,000 SQL Server professionals across the world. While we found companies were keen on implementing DevOps, with 80 percent saying they will have adopted it within the next two years, only 20 percent are currently applying DevOps practices such as version control, continuous integration or automated deployment to their databases.

    The Need for Change

    This isn’t because companies don’t recognize the need for database DevOps—37 percent said they are deploying database changes either more than once a week or daily, demonstrating a requirement for automation. Indeed, the main driver for adopting DevOps in this area is to increase the speed of database changes (cited by 26 percent of respondents), followed by freeing up developers for more added value work (17 percent).

    Companies also recognize the drawbacks of continuing traditional, siloed database development practices—26 percent said it increased the risk of failed deployments or downtime when introducing changes. This was followed by slow development and deployment cycles and an inability to respond quickly to changing business requirements (both 18 percent). Sixteen percent also cited poor visibility of the change management process as another issue. This shows the vital importance of fast, successful database changes and deployments to business today.

    For companies with 5,001-10,000 employees, slow development and release cycles were seen to be more of a drawback than application downtime, while for organizations with more than 10,000 employees, the inability to respond quickly to changing business requirements was deemed to be a slightly more significant downside.

    Moving to DevOps for the Database

    As well as understanding the business need, the survey showed that organizations are now better structured to enable continuous delivery for database changes. The old picture of companies made up of silos of developers and database administrators that don’t work together, for example, is far from the truth. Seventy-five percent of companies had teams that included developers who were working across databases and applications, with just 25 percent having dedicated database developers. In 60 percent of companies, developers were responsible for building database deployment scripts. Fifty-eight percent of companies said the integration between the two was either good (38 percent) or great (20 percent), meaning teams work together on a planned or ad hoc basis. Just 12 percent classed integration as poor, with DBAs working in silos.

    However, the research highlights two challenges to overcome:

    Different processes and tools—Despite the levels of integration within organizations, there is still a gap between how companies operate and automate when it comes to applications and databases. For example, while 81 percent of respondents use version control for their applications, just 58 percent apply it to database development. Thirty-nine percent use continuous integration for applications, yet only 20 percent apply the same practice to their database. Clearly, there is still more that can be done to bring the two together through common practices and the use of third party tools.

    The skills gap—The greatest obstacle to moving to an overall DevOps way of working was a perceived lack of appropriate skills (cited by 21 percent of respondents), ahead of a lack of alignment between the development and operations teams (20 percent). Only 9 percent said that their executive leadership didn’t support the shift.

    When it came to integrating database changes into the DevOps process, consistency was top of the list of obstacles, with 32 percent believing that synchronizing application and database changes would be the biggest challenge, followed by overcoming different approaches to application and database development (25 percent). Perhaps unsurprisingly, 19 percent also mentioned preserving and protecting business critical data as the largest obstacle to overcome.

    It’s Happening Sooner Than You Think

    The good news is that many companies are confident they can quickly overcome the process and skills challenges. When asked how long they estimated it would take their organization to move from traditional database development practices to a fully automated process for deploying database changes, more than half (51 percent) said it could be done within a year, with 20 percent believing it could be achieved in six months.

    Digging down into the data, those that have already adopted DevOps in other processes are much more confident the switch can be achieved quickly. Sixty-eight percent of this group cite 0 to 12 months as the time it would take to move to a fully automated database development process, compared to 28 percent of those who had no plans to adopt DevOps practices. In fact, 43 percent of this latter group cited more than two years as their estimate.

    Our research shows that organizations recognize the potential of applying DevOps to the database to gain all of the advantages CD brings. They see that it speeds up software development, frees developers to do more value-added work and increases the rate at which database changes are delivered.

    At a time when organizations need to operate in a faster, more seamless manner to thrive in ever more competitive markets, introducing DevOps and CD to the database can transform their development process. To read more about the research and the “State of Database DevOps,” download the full management report.

    About the Author  / Matt Hilbert

    Matt Hilbert is a technology writer at Redgate Software with 20 years’ experience working for lots of the world’s biggest tech companies – and many of the smallest. He has a particular fascination for emerging technologies and is on a continuing mission to decompile, untangle, and explain techspeak to a wider audience, and excite people about the endless possibilities technology offers.

  • In the Application Economy, Every Company is a Software Company

    In the Application Economy, Every Company is a Software Company

    Recently, I was in New York to attend the Third Annual DevOps Virtual Summit. (Yes, I had to be at a “virtual” summit in person—while it’s virtual for the attendees, it occurs in real life for the speakers, panelists and support staff.)

    The broadcast took place in a small recording studio that featured a lobby adorned with autographed photos from celebrities who had used the facilities previously. Sylvester Stallone, Al Pacino and even John Stamos—complete with a full mane from the “Full House” days—were all looking down on us from the walls.

    Anyway, as the speakers were introducing themselves in this lobby prior to the event it occurred to me that the sampling of speakers was illustrative of the importance of DevOps across industries. The companies participating in the event were not what you’d traditionally consider a “software business,” but yet they’d made an investment in their software development practice—recognizing the importance of competing in the application economy. It underscored the point that in the application economy every company is a software company, regardless of what industry they are in. Which is why the concept of DevOps and Continuous Delivery should be top of mind, even if your company may not see itself as being “in” the software business. Regardless of what end product your company sells, you rely on your software and applications to do so.

    Take a look at the panelists and the companies they represent to illustrate this: Silvia Prickel from United Airlines. Matt McComas from GM Financial. John Hashimoto from CNN. And Kelly McCartney from Jeweler’s Mutual Insurance Company.

    An airline, an automobile financing company, a news organization and an insurance company that’s been around for 100 years (literally). These are not companies that were born digital, like many of the web-based entrants you might typically associate with DevOps. Instead, these are traditional organizations that recognized the need to make a digital transformation.

    And the results were pretty impressive. We often focus on speed when we talk about continuous delivery, but it’s important to remember you also can see improvements in quality. The below image was taken from the United presentation and shows the dramatic decline in production outages.

    This is not to say that improvements in speed were not seen as well. Matt McComas from GM Financial spoke about how his company is now averaging 1,800 deployments per month, with each one taking just 3 to 5 minutes.

    I won’t give away any more important details here. If you’re interested to find out more, I encourage you to watch the replay of the webcast: http://bit.ly/DOVSreplay

    — Brendan Hayes

  • Beware False DevOps Metrics

    Beware False DevOps Metrics

    Metrics are important for any manager seeking to continuously improve critical work processes and the resulting work-product.

    That’s why DevOps leaders need DevOps metrics. With the right ones, those leaders can guide their organization’s adoption of DevOps best practices—progressively optimizing staff productivity, business agility and customer experience.

    But, what are the right metrics for DevOps success? And, what are the wrong ones?

    These next few blogs will offer field-proven guidance on DevOps metrics. Much of the material in these upcoming blogs can be found in the book, “DevOps for Digital Leaders,” which I co-wrote with Kieran Taylor and Peter Waterhouse. And I’d love for you to comment below so we can all share lessons learned about how to effectively measure DevOps maturity—and, just as important, how not to measure it.

    Change Your Metrics, Change Your Culture

    DevOps leaders tend to struggle with metrics because DevOps entails changes that are both operational and cultural—and span different teams or functional areas, including development, test and operations. So inexperience may cause leaders to adopt new ones that are counterproductive—or, as is often the case, continue using traditional ones that don’t align well with the cross-functional nature of DevOps objectives.

    Three pitfalls have proven to be especially common in the transition to DevOps:

    • Vanity metrics. These appeal to the “hero” mentality of individual coders, rather than the collaborative process goals of cross-disciplinary DevOps and continuous delivery teams. They include total lines of code produced and total function points created. If incentives are linked to these metrics, developers can focus too much on volume at the expense of other essential activities such as refactoring and design simplification.
    • Those that promote interteam conflicts. Metrics that pit teams against each other undermine critical DevOps behaviors—such as code-sharing, mentoring and constructive peer reviews. So, while gamification may be useful in other areas of the business, Agile leaderboards that rank teams based on the prevention of extraneous deployments and changes can actually work against DevOps culture.
    • Traditional metrics. Many DevOps leaders persist in utilizing traditional/incumbent ones such as mean time between failures (MTBF) and the ratio of full-time equivalents (FTEs) to servers. But while these may have been useful in the past, they can work at cross-purposes with DevOps transformation. After all, with faster delivery of services, some failure is to be expected. And as IT improves its responsiveness to the business, it must look past costs alone to total economic impact.

    What Makes a DevOps Metric Useful?

    While future blogs will propose some specific metrics that DevOps leaders should find useful in their efforts to bring about cultural transformation, here are a few general guidelines to consider:

    They must be obtainable.

    A metric won’t do you any good if you can’t actually capture it. So you need DevOps tools that help you capture the metrics you need. You may also have to measure some attributes indirectly. For example, rather than measuring employee morale directly, it may make sense to use staff retention rates—which you can readily capture—as your performance indicator.

    They must be reviewable.

    Any metric you choose to employ as part of your DevOps leadership effort must be able to stand up to rigorous scrutiny in a business context. Your credibility as a leader depends on your ability to communicate its relevance to the stated values and mission of the team.

    They must be incorruptible.

    It’s never a good idea to promote metrics that can be influenced by individual team members or any group as a whole. You should also be on guard against metrics remaining entrenched—despite the fact that they may inhibit change—because bonuses are linked to SLAs associated with them.

    Metrics must be actionable.

    Useful metrics must enable DevOps leaders to make better decisions about workflows, incentives, policies, training, tools or some other “lever” of transformation. Not every such decision will be a game-changing one—but incremental gains in knowledge lead to incremental improvements in process, which collectively do change the game.

    In my next blog, I’ll discuss the metrics that DevOps leaders are using today to successfully drive the digital transformation of their businesses. In the meantime, if this is a topic that interests you, be sure to request a free copy of my book, “DevOps for Digital Leaders.”

    — Aruna Ravichandran

  • DevOps in Legacy Systems: A Mission Impossible?

    DevOps in Legacy Systems: A Mission Impossible?

    The word “legacy” evokes mixed reactions from the IT and business crowds alike. For those who depend on legacy systems such as the zSeries and iSeries platforms, it is a way to deliver stable and reliable applications into production. Business stakeholders who may not know the intricacies of the technology often take a view of, “Why fix it when it ain’t broken?”

    mainframe

    On the other hand, for those who haven’t experienced the joys and pains of legacy systems , legacy world will continue to defy their logic and assumptions because their view of the world is shaped by the open system world which has completely different characteristics: being more malleable to change and easy to maintain etc. Hence there is this great divide between the legacy and new worlds. Add Agile and DevOps to the equation with the lofty goals of increased release frequency and reduced lead times, we end up with a divide that is too far to bridge.

    cobol code

    The likes of Gartner have come up with constructs such as two-speed delivery to help deal with this legacy challenge. In my view, there is a problem with the idea of dual-speed because it is based on a wrong analogy to start with — that of allowing vehicles to move at different speeds depending on whether you are on the motorway or a city road. That analogy works well for traffic management scenarios but not for complex IT architectures we see in the enterprise. The dependencies across the systems in the enterprise landscape are just too intricate and too tightly coupled to be defined using such a binary logic. In my experience, I find that many Agile and DevOps transformations hit the roadblock and never achieve the desired levels of productivity and effectiveness because of an oversimplified view of these complex dependencies across legacy and open systems.

    The three biggest challenges that prevent a legacy DevOps transformation are mindset problems, architectural constraints and tooling limitations.

    The Mindset Problem

    “There are no legacy systems, just legacy thinking.” That’s a great quote I heard recently from an engineer in a large organization trying to adopt DevOps at enterprise scale. The initial resistance from teams to move into DevOps ways is due to reluctance to change, emerging from an inertia of doing things a certain way for years if not decades. The mindset problem is as much a problem for those outside the legacy world as those inside. The “outsiders”—typically leadership, management team and consultants—tend to oversimplify the problem and, too often, rely on examples from the open systems world. This just alienates the incumbent legacy teams and makes them even more skeptical of DevOps driven change. The outsiders need to have a genuine appreciation of the challenges with legacy architecture and need to work in collaboration with the legacy teams if they really want to transform their organization.

    Architectural Constraints

    Legacy systems have certain characteristics such as being static, tightly coupled and monolithic blocks of procedural code. These systems were not built for the modern Agile workflows that focus on incremental and iterative deliveries. A significant majority of these systems are written to operate in batch mode.Testing the systems requires cycling through multiple business days, which simply limits the level of agility you want to introduce in the organization. I am currently working with a legacy engineering team in the financial services space where you have to cycle through a minimum of 20 business days to simulate even 70 percent of the test scenarios. However, there are a number of solution options to overcome the architectural challenges; they range from killing applications to migrating to modern platforms or re-engineering legacy code. Not surprisingly, the right approach depends very much on your context.

    Tooling

    Tooling in the legacy world to support a DevOps ways of delivery has always been a challenge. But it is fast changing now, thanks to a number of startups and even old-school IT product companies investing in this space. If you consider zSeries as an example, there are better IDEs (integrated development environments) such as RDz that can replace traditional code editors typically based on “green screens.” These IDEs provide great features such as better navigation, debugging, split views and searches on the legacy code. One of the CIOs I worked with made it clear to his team that he wanted to get rid of “green” screens from the IT floor. There are tools such as SmartBear Collaborator that support a great peer review experience and help bring one of the original XP practice of pair programming to the legacy world. There are tools such as endeavor from CA that promise automated build and deployments in the z-series world. Similarly, there are tools that support test automation of mainframe applications and tools for code coverage and code quality checks on legacy programming languages like COBOL. XATester looks like a promising tool for unit test automation from a Danish startup called Xact.

    While this write-up is not intended to be a compendium of all DevOps and engineering tools, the central message is that there are a number of DevOps tools emerging on the legacy landscape which can help your legacy delivery teams get on to the DevOps and Agile journey.

    What is more important is for organizations to realize that many of these legacy systems will remain part of their IT landscape for the next foreseeable future and it is time to invest in these core systems. They may not look as appealing as the digital platforms, but they are fundamental for businesses and you cannot ignore them on your DevOps journey. DevOps in the legacy world is not going to be smooth, but it is certainly not a mission impossible!

    — Aditya Vadaganadam

  • Help Us Understand CI/CD, DevOps & Job Scheduling

    Help Us Understand CI/CD, DevOps & Job Scheduling

    While plenty of vendors tell us what they think is happening in the market, the only way to really learn is from you, our readers. So we are asking you to take a few minutes to take this important survey on the adoption of CI/CD, DevOps and job scheduling.

    We will be giving a $50 Amazon gift card to one out of every 50 respondents. More importantly we will be publishing the results here on devopsy.kinsta.cloud to share with everyone. The survey is sponsored by BMC and co-produced with devopsy.kinsta.cloud.

    You can take the survey right here:
    Create your own user feedback survey
    Thanks for taking the time and helping us help us all.

    — Alan Shimel

  • CD 101 Podcast: How to Build the Modern Software Factory with Agile Development and Continuous Delivery

    CD 101 Podcast: How to Build the Modern Software Factory with Agile Development and Continuous Delivery

    And now, for Episode 3 … you decide if it beats “Star Wars – Revenge of the Sith.”

    I think we accidentally just set the bar pretty low, but …

    … here we go. Let’s begin with a prerequisite pithy quote from an objective industry observer:

    “If agile software development was the opening act to a great performance, continuous delivery is the headliner,” according to Kurt Bittner of Forrester Research.

    Kurt explains this concept further by saying, “The pace at which consumers expect change is causing a crisis in application development and delivery … agile was a good start, but it wasn’t sufficient to drive better business results; continuous delivery forges the broken link in the value chain, connecting business strategy with business results.”

    So there, you have it.  At least according to Forrester, continuous delivery (CD) is one of the most important evolutions of agile methodology within the modern software factory. But, there’s more.

    Andrew Binstock from Dr. Dobbs fairly well echoes this sentiment, saying: “It’s becoming clear that agile alone is no longer sufficient to keep up with the demands of the present day … software complexity and the need for constant updates are making it difficult for leading edge organizations to keep up without moving beyond agile and embracing continuous delivery.”

    Episode 3 of our podcast series discusses the importance of Agile Transformation along WITH Continuous Delivery to automated some of the development, test, and release efforts to remove some of the bottlenecks downstream in the software development life cycle in the build, test and release phases.

    Lean Agile + CD can create these 1+1=3 types of results.

    Episode 3 of the Continuous Delivery 101 Podcast can be found by:

    • Listening to it on your computer on YouTube
    • Downloading it from iTunes store
    • Downloading the podcast from your Podcast App by searching for “Continuous Delivery 101 podcast”

    Thanks for listening!

    — Scott Edwards

  • Digital Transformation and the DevOps Chaos Theory

    Digital Transformation and the DevOps Chaos Theory

    With the threat from agile startups disrupting markets across the globe, large enterprises are increasingly focused on moving to a more agile approach to business. Most organizations already have begun the digital transformation (DX) journey, but the fact that this transformation is still in its infancy and not completely finalized for many demonstrates the fluid and evolving nature of DX. Multiple IT technologies, processes, applications, systems and protocols need to be adopted and updated on a regular basis for businesses to keep abreast of changes. This does, of course, result in significant disruption for all involved.

    With this in mind, DevOps principles are beginning to have much greater impact, with the true value of this methodology being identified through a new approach. As development velocity increases, and the scale of the enterprise also increases, businesses will develop a much greater reliance on DevOps principles to rein in the chaos associated with continuous development that’s being spurred on by the pace of digital service development and increased automation.

    The key to managing and minimizing the resulting chaos is close collaboration and communication between the IT team members responsible for service development and delivery. Without this, the need for a continuous delivery pipeline operating at speed and scale, will quickly get out of control.

    The DevOps Chaos Theory

    To understand the challenges in this new business environment, it is useful to look at this at a more granular level, through the framework of the DevOps Chaos Theory.

    The pace of innovation is measured as the Velocity (V), or the number of new software releases deployed in a production environment in a defined time period. The Scale (S) factor is measured as the overall number of IT staff involved in service delivery and management in production environments, such as DevOps, SecOps, QA, system architects, DBAs, NetOps and help desk. Interaction between these team members brings the potential for miscommunication, which will increase the overall chaos. The maximum number of interactions between these IT members is S * (S – 1)/ 2, and for high-scale organizations it approaches S2/2. Based on these considerations, a logical hypothesis would identify the system-level Chaos (C) in production environments as C = K * V * S2.  K is the normalization factor that may change based on the overall adoption of DX in a specific industry and the effectiveness of collaboration and communication between the IT team members.

    Minimizing Disruption

    Collaboration between departments within the business is important to ensure the chaos is controlled. There is no doubt that automation tools play an important role in continuous delivery and allowing organizations to operate at the speed and scale they need to. However, enterprises must identify the level of constraint placed upon the IT operations team. This is crucial to establish what changes need to be made and what service performance management technology must be introduced, to prevent operations from becoming a bottleneck to the continuous service delivery cycle inherent to DX.

    Key performance indicators (KPIs) are critical when it comes to service delivery, particularly when you consider that this combines the entire stack into a single system. This stack is comprised of physical, virtual, wired and wireless layers of compute, storage and networking infrastructure and application that run at the top of the stack to deliver services consumed by end users. Using a variety of network, infrastructure and application performance management tools will obscure the system-level view across all the layers of the service stack and their interdependencies. Monitoring system-level KPIs requires access to reliable data sources, such as network traffic. An effective instrumentation of these data sources will play a key role in proactively identifying the root cause of service issues and thus reining in chaos.

    Enterprises must be able to effectively analyze the monitored data to gain insight into all the infrastructure subsystems and applications interdependencies to establish a comprehensive view of their services, accessing both real-time and historic information. In addition, effective management at a human level also should form an important part of a company’s DX strategy, if chaos is to be mitigated and crisis averted.

    More and more companies are realizing that DX offers a wealth of opportunity, but it’s crucial to understand that harnessing these opportunities requires strategy and forethought. Monitoring and managing systems will be vital, as will DevOps principles, as more processes are automated and data plays an increasingly central role to any company.

    About the Author / Michael Segal

    Michael Segal is vice president of strategy at NetScout. His product management experience spans across 10 years at Cisco Systems, where he managed all aspects of product line life cycles for several successful product lines. Michael’s technical areas of expertise include SaaS/cloud, virtualization, mobile IP, security, IP networking, Wi-Fi/wireless, VoIP and remote access. Michael holds patents in areas of networking and wireless mobility. Connect with him on LinkedIn and Twitter.