devopsy.kinsta.cloud

Category: DevOps Toolbox

  • Top Reasons to Use Node.js

    Top Reasons to Use Node.js

    When considering the architecture of a new web app, it might be a good idea to include Node.js in the discussion as a key part of that architecture. Node.js could provide you with additional speed, can be used on numerous platforms and has an active community from which to get modules and other assistance.

    What is Node.js?

    Node.js is a server-side platform developed by Ryan Dahl in 2009. It was built on Chrome’s V8 JavaScript engine and designed with a non-blocking, event-driven I/O. It is also cross-platform, able to be run on Windows, Linux and OS X. Its non-blocking I/O is due to its use of asynchronous functionality, which keeps the server from waiting on data to be returned before being able to move on to other tasks.

    Why Use It?

    Given the architecture of Node.js, a number of advantages are provided both to developers and the resulting apps they develop. A few of these advantages include speed, programming language consistency, real-time apps developed easily and a large number of modules/packages for developers to use while creating new apps.

    Speed

    Node.js offers speed advantages in many cases. First, it uses the Google V8 engine, which is optimized for JavaScript speed. Also, its use of non-blocking asynchronous behavior can ease the time it takes for users to see actions as complete, since things can be going on in the background while the user moves on to other tasks.

    One Language

    JavaScript is listed as the most popular programming language. Source: Thenewstack

    Using Node.js, front-end and back-end code can both be in JavaScript, so your developers don’t have to transition between languages as they move through the stack. Some architectures, such as the MEAN stack (MongoDB, Express.js, Angular.js, Node.js), are designed around this principle. In this case, the stack uses JavaScript for Express, Angular and Node. MongoDB uses BSON for database queries, which uses the same syntax as JSON (JavaScript Object Notation).

    Seeing full-stack developer as the largest occupation and JavaScript as the top language among both back-end and front-end aligns with what we’ve been seeing in overall JavaScript adoption.

    Source: Mikeal Rogers, Node,js Foundation Community Manager 

    For developers, this can help both with moving across the stack during development as well as help communication among developers, as they will be referencing the same language (JavaScript) while discussing programming decisions.

    Real-Time Apps

    Node.js is a great platform for creating real-time apps. With a module such as Sockiet.io, you can create real-time chats, games and more. Along with its non-blocking asynchronous architecture, real-time apps can be developed with plenty of speed and responsiveness.

    Modules

    The Node Package Manager (NPM) makes it easy to install packages that others have developed to perform any number of tasks. This helps to encourage code-sharing among developers and to reduce the amount of custom development that has to be done when one or more packages can take care of that coding for you.

    Notable Examples

    Some notable examples of Node.js use are Paypal and Uber, showing that it can, indeed, be useful for large projects.

    Paypal

    Node.js and an all Javascript development stack helped PayPal bring efficiencies in engineering and helped rethink and reboot product, design and operational thinking.

    Source: Sameera Rao, Paypal Sr. Business Products Engineering Manager 

    Back in 2012, Paypal had a single app running all things, including UI, API and more. This included a lot of duplicated code that was copied and pasted from one app to another to make apps for different locations.

    Using Node.js, the company was able to modularize and clean up its code and processes, which in turn sped the development of apps it needed at any given time.

    According to Paypal’s engineering team, they were able to build the Node.js app with 33 percent fewer lines of code and 40 percent fewer files, and were able to build it twice as fast and with fewer people than before. Certainly, Node.js proved to be advantageous to their development.

    Uber

    Uber used Node.js from the beginning, and its distributed system was a perfect fit, since a lot of network requests are made. The non-blocking asynchronous I/O used by Node.js helped to ensure those requests could be made and handled quickly to provide the best service possible.

    Conclusion

    The benefits of Node.js have been proven. It helps speed development, can be used on numerous platforms and has an active community from which to get modules and other assistance. Include Node.js in your future conversations about website architecture.

    — Brian Wheeler

  • Are You Paying a ‘Wait-Time Tax’ on Your Global DevOps?

    Are You Paying a ‘Wait-Time Tax’ on Your Global DevOps?

    Business increasingly is global, so DevOps has become global as well. In fact, for many enterprises, DevOps entails collaboration between development, test/QA and ops teams across multiple continents.

    This is in part due to the economics of IT labor and in part because applications for global enterprises have to be deployed globally.

    Unfortunately, enterprises executing DevOps globally must confront the physical realities of global code distribution. And those physical realities impose a “wait-time tax” that fatally undermines Agile processes and culture.

    But they don’t have to. There’s a tax-free way to share large volumes of code across globally dispersed DevOps teams.

    The Wait of the World

    Every time you ship a massive build worldwide, you have to wait. Assume, for example, that you’re sending a 10GB build to five different locations over a 10Mbps MPLS connection. If a couple of your remote locations have 5Mbps connections, it will take them five hours to get each build. But if you have three locations with only 2Mbps connections (as is likely the case in Asia), they will have to wait 12.5 hours.

    That wasn’t a problem when you iterated your builds with limited frequency. But as you try to become more aggressive about highly iterative Agile development and testing, this “time tax” becomes punishing.

    For one thing, you lose productivity as your overseas developers and test teams wait idly for new artifacts. And these chronic wait times fundamentally undermine the Agile culture you’re trying so hard to build across your organization.

    In fact, if your current model for distributing code globally remains as-is, you’ll never achieve the speed and iterative frequency your business needs. And you’ll certainly never come anywhere close to continuous delivery.

    Faster Cycles from Dublin to Dubai

    Fortunately, you can eliminate DevOps wait times, regardless of how globally dispersed your teams are and how thin their connections to your enterprise network may be.

    The solution entails a hub-and-spoke model that:

    • Lets you maintain a “gold copy” of your current codebase(s) in the cloud;
    • Provides all your remote locations with their own local codebase copies that get continuously and automatically updated with any incremental changes as they occur; and
    • Leverages each location’s high-throughput internet connection.

    The combination of small, incremental code updates and high internet bandwidth dramatically changes the rules of the global DevOps game, allowing even the most geographically dispersed teams to work with even the largest codebases extremely iteratively and without delay.

    It’s difficult enough to sync dev and test teams on different continents. And the pressures on your organization to respond immediately to ever-changing market and regulatory demands will only intensify. So now’s the time to consider abandoning your obsolete approach to global code distribution and adopt a model that better enables you to be fast, super-iterative and global.

    — Barry Phillips

  • 5 Reasons Why Companies Should Analyze Their Logs

    5 Reasons Why Companies Should Analyze Their Logs

    At first thought it might seem as though analyzing logs is something that only affects the IT portion of your business. However, nothing could be further from the truth. In reality, log analysis affects every aspect of your business. This includes finance, legal, human resources, sales and marketing, security and operations. In fact, with log analysis, you’ll discover issues either before or as they happen and avoid both lost time and unnecessary delays.

    There are many benefits to properly analyzing your logs. Your organization can increase revenue, reduce expenses, better manage employees, respond to incidents and even keep yourself out of legal hot water. Read on to learn about the five major business areas impacted by log analysis and why every business needs to analyze its log—and the sooner the better.

    Business Operations

    Probably the most obvious use of log analysis is in supporting business operations. These functions include the following:

    • Detecting critical system errors
    • Identifying key trends
    • Improving operational efficiency
    • Supporting service level agreements (SLA)
    • System troubleshooting

    Early indication of problems can pinpoint money-wasting processes. It also can mitigate risk in support of business continuity. This will help avoid service disruptions that can lead to unhappy customers and lost sales.

    Compliance and Legal

    Every organization has internal policies: security policies, employee policies, legal policies, etc. In fact, most organizations have regulations (such as HIPAA and Sarbanes-Oxley) they must adhere to. Often these regulations are imposed by the government. The only feasible way to ensure policies are followed and regulations are met is through log analysis.

    Log analysis simplifies compliance. It also supports audit requirements, litigation needs, subpoena requests and forensic investigations. In short, log analysis is the only way to ensure an organization stays out of legal trouble.

    Proper Resource Sizing

    One of the worst things any company can do is have the wrong amount of resources to support its business operations. From network bandwidth to storage capacity to CPU cycles, businesses have to get these quantities just right. Not enough resources in any area can lead to poor performance, frustrated customers and, eventually, lost sales.

    Of course, the easy thing to do is just overspec these resources. However, doing that leads to increased expenses, which adversely impact the bottom line. Using logs to determine proper resource sizing is the only way to ensure you have maximized your profits.

    The use of logs can also be used to verify third-party billing. For instance, your company can use your logs to make sure the amount of bandwidth you were charged for by your service provider is actually the amount you used. This, too, can benefit the bottom line.

    Sales and Marketing

    It may not seem as though log analysis can be used to directly impact sales and marketing, but it certainly can. For starters, log analysis can be used to quantify the effectiveness of a sales campaign. Information such as referring sites, pages accessed and conversion rates all can be used to tweak different aspects of the sales effort.

    Log analysis also can be used to optimize website performance in the support of a sales funnel. Information such as customer navigation, traffic loads and conversion errors can be used to improve the sales process. In fact, analyzing logs can be used as a revenue driver in any organization.

    Security

    One of the most important areas in any business today is security. Logs can be used to both proactively protect an organization, as well as forensically investigate security breaches.

    Proactively, logs can be used to accomplish the following:

    • Optimize blocking and filtering of incoming traffic
    • Maximize email virus protection
    • Monitor failed authentication attempts
    • Verify configurations in firewalls

    After the fact, logs can be used to do the following:

    • Assess security policy compliance
    • Initiate incident response
    • Investigate security breaches

    You cannot understand security risks or respond to security incidents without the ability to analyze your logs.

    In short, whether you were aware to it or not, log analysis impacts every department in your organization. It directly and indirectly impacts every company’s bottom line by increasing revenue, decreasing expenses and delighting customers.

    You now know that log analysis is not just a “back office” IT function. Every department in your company needs to use the intelligence contained in your logs to do their jobs better. It is why every business needs to analyze their logs, and now is the time to start.

    About the Author / Ariel Assaraf

    arielassarafAriel Assaraf is a co-founder and CPO at Coralogix. Previously Ariel was a GL at Verint and a Product manager at an elite Israeli intelligence unit.  He is into Software design, UX, growth hacking and basketball. Ariel holds a BEc in Economics and Math from the Open university of Israel. Connect with him on LinkedIn and Twitter

  • Mainframe Apps and the Long View of DevOps Strategy

    Mainframe Apps and the Long View of DevOps Strategy

    IT is constantly under intense short-term pressures. Line of business (LOB) executives demand a competitive advantage in digital customer engagement now. Marketing leaders demand better customer analytics now. Legal demands better audit reporting now.

    In fact, much of the investment that companies are making in Agile, DevOps and continuous delivery is due to the urgent, short-term demands of the business.

    But short-term actions alone are insufficient for business success. IT leaders must also strategically navigate the vagaries of the digital economy over the long term—or, as Wayne Gretzky’s father taught him, to “skate to where the puck is going to be.” Otherwise, today’s small wins inevitably will lead to tomorrow’s egregious failures and inefficiencies.

    And integration of your mainframe apps and data into your enterprise DevOps tool chain is central to that long-term digital strategy.

    Here’s why.

    The Mainframe: Big, Nimble Iron

    A DevOps tool chain is only as fast as its slowest link. For most large enterprises, that slow link is the mainframe.

    Slow, waterfall-bound and underautomated mainframe dev/test processes are non-trivial—because mainframe applications and data are non-trivial. They typically are an organization’s most important apps and data. So if you can’t modify your mainframe application logic quickly or leverage your mainframe data in new ways easily, you’re uncompetitive toast in today’s fast-moving markets.

    Also, mainframe applications and data typically serve as the back-end systems of record for front-end mobile and web-based systems of engagement. Your ability to innovate and remain competitive when it comes to digital engagement and digital perceived value is, therefore, almost universally contingent on lessons learned from frequent and timely mainframe code drops.

    In other words, everything IT is under pressure to deliver—new mobile apps, better analytics, fulfillment of increasingly stringent compliance mandates, etc.—depends on bringing Agile, DevOps and (ultimately) continuous delivery to the mainframe.

    Just about everyone realizes this. Sure, there are some holdouts who still advocate something they call “bimodal IT”—which prescribes allowing the mainframe to languish in obsolete waterfall and dev/test practices while the rest of the enterprise moves forward. But bimodal lT has proven to be a mistake. An Oct. 31, 2016, Forrester Research report titled, “Predictions 2017: CIOs Push For Speed Amid Volatility,” shines a light on the fatal realities of bimodal IT:

    “In early 2016, many CIOs fell for the false promise of a bimodal strategy. They only pushed part of their team to adopt an accelerated pace — and left others unchanged. But those CIOs are already experiencing the shortcomings of operating at two speeds. Business peer frustration, polarizing and deteriorating cultures, and unsustainable operational complexities will continue, ultimately spurring most of these CIOs to course correct in 2017.”

    Given the explosion of mainframe-related innovation that has taken place in the past couple of years, enterprise IT leaders and industry analysts agree that “mainstreaming” it into enterprise DevOps is the right and necessary strategy (see as examples here and here).

    Some Good Things Take Time

    While instant gratification may be the order of the day—and gain without pain may be an all-too-frequent vendor promise—the reality is that great IT achievements all require effort, investment and time.

    Inclusion of the mainframe into your cross-platform DevOps tool chain is no exception. No CIO can achieve mainframe agility and cross-platform DevOps orchestration by simply waving a magic wand. But there is a clear, practical path to these goals for those with the requisite foresight and leadership.

    That path includes:

    • Modernizing the developer workspace. Most mainframe development is still performed in “green screen” TSO/ISPF environments that require specialized knowledge and limit staff productivity. A modernized, graphical workspace as part of a preferred DevOps tool chain will allow staff at all experience levels to perform dev/test tasks on both mainframe and non-mainframe applications.
    • Graphical visibility into existing code and data structures. Mainframe applications typically are large, complex and poorly documented. These attributes are a major impediment to mainframe transformation. The right tools enable new participants in mainframe DevOps to quickly and easily “read” existing application logic, program interdependencies and data relationships.
    • QA “shift left.” Early detection and correction of quality issues is a central principle of modern programming—and it’s particularly important on the mainframe, where there’s an especially low tolerance for error.
    • Initial Agile training and adoption. Once you put the right development environment in place, you can start shifting your process from a traditional waterfall model with large sets of requirements and long project timelines to a more incremental model that allows teams to quickly collaborate on user “stories” and “epics.”
    • Use operational data throughout the dev/test life cycle. By using operational data in the dev/test life cycle, your DevOps team can ensure that applications perform optimally in production. They also can significantly reduce your MIPS/MSU-related costs by avoiding excessive CPU consumption caused by inefficient code.
    • Agile-enable source code management (SCM) functions. Traditional mainframe SCM environments are inherently designed for waterfall development and thus are incapable of supporting Agile requirements such as parallel development on user stories. The shift to Agile-enabled SCM is, therefore, absolutely essential for mainframe transformation.
    • Automated release management. To keep pace with today’s fast-moving markets, you must be able to quickly get new code into production. That means automating and coordinating deployment of all related development artifacts across all targeted environments.
    • Cross-platform DevOps. Ultimately, you need to fully de-silo your enterprise environment so the mainframe becomes “just another platform”—albeit an especially scalable, reliable, high-performing, cost-efficient and secure one—that can be modified quickly as needed to meet the relentlessly evolving needs of the business.

    ‘What’s My Motivation?’

    With as much as IT departments have on their plates, it’s easy to continue putting off mainframe DevOps integration. But, ultimately, there are only two alternatives to doing so:

    • Re-platform your mainframe applications. This alternative has proven to be a costly and extremely risky undertaking that delivers zero benefits while it consumes resources and potentially disrupts IT operations, or
    • Continue doing nothing. This alternative puts your company at further competitive disadvantage and hastens the day when you’ll experience total mainframe paralysis due to the loss by attrition of all your veteran mainframe developers, QA specialists and operations staff.

    Viewed in this context, genuine investment in the mainframe—whether you call it modernization, mainstreaming or “Millennializing”—is the only reasonable course of action. Such investment starts improving your digital competitiveness immediately, while paving the way for a future where you can make the most of all your resources and confidently transfer platform responsibilities to your next generation of IT professionals. Including the mainframe in your broader DevOps efforts (while extremely valuable) does require commitment, and there are resources available to help. Just be wary of those who may talk a good game, but whose actions don’t really convey a strong commitment to users’ best interests. Fortunately, there are truly dedicated experts who can help every step of the way, and have direct experience modernizing on platform within their own organizations.

    It’s the smart, viable long-term DevOps strategy every CIO of a mainframe-enabled enterprise needs to adopt ASAP.

    About the Author / Christopher O’Malley

    chris-omalley2Christopher O’Malley is CEO of Compuware. He has nearly 30 years of IT experience with past positions including CEO of VelociData, CEO of Nimsoft, EVP of CA’s Cloud Products & Solutions and EVP/GM of CA’s Mainframe business unit, where he led the successful transformation of that division. Connect with him on LinkedIn and Twitter.

  • From the Trenches: Zesty.io’s Google Cloud Migration

    From the Trenches: Zesty.io’s Google Cloud Migration

    Zesty.io recently migrated its sites to the Google Cloud Platform from Rackspace—a process the company believes will make it more agile while streamlining its operations. Zesty.io community manager Phil Ballew recently spoke with Chris Hiestand, Zesty.io’s DevOps leader, to discuss some of the reasons for the migration and get a feel for just what the new cloud system brings to the company.

    Ballew: So what exactly is Google Cloud, and how does that compare to Rackspace?

    Hiestand: Google Cloud Platform is similar to AWS, Rackspace or Azure; they manage infrastructure and services to host your applications and data on. With Zesty.io being SaaS, there are servers running the user’s website that we will manage, and these names are the companies that provide the infrastructure.

    Ballew: So what about Google Cloud made you choose that over Rackspace?

    Hiestand: The difference is that Rackspace has been around for a long time and has produced great business tools, but Google’s tools, which they have been using internally for a while and are now making publicly available, will fit the needs for Zesty.io as we scale out at a rapid pace. Some tech that Google created about 10 years ago is considered cutting-edge today.

    Ballew: What was the reason and initial appeal for moving from Rackspace to Google Cloud?

    Hiestand: Google’s infrastructure is a well-designed fit for containers using the Kubernetes project. Their architecture is designed to minimize operations work, saving labor. Using GCP, we can focus more on writing good software and less on the drudgery of making that software run. Google’s network is remarkable in every respect: good security, bandwidth within the data center, between data centers and out to the greater internet. And their compute nodes are set up in such a way that when they do maintenance, they can move our systems around and we don’t even know it. We stay running and focused while they maintain the hardware.

    Ballew: Kubernetes? What is that exactly?

    Hiestand: Kubernetes is a container orchestration tool based on Google technology that we use on GCP to help us automate and scale the deployment of our software and websites for the users. It helps us think about our applications first and operations a distant second. It also empowers Zesty.io engineers to own what’s running in ops.

    Ballew: What other technologies are you using?

    Hiestand: Kubernetes is built on container technology, so we also use Docker extensively in development.

    Ballew: What were some of the major obstacles that you ran into when migrating to Google Cloud?

    Hiestand: The technical parts are pretty straightforward. The hardest part is probably coordinating with customers. We are mitigating this by keeping Rackspace and Google Cloud in production at the same time, giving customers time to update their DNS records.

    DNS instructions can be found at: https://developer.zesty.io/docs/satellite-sites/launching-a-satellite-site

    Ballew: What are some of the benefits Zesty.io users will see from this switch?

    Hiestand: The most obvious benefit will be the faster page load speeds of Zesty.io-developed websites, including slightly faster application load times for Zesty.io developers. In the not-too-distant future, international load times will be very good, particularly for our enterprise customers. In addition, we are pushing a lot of new features to Zesty.io that will be on the Google Cloud version soon for people to try out. The rollout to GCP also improves our software engineer development velocity so we can get improvements delivered to our customers faster.

    Ballew: Even though Zesty.io users have not all been moved to Google Cloud, is there the option to move now if somebody wanted to, and if so, how can they do that?

    Hiestand: Sure! I would love to help out with that. They can send me an email at support@zesty.io. After we migrate their site, they can update their DNS to point to the new CNAME when they are ready, and customers won’t be able to update their Rackspace website anymore. They will also want to update their bookmarks because the administration URLs have changed, but for the time being, they will get automatically redirected to the new ones. We can help them with any questions they have. Eventually, we will be updating all websites for all users. If you do not update your CNAME, it will take a little extra time for your site to load because of additional network routing from Rackspace to GCP.

    About the Author / Phil Ballew

     

    philballewPhil Ballew is the Zesty.io community manager, working with developers and building the Zesty.io community through many unique outlets. Connect with him on LinkedIn.

     

  • How Agile Are Your User Stories?

    How Agile Are Your User Stories?

    Around the world, enterprises are adopting the concept of Agile development. This methodology is based on a set of principles for software development by which requirements and solutions evolve through the collaborative effort of teams. User stories form the foundation of the Agile requirements process, and teams gather to create them by exploring how users will interact with their solutions. To facilitate planning and discussion, teams typically jot down stories on pieces of paper and then physically arrange and rearrange them. It is a curiously old-school process for such a forward-thinking method.

    User stories are so vital because they give developers just enough information to estimate the level of effort needed to implement the functionality described by the story. When written well, they can be powerful, because they help developers and testers view requirements from a customer’s perspective. They provide context and an understanding of what motivates the people who will use the solutions they deliver.

    Struggles with Stories

    Agile represents a significantly different workflow than developers have previously used. As enterprises work to expand and mature their Agile practices across their projects and teams, they struggle to figure out how user stories fit, because:

    • Developers and testers may prefer user stories over a lengthy, text-heavy business requirements document, but user stories are new to most business stakeholders and business analysts. As organizations try to transition to Agile, these individuals struggle to write high-quality user stories that accurately and sufficiently describe all customer needs. They get bogged down writing and managing them and lose focus on the bigger picture.
    • The number of user stories can get out of hand quickly due to complexity. Most enterprise projects involve the development and enhancement of multiple, integrated systems to deliver functionality for a variety of roles. Users can take many paths that often interconnect and overlap with other processes. This can mean hundreds of user stories for a project of any size. Manually recording and managing the wealth of information that the development team needs is not feasible, and user stories can be missed or misinterpreted in the process.
    • Critical nonfunctional requirements—such as those for compliance, security and performance—complicate the adoption of user stories because they weren’t designed to represent those types of requirements. User stories scribbled on sticky notes or created in an Excel spreadsheet don’t support the rigor enterprises need for audits, change management, history and traceability.

    Enterprises want to reap the benefits of Agile for their complicated, expensive, high-risk projects, but they want to manage complexity and risk as well. They need a robust Agile requirements tool to help them, especially for the creation of reliable, consistent and high-quality user stories.

    Making the Shift to an Agile Requirements Tool

    Just as approaches to development have changed, so has the workplace, necessitating a new approach to teamwork. PricewaterhouseCoopers noted in its paper, “Adopting an Agile Methodology, Requirements Gathering and Delivery“: “As companies shift from small projects and teams engaged in Agile to more complex projects and potentially distributed teams, there is a need to shift from paper and spreadsheets to tools that provide workflow, persistence and traceability.”

    The power of best-in-class requirements management tools is that they enable teams to use user stories for development while making it easier for business stakeholders and business analysts to create them. These tools eliminate the need to manually create user stories within the development teams’ Agile tools. They bridge the gap between traditional and Agile requirements, enabling enterprises to scale Agile while managing enterprise concerns.

    The powerful capability sets of tools such as this help teams to:

    • Generate high-quality user stories and tests automatically from process flows. Product owners and business analysts use customer journeys to automatically generate user stories and acceptance criteria with the click of a button. They push these artifacts into the development teams’ Agile management tool of choice, where developers and testers also have access to related requirements information, such as regulatory information, visual models and constraints, supporting a comprehensive understanding. User stories are reliable and consistent, and there is no longer a need to spend time and money to teach business stakeholders or business analysts to write them.
    • Work together to visually define customer journeys. This allows stakeholders to “tell their stories” as they work with product owners and business analysts to collaboratively define customer journeys. Using the familiar construct of user models—with steps, decision points, actors and condition statements—the entire team collaborates to record and analyze processes in a shared workspace. Teams maintain a focus on strategic objectives when making prioritization decisions and spend less time managing a huge list of user stories manually.
    • Reap the benefits of Agile while using enterprise-level capabilities for visualization, traceability and reuse. Support for visual models and the ability to relate them to one another and other requirements artifacts helps teams establish the precise traceability they need to ensure full requirements and test case coverage. It also supports improved change management and decision-making, ultimately leading to higher-quality software. Customer journey models can be reused across projects and teams, as can user stories and other requirements artifacts, saving time and improving consistency.

    Great strides have been made in Agile requirements management tools recently, but many enterprises are still using sticky notes to create and organize their user stories because they don’t know about what these tools can do. A next-generation tool enables organizations to scale Agile across the business and create a common ground for IT and business to meet. With capabilities including visualization, collaboration, traceability, management analytics and reuse, teams are more effective and applications are delivered faster.

    About the Author / Ruth Zive

    ruthziveRuth Zive is Vice President of Marketing at Blueprint. She is a metrics-driven marketing strategist who has worked for two decades serving B2B clients in the technology, healthcare and financial services industries. At Blueprint, Ruth is responsible for product marketing, analyst relations, branding, demand generation and inside sales initiatives. Connect with her on LinkedIn and Twitter.

  • Keep Applications Running by Testing Early, Continuously

    Keep Applications Running by Testing Early, Continuously

    Almost every product we buy or use today goes through some form of extensive testing before it’s released on the market; typically, these tests are designed to replicate the worst-case scenarios that a product may encounter under real-world conditions.

    Automobiles, for example, go through extensive hot and cold weather testing in remote locations, and are driven over obstacle-laden tracks designed to push the limits of their durability. Even seemingly mundane objects such as children’s toys are exposed to abusive simulations to ensure that they don’t harbor any potentially hidden and harmful effects.

    On the current landscape, applications are the face of your business, often serving as the battleground where consumer loyalty is won or lost. Given this reality, it seems crazy to think that any applications are released having only been exposed to a minimum of tests. Yet, in the name of getting to market quickly, that’s far too often the case.

    As disciplines including agile and continuous delivery continue to increase the speed at which development teams can release software, it’s easy to put the crunch on testing in the rush to get an update released. But that’s a costly mistake in the long run, as research has shown that it is 50 percent less expensive to find an error in testing than to address the same error once it has reached production, and more than 90 percent cheaper  when addressed in the development/coding phase.

    Given the dramatic cost savings of addressing errors in development, clearly the goal of any organization focused on continuous delivery should be to shift testing left so a greater number of defects are found early in the development process. This is what we refer to as continuous testing, and it’s a huge part of the reason why I am so excited about CA’s acquisition of BlazeMeter.

    Performance load testing should be a core element of any continuous testing strategy because it is a critical factor in determining system reliability. Similar to the worst-case scenario testing that consumer products undergo, performance testing can quickly uncover applications defects that might otherwise go unnoticed. After all, the manner in which a system operates with a few hundred concurrent users may be very different than how it runs with thousands of users.

    BlazeMeter enables today’s development, operations and DevOps teams to conduct this critical performance testing earlier in the development process, saving countless hours and resources. The company is also competing—and winning—in a highly competitive marketplace where it has been pitted against much larger, more entrenched competitors.

    Part of the reason BlazeMeter is winning is its focus on offering a SaaS-based solution that is highly cost-effective and easy to engage. Using this model, customers can try  out the testing software with a few clicks, and then quickly move forward to broadly adopt the system. Another catalyst to its success: BlazeMeter was built using open source tools, such as Apache JMeter, used widely in the development community for performance and load testing. As a result, BlazeMeter can easily run and analyze JMeter and other open source-based load tests—such as Selenium, Gatling and Locust—within a commercial-grade performance testing system.

    This combination of a SaaS delivery model and incorporation of open-source tools, augmenting strong performance testing functionality, make BlazeMeter very complementary to CA’s existing continuous delivery solutions, including its CA Service Virtualization, CA Test Data Manager, CA Agile Requirements Designer and CA Release Automation products.

    At CA, we’re committed to providing the most comprehensive, integrated and open continuous delivery ecosystem, allowing our customers to release swiftly, test agilely and constantly increase reliably. The addition of BlazeMeter will further enable customers test continuously, thereby increasing agility and the overall quality of their software, to ensure that applications don’t break down on the side of the virtual road.

    — Aruna Ravichandran

  • How the Lure of an ‘Easy Button’ Installer Traps Projects

    How the Lure of an ‘Easy Button’ Installer Traps Projects

    As preamble for this post, I wrote about the Unicorn Installer challenge. The challenge is that user needs are too deeply heterogeneous for good reasons to create a vertically integrated way (aka an “easy button”) that works for all users. The problem is not just current diversity, it’s also about how we cope with needed changes in the future that would force users to straddle configurations.

    Don’t despair! There is a practical solution to shared ops on project installers: composability.

    If projects focused on maintaining install modules for their specific components, then the community could collaborate on well-bounded problems that should be infrastructure-agnostic. Even building in multiple scripting DSLs (Ansible, Chef, Puppet, Salt or Bash) is a much smaller-scope problem. Further, all of these DSLs have modularity concepts so it’s feasible to pull in community roles/modules/gains/plays into larger automation script. This approach allows us to decouple snowflaked infrastructure and prerequisite configuration from the project configuration.

    At that point, the proverbial “easy button” becomes an opinionated set of project, prerequisite setup and infrastructure-specific steps. These then become vendor-specific buttons: “Press here for the fast AWS install with Dejour SDN and L8istSh9y O/S” as was showcased to great effect at Dockercon.

    With that said, let’s look at how trying to create an “easy button” installer inside the project backfires.

    Cloud Foundry chose to have an in-project installer, BOSH, that has an opinionated installation approach based on generated images. To work, the installer has to maintain an increasing number of BOSH-specific integrations and image generators. These efforts are redundant with other tools non-core to installing Cloud Foundry. Further complicating the story is Pivotal, a primary Cloud Foundry maintainer, which considered BOSH a product differentiator. This creates ecosystem conflict with other Cloud Foundry vendors. While BOSH helps create robust installations, it also drags along a lot of baggage that keeps those improvements from being more generally accessible. For all of the expected ease of installation for BOSH, Cloud Foundry is still considered difficult to install.

    OpenStack installers are a very confusing story, with multiple competing in-project installers. Starting from the single-node DevStack, which provides all of the continuous integration/continuous delivery (CI/CD) testing to the confusing over-cloud/under-cloud OpenStack-On-OpenStack (aka Triple-O) concept. Neither of these initial in-project attempts created practical, universal installers. In fact, they’ve spawned waves of other installers and configuration scripts. Recently, OpenStack decided to allow many of these attempts into the project (aka “big tent”) without coordinated governance making it impossible for operators to determine which efforts they should be tracking. Having some efforts stamped with “community approved” only makes it more difficult to solve cross-platform issues. The effect is further balkanization, as vendors and advanced operators attempt to build working automation silos against competing interests.

    As a positive example, Linux pre-seed and kickstart options are distribution-specific and support a healthy ecosystem. It would be easy to think of consolidating these installs into a single system; however, the likely result would be to lock together rivals and slow down innovation.

    What we want is a thriving ecosystem around a project without endorsement as one being THE install process.

    Why is lack of endorsement critical? It allows the project to advance without being slowed down by installation entanglements created when a single way is anointed. Worse, it discourages ecosystem activity around the project.

    The first successful installers were targeted at a single vertical stack, such as Kubernetes on CoreOS or a version of Salt Kube-up on Google. These monolithic install approaches are helpful jump-starts but limit community diversity. Yet, if they are presented as “the right way,” then other participants will be discouraged from innovating. It only takes one variation in a vertically integrated process for the whole process to become exception hell.

    Our early experience with Crowbar was that our initial installation ideas were not going to remain useful as the project and targets expanded. It was important that we could maintain Crowbar AND replace it with something better. The fundamental lesson is that operations around projects is an evolving practice.

    Hindering the evolution by selecting winners early actually slows the development process.

    This may be counter intuitive; most projects want to embrace “easy button” installers to reduce start-up costs. These critical builders of initial traction may actually undermine projects as they enter a operational maturity phase by creating a fidelity gap between initial and sustaining use cases.

    — Rob Hirschfeld

  • Get to Know the New DevOps Stack

    Get to Know the New DevOps Stack

    Deploying software has always been manual and labor-intensive. That’s fading into the past as continuous innovation has become a competitive differentiator for companies of all stages and sizes across every industry.

    DevOps has become a hot investment market in recent years. The mega-rounds we’ve been seeing show the massive demand for tools that enable optimal runtime. Moreover, IT leaders are ready to spend to promote the DevOps culture.

    Companies increasingly are turning to DevOps processes and tools to automate the development life cycle and position themselves to adapt and scale nimbly. But for some companies, the idea of completely overhauling their IT operations processes can seem daunting—they don’t know where to begin. The thing is, adopting DevOps is actually quite simple.

    Examining the Stack

    Developers should bake in automation and agile development from the beginning. The good news, though, is that a defined set of tools to automate each step of the deployment process has emerged. Here’s a breakdown of the new DevOps stack, layer by layer:

    Source Code Repository: Let’s start from the bottom of the stack—from the moment you write the first line of code. Before the open-source movement took off and the cloud enabled easy sharing, developers had to work much more slowly to ensure that the necessary improvements were inserted in the right version of a product. Companies such as GitHub, Atlassian and GitLab provide source-code repositories that facilitate collaboration. Now, engineers can work on the same projects at the same time from halfway around the world without worrying about duplicating efforts.

    Continuous Integration: Newer tools such as CircleCI and TravisCI, as well as older tools such as those offered by Jenkins, automate testing and integration and provide repeatability in the deployment process. Using these tools, developers are able to test their code for failures and bugs as they write it so the code can be fixed before users ever experience the bugs. CI tools give developers something they desperately need to succeed in today’s competitive marketplace: timely feedback. By spending less time waiting on feedback for new code, engineers can resolve software bugs before they grow into full-scale crises.

    Binary Repository: Jfrog and Sonatype provide an end-to-end solution for storing and managing binary artifacts. The binary repository is the system of record closest to the production runtime environment. When something goes wrong, a binary repository such as Artifactory is the first port of call for operations teams. In addition, these tools scan for security vulnerabilities and make it easy to seamlessly distribute software to every corner of the globe.

    Continuous Development: Companies including Chef, Puppet and Ansible automate the next phase of taking data into production. The holy grail for software developers is zero downtime; continuous development tools allow developers to update software while everything remains up and running. These tools also streamline the delivery process by automatically configuring servers and ensuring repeatability of the deployment process.

    Production Environment: Google Cloud, Amazon Web Services and Microsoft Azure are in a war over runtime market share in the public cloud. Behind the firewall, Mesosphere, Kubernetes and Docker are trying to dominate the orchestration layers of enterprise private clouds. Customers want the agility and automation that comes from a cohesive DevOps stack but are refusing to be locked into a single vendor at any level. Whichever stack emerges as the default DevOps stack will have to ensure compatibility across public and private clouds.

    About the Author / Andy Vitus

    andyvitusAndy Vitus is a Partner at Scale Venture Partners. He has been investing with Scale for over 12 years and focuses on next-generation enterprise software and new data center technologies, including virtualization and cloud computing. Andy sits on the boards of Arena, Crittercism, DataStax, JFrog, PubNub, Realm Stormpath and Treasure Data. Connect with him on LinkedIn and Twitter.

  • Alerts Keep DevOps Teams Responsive, in Sync

    Alerts Keep DevOps Teams Responsive, in Sync

    Continuing our five-part series on “Continuous Monitoring: The Role of DevOps and APM,” this blog post will talk about how alert notification can help DevOps teams be more productive and agile.

    Amid the constant, rapid change and fluidity typical in DevOps organizations, there are some undeniable absolutes. These absolutes, present at each stage of the DevOps life cycle, demand your teams’ attention: A build or test succeeds or it fails; an outage of an app in production is restored to service or it lingers unavailable; users applaud or they blast out complaints. And with each of these states throughout the DevOps life cycle there are absolutes common at every step: Your team is either communicating or they are isolated, and, subsequently, either in sync or operating in confusion and chaos.

    This is still common, but avoidable. The means of communication are more abundant than ever. We can text, email, use collaboration apps or even the obscure method of speaking into a telephone to another human in real time. Yet, despite all these methods, DevOps teams still find themselves taking too long to become aware of issues that need immediate attention or action. Or, we realize too late that multiple people are responding to or chasing down the same issue. Ironically, the easier it is to communicate and even automate communication across all of the different methods, the more likely we get inundated and saturated by what we really need to be paying attention to.

    What’s the best way to cut through all the noise? DevOps teams can:

    1. Filter the junk and cut down on the noise to begin with—without missing something important.
    2. Direct the relevant info only to those people that need to take action, and only when it’s the appropriate time.
    3. Require an acknowledgment that the recipient is taking ownership of the issue.
    4. Or, of course, all of the above.

    There have been tools available over the years to tackle these noise-slaying objectives. Some do a great job getting info to our team’s pertinent DevOps information, but with no way to filter out the irrelevant or redundant stuff. And others can do some of everything, but are complex to set up and maintain and expensive to use.

    DevOps teams I’ve talked to want to be able to:

    • Immediately and automatically send critical, actionable alerts.
    • Route alerts to the right people immediately, but only when it is the appropriate time to do so (don’t wake up the wrong teammate!).
    • Speed alert response with automated acknowledgment and escalation.
    • Automatically communicate through text, email, voice mail and mobile app, all custom-designed for DevOps use cases.

    The prevalent expectation goes beyond easy-to-use, simple to start and maintain solutions. Effective tools also must meet the increasing demand for agile and efficient collaboration among DevOps team members who often use multiple communication tools.

    • Filter out the noise, but don’t miss the important stuff:
      • Filter alerts, (e.g. by problem severity), so you can focus on real problems and disregard IT operations noise. Create custom groups of contacts to make sure the right notifications reach the right team member, and streamline the process of determining who to contact when specific problems occur and how to contact them.
    • Notification channels:
      • Define experts who can respond to problems and store their contact details so they can be notified automatically of problems in their area of responsibility. Users should be organized into groups and so you can send notifications to several users at once. Notifications should able to be sent by email, SMS, mobile app, voice message and chat apps.
    • Alert management and notifications:
      • Tools with policy-based notifications can let you define which type of subject matter you want each person or group to be notified about. You should be able to create filters based on the alerts that occur, customize the filters and assign users and groups who are notified when matching problems occur. Unacknowledged alerts should be escalated automatically after a set time period to the appropriate contact. Filters such as the severity and status of alerts can be created for your IT monitoring requirements.
    • Scheduling:
      • You’ll want to be able to easily set up on-duty and on-call shifts. The process of scheduling is considered to be one the most time-consuming tasks that prevent users from utilizing a notification tool. A good scheduling capability should be intuitive and require minimal steps to use. Users should have the ability to customize or use out-of-the-box templates. Users also should be able to manually create shifts or build in shift patterns, such as follow the sun, to get their scheduling done faster. Scheduling capabilities also should be smart enough to schedule users or groups for weeks in advance.
    • Alert viewer:
      • You’ll also need to be able to monitor alert and notification status in management views, perform actions on alerts, and track them in real time. Seeing key alert information such as alert severity can help you enable to easily prioritize your to-do list. You should be able to track alerts from their receipt by the system through acknowledgment to resolution and changes in an alert history, and use predefined and real-time filters to designate alerts you want to see.

    Utilizing the capabilities such as these can empower you to improve responsiveness, save resolution time and focus on actionable notifications requiring immediate attention.

    About the Author / James Moore

    james-mooreJames Moore is responsible for offering management and strategy for IBM IT Service Management hybrid cloud-based solutions including Runbook Automation and Alert Notification. James joined IBM from the Candle acquisition in 2004, where he was product manager for Candle’s Application Response Time product lines.  James has more than 15 years’ experience in business service management, application performance management and event management. Connect with James on LinkedIn and Twitter.