devopsy.kinsta.cloud

Tag: devops

  • Break the Rules and Win at Digital Transformation

    Break the Rules and Win at Digital Transformation

    The rules of the game have changed.

    In today’s digital world, it’s not about the big eating the small. It’s the fast that beat the slow and the nimble that outmaneuver companies constrained by old systems and business models.

    What’s more, the finish line is always moving. The rapid pace of innovation and disruption means businesses must continuously transform themselves to stay ahead of competitors.

    The takeaway for CIOs: Speed and agility are the name of the game.

    IT must respond faster than ever to shifting business needs. To succeed at digital transformation, companies must rethink how they design, build, deploy and source enterprise applications. They must adopt new approaches that accelerate delivery and anticipate, even bake in, ongoing changes.

    It’s time to update the IT playbook. Here are five best practices from digital leaders that are breaking with the past to deliver a successful future.

    Accelerate and Iterate Software Delivery

    Digital leaders outpace their rivals by adopting methodologies and mindsets that shorten software delivery cycles. They’re also really, really good at rapid, iterative change.

    Shift to Speedier Methodologies

    Digital leaders rely on Agile development methods—short sprints, scrum teams and user feedback loops—to accelerate the design and development of new applications. With its emphasis on rapid iteration and continuous improvement, Agile methodologies are a proven way for companies to churn out new innovations quickly. Studies show Agile approaches average 50 percent faster time-to-market than waterfall ones.

    DevOps is also a catalyst for speed. Automation and tighter collaboration between development and operations make it easier to put new code into production. Led by such companies as Facebook, Google and Amazon, the world is moving away from “that 10-year-old solution everyone’s afraid to touch” to flexible platforms that are updated multiple times a day.

    Adopt an Agile Mindset

    Of course, being agile requires more than just new methodologies. It also demands a willingness to take risks, learn on the fly and adjust continuously. Digital leaders don’t just respond to change; they embrace and design for change.

    Digital leaders test and then invest. They fail fast and then swiftly course-correct. The most successful experiment rapidly and incrementally so if they do stumble, they fall from the first floor, not the 10th. Digital leaders also think big. They don’t just digitize current processes. They completely re-imagine business models and technology systems—as Pitney Bowes did with its transformation into a digital commerce powerhouse or as GE did with its launch of the Predix industrial IoT platform.

    Adopt New Application Architectures and Infrastructure

    Digital leaders are building in flexibility with new application architectures and deployment models. They’re leaving legacy systems behind as they take to the cloud with microservices-based solutions.

    Move to Microservices

    Monolithic applications are hard to change, costly to maintain and ill-suited to today’s dynamic business environment. That’s why digital leaders are moving to an agile, microservices-driven architecture. Converts include Netflix, eBay, Amazon, Twitter, Gilt, Liberty Mutual and the UK Government Digital Service, to name just a few.

    Assembling applications from small modular services—each a self-contained building block—allows digital leaders to transform quickly and at scale. They can mix and match services to fast-track new digital offerings and processes. They can tweak and scale services to adapt to changing business models and customer expectations. And they can push innovation across the enterprise by reusing services—all without adding armies of developers or getting locked into any one technology.

    Step Up Cloud Migration

    The cloud is increasingly the preferred foundation for digital transformation. Digital leaders gain speed and agility by plugging into on-demand services and computing resources. They short-circuit long design and procurement cycles; focus on innovation, not infrastructure; and deliver differentiated capabilities and customer experiences at a fraction of the time and cost.

    Digital leaders are betting big on the cloud. According to IDC, at least half of IT spending will be cloud-based by 2018, reaching 60 percent of IT infrastructure spending and 60 percent to 70 percent of software, services and technology spending by 2020. Organizations across industries—such as Nike, Xerox, General Mills, Liberty Mutual, and the U.S. federal government—have migrated mission-critical workloads to the cloud to support new, more digital ways of doing business.

    Use Novel, Unexpected Sources of Innovation

    Digital leaders rely on open-source innovators and third-party collaborators to shorten development cycles. They’ve also abandoned long RFPs in favor of shorter, more effective proofs of concept (POCs).

    Embrace Open Tech with Open Arms

    Open technologies—including open source, open standards and open APIs—dominate the digital landscape. According to a study by North Bridge and Black Duck, open source software makes up more than 50 percent of the typical enterprise and application stack. Digital leaders including Ticketmaster, Box, eBay and Goldman Sachs rely on open-source software to speed solution delivery and draft off the innovations of millions of developers worldwide.

    Digital leaders also innovate openly with others. They hold hackathons. They crowdsource. They co-create with customers and partners. They also contribute back to the community with projects such as Walmart Labs, Ericsson’s open-source hub and GE’s Industrial Dojo for the Cloud Foundry ecosystem.

    Say RIP to RFPs

    Digital leaders have pulled the plug on massive RFPs for complex, multiyear development projects. Instead, they favor faster, more flexible procurement processes that reward innovation and accelerate vendor selection. They conduct POC tests to evaluate alternative technologies. They use iterative prototyping to refine requirements and determine the best solution for the job. They keep their options open with shorter timelines and smaller deliverables.

    Digital success calls for a more collaborative relationship with technology providers. It’s the whiteboard sessions, technical conversations and joint problem-solving that yield breakthrough results. That’s why progressive companies now approach the RFP like a “request for partnership.”

    Integrate Data and Systems in Transformative Ways

    Digital leaders use open platforms and APIs to integrate information, people and processes across their extended value chain. These connections lay the foundation for faster business flows and new digital products and services.

    Connect Quickly with APIs

    APIs (application programming interfaces) enable diverse applications to connect and share data. Digital leaders use API-led integration to combine services and expose information in ways that deliver unprecedented customer and business value. Within days and without lots of coding, they can roll out service innovations, revamp internal processes, enliven customer interactions, extend their digital reach and enable entirely new business models.

    This year, 2017, has been dubbed the year of the API economy. ProgrammableWeb lists more than 17,700 APIs and hundreds of digital leaders are opening up their APIs so others can easily integrate and innovate with their technologies. API marketplaces from Mastercard, ADP, Ford and FedEx are just a few examples.

    Achieve Digital Flow

    Silos slow the flow of business. Disconnected processes, departmental barriers and gaps in information all impede speed and agility.

    Digital leaders excel at silo-busting. They integrate disparate systems. They connect processes from end to end, across all customer touchpoints and organizational boundaries. They use open platforms to ensure that information—structured data and unstructured content—flows to the right person, at the right time, in the right context.

    The seamless integration of information, people and processes allows digital leaders to be much more nimble and responsive. They can also leverage these connections to make huge improvements in customer experience—such as the bank that slashed loan origination steps by 70 percent or the insurer that processes claims 25 percent faster.

    Bring Business and IT Closer Together

    Digital leaders treat digital transformation as a partnership between business and IT, with a shared focus on meaningful outcomes and better experiences for customers and employees.

    Engage and Empower Users

    Digital leaders put people—customers, citizens, employees and partners—at center of their transformation efforts. They prioritize design thinking, a user-centric approach that shifts the focus from adding features to adding value. They take advantage of platform tooling to involve users in solution design and invest in dedicated UX teams to deliver better experiences faster. Capital One, for instance, regularly solicits customer input at its three innovation labs. Other proponents of design thinking include Citrix, Airbnb, BMW, DHL and Procter & Gamble.

    Another winning strategy is to empower users with self-service tools that add business agility. Examples include self-service analytics for rapid decision-making and process services that make it easy for users to modify workflows as business needs change.

    Take a More Strategic Role

    Now that every company is a software company, IT is increasingly strategic to the enterprise. Successful CIOs have seized this opportunity and are making the most of it.

    Forward-thinking IT leaders have recast IT in the role of business partner and change agent for digital transformation. They look for ways to drive growth, deepen customer engagement and shorten time to market. They help colleagues understand and realize the value of digital initiatives. They build bridges and buy-in by adopting a service mindset—what one CIO calls “ego-free IT.” The new measures of success aren’t IT-specific metrics; they’re meaningful business outcomes such as dollars saved, customers added and market share gained.

    Conclusion

    Every digital transformation is different. But the underlying challenge is the same. How can IT accelerate the pace of delivery and change simultaneously? By following the lead of digital trailblazers, any organization can gain the speed and agility needed to compete in a digital-first world—and win.

    About the Author / Chris Wiborg

    Chris Wiborg is Vice President of Product Marketing at Alfresco Software. In his role, Chris is responsible for setting and executing the go-to-market strategy for Alfresco’s Digital Business Platform. Connect with him on Twitter and Linkedin.

  • Quality Culture, DevOps Forge Better Connections Between IT, Lines of Business

    Quality Culture, DevOps Forge Better Connections Between IT, Lines of Business

    The automation of software development does not obviate the need for a collaboration between IT and the lines of business, or the development of a culture of quality on the part of the development team. Achieving that quality culture requires more than a change in mindset as developers seek to align quality with the speed and automation goals of DevOps.

    There are five items required for a quality-driven DevOps environment:

    1. Understanding the proper role of “minimum viable product”
    2. Collaboration between IT and lines of business
    3. Having metrics in place
    4. Across-the-board buy-in

    “Quality culture is characterized by employees who are motivated to consistently strive for the highest level of quality,” said Al Riles, principal consultant at Tunnell Consulting, a firm providing solutions to the complex problems faced by life sciences and biopharma companies. “That’s one key component of it, as well as the ongoing process of continuous improvement.”

    Minimum Viable Product and Continuous Quality

    Aligning quality with the speed and automation goals of DevOps is critical to success, and part of this is gaining a deeper understanding of the role of “minimum viable product” and understanding when that approach contributes to the development process.

    “The concept of ‘minimum viable product’ does not, as its name would suggest, contribute to a substandard product,” said C.R. Venkatesh, CEO of Dot Com Infoway, a CMMI Level 3 IT company. “Quite the contrary, the concept is the basis of continuous and iterative improvement, and encourages the use of user feedback, which often results in insights that might be missed by the in-house team.”

    The concept of quality culture and MVP are not mutually exclusive in a DevOps environment, so long as continuous improvement is part of the equation.

    Collaboration Between IT and Lines of Business

    The collaboration that is a key part of DevOps is also a key part of quality culture. “You have a culture in place where everyone embraces that fabric that includes the right type of behaviors and the right values, including communication, reward and recognition, as well as clarity on roles and responsibilities,” said Riles. “That working fabric gets well-defined, because everyone is functioning around the same key elements that constitute that strong corporate culture.”

    “Collaboration has always been important in a DevOps environment,” said Venkatesh. “But we need to define precisely where that collaboration exists. In an internal environment, that collaboration is between IT and lines of business. Increasingly though, IT and development is an outsourced function, and in such cases, that collaboration must be preserved. What this means is that it exists not just within the confines of the outsourcing provider, but also between the provider and the client. The developer must seek out alliances not only with the client’s sourcing manager, but also with other internal stakeholders in the project.”

    If You Don’t Measure, it Doesn’t Exist

    Quality initiatives require an active role in changing the mindset of all stakeholders, but mindset alone won’t get the job done. Metrics must be in place and tied to those quality initiatives. “Even in the smallest of development projects, defined quality metrics must be set ahead of time, with clearly defined outcomes,” said Venkatesh. “By asking what we hope to accomplish with a development project, clearly defining the goals and adhering to a set of predefined quality standards, developers—whether in-house or third-party—are able to accurately predict what the outcome will be.”

    Riles’ process involves a quality management maturity grid, which identifies multiple dashboards. “It allows the organization to become familiar with what a quality organization consists of, and then you can use this maturity grid to measure where the organization is once you embrace this initiative. Then, you can implement the initiative, and come back and re-evaluate yourself to see the progress you’ve made,” said Riles.

    Everybody Has a Seat at the Table

    Incentivizing all stakeholders, throughout every level of the company, is paramount and, according to Riles, “It has to do with communicating what the vision is, and having a clear quality message that’s coming from leadership. Then, there has to be a shared quality mindset among all staff.” Riles notes that the quality mindset is shared by everyone in terms of understanding the organization’s objectives, policies and procedures and their roles in hoping to achieve them.

    This is reinforced by leadership at all levels, who are actively engaged in supporting the development of a quality culture, and they engage and motivate others to do the same. For it to be sustainable, we realize that success is based on the sum of all parts of the organization. They must be continuously monitored and fine-tuned as the organization grows, and this includes communication, reward and recognition, and engagement.

    About the Author / Dan Blacharski

    Dan Blacharski is a thought leader, advisor, industry observer and PR counsel to several Internet startups, and author of the book “Born in the Cloud Marketing: Transformative Strategies for the Next Generation of Cloud-Based Businesses.” He has been widely published on diverse subjects relating to cloud computing and technology, and he is editor of BoozeHub.com. He lives in South Bend, Indiana with his wife Charoenkwan and their Boston Terrier, Ling Ba.

  • New Relic Delivers on AI Promise to Automate IT Operations

    New Relic Delivers on AI Promise to Automate IT Operations

    New Relic at a FutureStack 2017 conference unveiled an instance of its monitoring tools that employs machine learning technologies and other forms of artificial intelligence (AI) to provide advice on how to optimize an IT environment.

    Nadya Duke Boone, director of product management for New Relic, says New Relic Applied Intelligence (NRAI) will be made available as part of the company’s Digital Intelligence Platform. In addition, New Relic is making available three dedicated NRAI capabilities in the form of Radar, NRQL Baseline Alerting and New Relic APM Error Profiles services. These services are designed to help IT operations teams spot abnormal behavior across billions of pieces of data and predict where problems may arise. The services ultimately will surface prescriptive recommendations for problems before they occur.

    The goal, says Duke Boone, is to leverage New Relic’s software-as-a-service (SaaS) approach to IT monitoring tools democratize access to AI technologies. That’s possible, she says, because of the massive amount of data New Relic collects. That data is then fed into machine learning algorithms to detect patterns indicative of a variety of potential issues.

    Duke Boone says that intelligence is then delivered as a personalized Radar news feed that is similar to a social media news feed. That news alert might, for example, might inform an IT administrator that, based on characteristics of their IT environment, an issue is likely to arise. IT operations teams also have the option to integrate that Radar news feed with a collaboration platform such as Slack to alert the rest of the IT operations team to a potential issue.

    NRQL Baseline Alerting allows IT operations team to set thresholds for specific applications that when exceeded will generate an alert. New Relic APM Error Profiles allows IT administrators to launch queries against NRAI to detect potential issues that might arise from outlier applications in the IT environment, adds Duke Boone.

    New Relic doesn’t see AI replacing IT operations teams as much as augmenting them, Duke Boone notes. Most organizations can’t hire enough IT staff to manage their IT environment. As those IT environments become more complex, it’s becoming clear that IT professionals will need to leverage AI technologies to keep all the applications they deployed running optimally, she says.

    New Relic has been investing extensively in AI ever since it previewed these capabilities in the form of a Project Seymour initiative it revealed last year. Now the company and its rivals are locked in an arms race to develop and implement machine learning algorithms that should enable IT operations teams to manage applications at unprecedented levels of scale.

    In some quarters AI is viewed as an existential threat to the IT operations teams. In reality, AI will be used to automatically resolve low-level issues that collectively conspire to rob IT organizations of the time they need to focus on new innovative ideas that add value to the business. In fact, assuming many of those issues will be soon disappear it’s conceivable that all the drudgery associated with managing IT today made soon just fade away.

    — Mike Vizard

  • Don’t Let Cultural Debt Bring Down Your Business

    Don’t Let Cultural Debt Bring Down Your Business

    Businesses today are faced with myriad challenges to innovate and optimize. In doing so, we often need to make decisions that borrow against the future. This can come in a tangible form, such as taking money from investors to fund growth, or in more inconspicuous forms, such as making tradeoffs in technical decisions. When we make such tradeoffs, we expect them to come at the cost of additional technical work in the future; however, these decisions can also borrow against an externality that is often overlooked: culture.

    Technical Debt

    Technical debt occurs in the development of software when the “easy path” is taken to solve a problem rather than using the best overall solution. The concept implies that the time saved now will need to be paid back later (with interest!) in the form of rework.

    Ideally, technical debt is incurred knowingly, i.e. a decision to take a shortcut is made deliberately to increase velocity, and it is flagged to be fixed before becoming a compound problem. In calling out technical debt as such, we can better analyze our decisions, considering their net present value and future opportunity costs. By documenting it, we gain better insight into the responsiveness and flexibility of the code base. Having a strategy around managing technical debt has therefore become an essential discipline in the management of software products.

    Technical decisions also have an impact on the culture of a company, though, and this facet of technical management is often overlooked. “Cultural debt” can be taken on unknowingly, and the unmanageability that flows from it can be difficult to ascertain.

    Cultural Debt

    Cultural debt is when we make a technical decision that borrows against the culture of the organization. Such decisions can introduce divisions between teams, deteriorate communication or even weaken the effectiveness of leadership. These types of decisions need to be made regularly throughout the software development life cycle, and it is therefore critical that we begin to identify them, call them out and manage them accordingly. If the interest goes unpaid we can end up with a toxic environment that is difficult and onerous to manage—we risk a paradox wherein attempts to innovate can stifle progress.

    A Closer Look

    Just as with technical debt, a business ideally will have a strategy for managing cultural debt rather than allowing it to accumulate and bog down the business. Typically, decisions that involve cultural debt will impact on areas such as:

    1. Mission: Ensuring connection to the company mission and that there is minimal variance in how the mission is understood by different teams and departments.
    2. Values: Effective and reliable transmission of company values, the ability to apply them consistently and ensure that they are clear and tangible.
    3. Communication: Fostering openness and the ability to discuss, providing channels for communication and using channels effectively with clarity of purpose and minimal “noise.”
    4. Visibility: Providing the ability for everyone to see up, down and across their organization.
    5. Attitude: Happiness of employees, their effectiveness and retention.
    6. Leadership: Ability for leaders to govern effectively, and influence across departments

    When making technical decisions, we need to think outside the box and consider how our decisions will affect these cultural dimensions of the business.

    To reinforce the concept, let’s look at some current issues that the modern enterprise is being confronted with:

    1. Domain-Specific Tooling – As roles become increasingly specialized, a best-of-breed approach provides the tools needed to perform complex tasks. Unfortunately, tools create silos, and more tools means more fragmentation. Furthermore, in a world where tools are proxies for business processes, domain-specific tooling can cut off teams from processes, reducing visibility and impeding communication. This mean the business will need to devote cycles to meetings, emails and other noisy distractions just to pay down the interest and get the visibility and coordination needed for cross-functional effective leadership. Paying down the debt requires tool integration, and without an infrastructure to reduce the cost of integrating, domain specific tools can be a heavy debt to carry.
    2. Customization – Often we see the introduction of new tooling without the requisite investment to customize them to the business. Tools such as ServiceNow or Salesforce are highly valuable for being specialized to their use cases, but they still require customization to be effective. For example, when customer data is stored in notes fields rather than using structures data captured in the tool, visibility and reporting is gradually reduced and employee frustration abounds. The decision to introduce tooling should be made with the understanding that customization will be required and must be maintained over time to keep pace with the organization’s culture as it changes.
    3. Bimodal IT – One strategy for enabling cutting edge technologies is the use of a “bimodal” structure with essentially two IT organizations that operate under different constraints. While bimodal IT may be valuable for encouraging adoption of new technologies, it also encourages confusion of responsibilities, cross-team rivalry, knowledge silos and double work. The longer the structure remains, the deeper cultural divide and the more difficult an eventual transformation will be. The decision to introduce new technologies or practices using a bimodal structure should be done deliberately, cautiously and with clear goals in mind.
    4. Technology Selection – Often we are presented with new technologies that promise to deliver great benefits. Let’s assume that the new technology passes a technical assessment and is found to deliver on the promises: e.g. a language such as Clojure or Kotlin really will simplify development and increase velocity, or a new delivery tool does indeed remove a key bottleneck. Adopting such a technology invariably creates some cultural debt—developers unfamiliar with the new language may feel alienated, new silos will form around knowledge and expertise and tribes could be forcibly broken apart. Technology decisions that ignore cultural debt will involve more cycles and greater risk, while those that recognize it will implement new technologies with those issues in mind, providing the cross-training and collaboration needed to ensure success.
    5. DevOps – The dirty secret about DevOps is that it involves the introduction of a number of technologies and technical practices that carry a large amount of cultural debt. This can actually tear an organization apart if that debt is not managed closely, and it is therefore why DevOps—despite being closely associated with technical practices such as automation, infrastructure-as-code and CI/CD—is regarded as primarily a cultural movement. In the enterprise, DevOps technologies threaten roles, responsibilities and livelihoods. The cultural debt actually does outweigh the benefits, and DevOps transformations need to focus on cultural issues first before attempting to implement the associated technologies. This is common sense that we learned from observing lean manufacturing transformations in the nineties, which involved a similar change situation where technical practices became a function of culture.

    Cultural debt, like technical debt, can be hard to detect. If your business does not have a strategy for paying off the debt, or at least one for paying down the interest, the result will be a heavy load of cultural issues weighing down decisions and processes.

    Key Measures for Managing Cultural Debt

    • Understand your stakeholders: In an agile world it is easy to overlook stakeholders. We can learn from the disciplines of business analysis and project management, though, where stakeholder analysis is a core componency and key practice in work management. It is not possible to do a thorough breakdown of stakeholders for every technical decision, but an actively maintained stakeholder matrix will increase your mindfulness and can be referred to for a free sanity check when needed.
    • Understand your culture: Your organization is filled with cultural artifacts—if you are not aware of them and taking the time to manage them, you will not be able to assess cultural impacts. These artifacts can include mission and value statements, organizational and tribal structures, meeting cadences and compositions, control systems, routines, rituals, symbols and even legends. Think through your cultural artifacts and ensure that they match intention and are understood by leadership.
    • Obtain buy-in: Getting buy-in means giving people a stake in the game. Imposing technology decisions on people will not only result in a requirements mismatch, but it fosters discontent and mistrust. You can make a large downpayment on your cultural debt by bringing affected parties into the decision-making processes early. This has been shown to be very effective in helping pass decisions even when they have negative consequences.
    • Embrace the ‘Three Ways’: We can borrow from the DevOps movement the Three Ways, which provide a solid foundation for undertaking any technical work: Systems thinking encourages considering how decisions impact the whole system, not just the local area; feedback loops allow continuous feedback and communication about impacts; and continual experimentation and learning ensures that the organization is adaptive and resilient. Leadership that embodies the Three Ways is issued debt on very good terms.

    Conclusion

    Peter Drucker is quoted as saying that “Culture eats strategy for breakfast,” and the examples we’ve seen here highlight this. The best technology in the world cannot compensate for a broken culture, and a broken culture can be the result of a succession of ill-considered technology decisions. Understanding cultural debt is a key differentiator in being able to leverage Conway’s Law rather than be trapped by it. A key aspect of managing technology going forward will therefore be to identify and control this cultural debt as it is being created, and ensure that the organization is able to function with full-capacity engagement.

    About the Author / John Rauser

    John Rauser is the IT Manager of Tasktop Technologies, a global enterprise software company. As part of his role, he is responsible for specifying, managing and administrating software lifecycle tools, witnessing first-hand the impact they have on users and the business on a broader scale. He also serves as VP Operations at the board of the Project Management Institute – Canadian West Coast Chapter, providing leadership and expertise on technology issues. He has deep passion for discussing the business impacts of technology and analyzing strategies for IT management.

  • CloudBees Applies Analytics to DevOps

    CloudBees Applies Analytics to DevOps

    At the heart of many DevOps initiatives is an automation server based on open-source Jenkins software developed by CloudBees. At the recent Jenkins World 2017 conference, CloudBees showed that Jenkins is now becoming the foundational element through which DevOps teams can now manage a continuous integration/continuous deployment (CI/CD) process end to end.

    New additions to the portfolio include CloudBees DevOptics, an analytics application to aggregate data from software development pipelines as it moves through a Jenkins server. CloudBees CTO Kohsuke Kawaguchi says once organizations are armed with those insights they will be able to, for example, identify potential bottlenecks in their software development processes. CloudBees DevOptics creates a single point of reference on the status of any application delivery project, all the way down to individual commits, says Kawaguchi.

    CloudBees also launched CloudBees Jenkins Advisor, a free plug-in for a Jenkins environment that Kawaguchi says provides another watchful eye capable of continuously analyzing a Jenkins environment using algorithms the company developed. Cloudbess Jenkins Advisor comprises a knowledge base and diagnostic tools that can be employed to detect and resolve issues ranging from simple configuration issues to security and other forms of best practices.

    Finally, the company revamped the user interface provided with CloudBees Jenkins Enterprise, an implementation of Jenkins that comes pre-configured with built-in security and integrations with more than 1,400 DevOps tools. It also announced that a few select managed services providers are now building a practice around Cloudbees Jenkins Enterprise.

    CloudBees reports there are now more than 150,000 active installations of Jenkins worldwide that span a user community of more than a million. Those numbers represent a 50 percent growth rate from January of 2015.

    Most organizations employ DevOps processes to accelerate development of software. But along the way it’s not uncommon for many organizations to lose control over those processes as the amount of software being developed begins to overwhelm their management teams. CloudBees clearly sees providing tools to optimize DevOps processes based on Jenkins as its next big opportunity.

    In general, DevOps is much more of a journey than an event. But rather than being a simple article of faith, analytics can be used to show all the participants in a process that progress is in fact being made. Otherwise, resistance to increasing the number of application development projects being undertaken at the same time starts to mount. Whatever the reason for gathering those metrics, however, that old “Things Measured Are Things Done” maxim continues to apply. Those metrics also help IT organizations justify the return on investment in DevOps to senior managers.

    Of course, managing DevOps processes at scale is as much a cultural challenge as it is a technical issue. When many organizations first embrace DevOps there’s a lot of initial enthusiasm. But the further down the DevOps maturity curve an organization gets, the more they encounter various hurdles. Metrics collected and analyzed by the tools provided by CloudBees help organizations determine to what degree those hurdles are related to simple technical glitches versus organizational issues that are almost always a lot harder to resolve.

    — Mike Vizard

  • Quest Software Extends DevOps Reach to MySQL Databases

    Quest Software Extends DevOps Reach to MySQL Databases

    Thanks mainly to the rise of DevOps, the number of open-source database deployments in the enterprise has risen sharply in the last two years. The driving force behind those deployments has been developers that have shown a marked preference for downloading an open-source database instead of requesting access to a commercial database from a database administrator (DBA) working in an IT department. As more of those applications get deployed, however, either the developer or a DBA still needs to find a way to manage those open-source databases.

    To address that issue, Quest Software is now making available its Toad Edge tools for managing databases available for IT organizations that have deployed MySQL databases. Greg Davoll, executive director of product management and marketing, says Quest Software already supports open-source Postgres databases and will be adding support for other open-source databases.

    In a lot of instances Davoll says the same DBA that is being tasked with managing IBM DB2 or Oracle databases is being tasked to manage MySQL databases. Most of them are already familiar with Toad database tools that are already widely used by internal IT organizations. Davoll says Quest Software is well aware there are a variety of open-source tools available for managing open-source databases. But Davoli says most of those offerings are not nearly as rich in terms of features as Toad database management tools that have been continually enhanced now for the better part of a decade.

    For example, Toad Edge can be used to identify and manage schema differences and initiate changes in real time. Toad Edge can also be used to explore and edit JSON data as well as keeping all the scripts being employed to manage databases up to date.

    Davoll notes that Toad Edge has also been integrated with Jenkins Continuous Integration server to allow databases to better facilitate continuous integration/continuous deployment (CI/CD) processes.

    Most DBAs have come to terms with the fact they are being required to support multiple databases. They don’t always appreciate the extra work that involves. But as the number of applications deployed across the enterprise dramatically increases, the cost of employing commercial databases to support them all would be prohibitively expensive. Developers often download an open-source database on their own because they don’t want to spend time participating in an extended procurement process involving salespeople and purchasing managers.

    Developers don’t always appreciate what’s required to keep the databases their applications up and running. Many of them initially believed they would be able to eliminate the need for a DBA altogether. But developers soon discover how much time is required to do routine database maintenance. DBAs may not always get the respect they deserve. But as most developers have come to appreciate, it’s difficult to have a truly integrated DevOps process that doesn’t include at the very least someone willing to manage those tasks.

    — Mike Vizard

  • Differentiating Between CI Pipelines and DevOps Assembly Lines

    Differentiating Between CI Pipelines and DevOps Assembly Lines

    “What’s in a name?” wrote the greatest Bard who ever lived.

    All I can say is, he lived in simpler times, when DevOps and the confusing world of continuous integration (CI), continuous deployment (CD), CI/CD, the other CD, application release automation (ARA), pipelines, assembly lines, etc., didn’t exist.

    The biggest challenge today is that vendors use overloaded terms to describe completely different use cases. CI vendors claim to support CI/CD, even though the “D” here is just a deployment to a development environment and not staging or production. ARA vendors claim to enable continuous delivery, even though they do not include critical steps such as infrastructure provisioning, security patches or configuration management.

    The terms “pipelines” and “assembly lines” are relatively new. Multiple vendors have recently announced pipelines and assembly lines, and the differences aren’t obvious at first glance. So let’s do a deep dive.

    What are CI Pipelines?

    Continuous integration has evolved tremendously over the last few years. What started as a simple process of automating build and unit tests for each code change has evolved into a pretty complex workflow.

    As an example, look at the different CI workflows shown below:

    In classic CI, all build and test instructions were included in a single blob and instructions were executed serially. If all instructions executed successfully, CI was successful, and if anything failed, then it was marked failed. Almost all CI providers, until recently, offered just classic CI.

    However, as CI became mainstream, developers wanted to define more complex workflows with decision trees and parallelism. CI pipelines support this complexity, such as:

    • Defining the CI workflow in stages, so you can get results from a stage faster without having to wait for the entire workflow to finish. This also enables scenarios such as running a basic test group and only running the extended test suite if the basic tests pass.
    • Running stages in parallel, so you can split tests or test your code against different environments or language versions, and get faster feedback.
    • Forking your CI logic based on the results of a stage.

    While CI pipelines are a huge improvement over classic CI, they are still limited to developer-focused workflows. Jenkins, Teamcity, Shippable, Gitlab and CircleCI all support some form of CI pipelines.

    What are DevOps Assembly Lines?

    DevOps assembly lines are focused on automating and connecting activities performed by several teams, such as CI for devs, infrastructure provisioning and configuration management for ops, test automation for test, security patching for SecOps, semantic versioning and approval gates for release managers, deployments for multiple environments, and so on.

    Today, most organizations use various tools for automating specific DevOps activities. However, the DevOps toolchain is fragmented and gluing it together to achieve continuous delivery is a challenging task.

    Most teams adopt one of the following approaches:

    • Using cultural collaboration as the glue between silos. As an example, when a new application version is available, the test team creates a JIRA ticket and assigns to ops. Ops then copies the application version information into their deployment scripts and run them.
    • Writing ad-hoc scripts to trigger one activity from another, including passing state information through intermediate storage such as S3. In this case, the DevOps engineer would likely write a Jenkins job that runs automated tests, and then pushes application version information to S3. Another Jenkins job polls the S3 bucket, and when a change is detected, runs the deployment scripts.

    The first approach introduces inefficiency and unnecessary human-dependent steps. The second approach is better, but only works well for small teams working on a single application. It creates a Frankenstein of spaghetti automation when you have many applications or even a single application with microservices.

    DevOps assembly lines address this problem by focusing on gluing together these various DevOps activities into streamlined, event-driven workflows with the ability to easily share state and other information across activities.

    The following image shows a typical assembly line for a single application or service:

    See the difference? The CI pipeline is just ONE activity in the assembly line. Each of the boxes in the picture above is a pipeline, just for different activities! Each has specific needs with respect to tools integration, runtime, configuration, notifications, etc. In addition, each pipeline is owned by a different team, yet needs to interact with other pipelines and exchange information. For example, if the ops team changes provisioning scripts for staging environment, the test team needs to redeploy the application and run performance tests.

    A DevOps assembly line is, therefore, a “pipeline of pipelines.”

    DevOps assembly lines need to support the following:

    • Ability to easily define workflows across multiple pipelines.
    • Versioned and reusable workflows, enabling rapid changes and scaling for multiple applications and/or microservices.
    • Integrations with all popular source control systems, clouds, artifact repositories, DevOps tools, languages, services, etc.
    • Runtime to execute every pipeline, including pre-installed tools and CLIs, and the ability to use the right runtime, depending on pipeline type.
    • Accelerators/playbooks for common pipelines and tools.
    • Ability to pass state and other information while triggering dependent pipelines.
    • Automatic triggers or manual approval gates between each pipeline.
    • Configurable notifications for each stage of every pipeline.
    • Release automation features such as semantic versioning of packages.
    • Audit trails for each pipeline, with the ability to go back or forward to a specific state.
    • Abstraction of all sensitive information like passwords, tokens, keys, etc. for security reasons.
    • Roles and permissions restricting assembly line and pipeline actions.
    • Manage DevOps infrastructure, including spinning VMs and containers up and down as required.
    • Visibility into each pipeline and stage, including logs, status and versioned data.
    • Metrics and Analytics across pipelines to help identify bottlenecks.

    In short, DevOps assembly lines help automate and scale end-to-end workflows across all teams and tools, enabling continuous delivery.

    About the Author / Manisha Sahasrabudh

    Manisha Sahasrabudhe is co-founder and vice president of product management at Shippable. Prior to Shippable, Sahasrabudhe was senior program manager at Microsoft. She can be found online at Linkedin.

     

  • NodeSource, AppDynamics Ally for Node.js DevOps

    NodeSource, AppDynamics Ally for Node.js DevOps

    The prevalence of Node.js as the most widely deployed instance of JavaScript is putting a lot of pressure on DevOps teams to find ways to automate the management of Node.js applications. To provide the visibility needed to achieve that goal, NodeSource has announced it has integrated its N|Solid development environment with the application monitoring tools developed by AppDynamics.

    Pravin Halady, product manager for NodeSource, says one of the things that differentiates N|Solid, which is based on Node.js, most is the amount of telemetry data NodeSource is able to collect from Node.js applications. The alliance with AppDynamics provides a mechanism for DevOps teams to now tap into that data to better manage those applications alongside the panoply of existing applications they need to support and manage, says Halady.

    AppDynamics already supports Node.js. But Halady says the alliance with NodeSource provides deeper visibility into, for example, asynchronous activity. Advanced metrics can be used to augment the basic Node.js metrics that AppDynamics currently provides to better ensure service level agreements (SLAs) are both achieved and maintained, Halady says.

    Adoption of Node,js exploded across the enterprise once it became feasible several years ago to deploy JavaScript code on both the client and the server. That capability in many ways fulfilled the early promise of Java in the eyes of many developers. But DevOps teams have been slow to include support for JavaScript applications within their DevOps processes, mainly because of a lack of visibility. Now that the base of Node.js applications in the enterprise has reached a level of critical mass, tools vendors such as AppDynamics are looking to provide that visibility by expanding the scope of support they provide for Node.js.

    Halady says NodeSource chose to partner with AppDynamics in part because in the wake of AppDynamics being acquired by Cisco Systems, it’s clear that NodeSource needs to participate in the end-to-end visibility platforms that larger companies are starting to build. Cisco has already made significant investments in a Cisco Tetration analytics application in addition to committing to imbuing the IT infrastructure environment it provides with machine-learning algorithms. That data will then be combined with data about applications collected by AppDynamics to identify the source of any potential performance issue faster. NodeSource, adds Halady, will look to foster similar agreement with other providers of application monitoring tools.

    In the meantime, DevOps team can take heart in the fact the Node.js Foundation is making providing visibility into DevOps environments a higher priority. The number of Node.js applications in the enterprise is only expected to increase. There’s a high correlation between usage of Node.js and containers that collectively are being used to facilitate the development of microservices. As those microservices bgin to replace monolithic applications developed originally developed in languages such as Java, the size and scope of the DevOps challenge exponentially increases.

    Application development in the enterprise is going to be polyglot well into the next decade. Today there are more programming languages being actively used in the enterprise than any time in history. In fact, the days when IT operations could mandate what technologies developers can employ are long over.

    — Mike Vizard

  • Incorporating Performance Engineering into DevOps

    Incorporating Performance Engineering into DevOps

    Performance testing historically has been a challenge for most IT organizations. It is highly misunderstood, from executives down to the few resources allocated to perform an extremely difficult task. Most companies consider performance testing expensive and unnecessary and simply refuse to acknowledge it is a critical component of the software development life cycle.  Have we forgotten about major performance failures such as Healthcare.gov? It is time companies start to take performance engineering seriously.

    IT organizations using the Waterfall methodology relied on a few performance testers to develop performance scripts and run them after most of the testing had been completed and the application was a few weeks from a production deadline. If the performance team was lucky, they would get in one or two successful tests, typically working off-hours and executing performance tests on small, patched-together ancient hardware. The problem with this approach is when issues are found, there is not enough time to fix them adequately and performance degrades over time.

    Developing a Performance Engineering Strategy

    Here are some items to consider when incorporating performance engineering into DevOps:

    • Learn—It is important to learn what performance engineering is and how it can be incorporated into your organization. If you don’t have the right skills within your organization, go find someone who can help. There are many organizations that can help with an initial assessment of where things stand.
    • Develop Your Strategy—This is perhaps were most teams fail. Upfront planning is critical and it will save you in the end. It is important to develop a performance engineering road map. This is not something that will be built in a few weeks; a solid performance road map could take 18-24 months, so be patient and build it right.
    • Build Your Baseline—It is important that you build a baseline of your performance tests so that you can determine if you have successful results. Performance testing is different from functional or automation because you have to consider the performance run as a whole. A few high transactions usually are acceptable. It is a good idea to build a history of runs to see how the application performance is trending.
    • Identify Key Business Workflows—It is too time-consuming to build hundreds of performance tests, so focus on those that are the most critical to your business. Typically those workflows will account for the majority of your performance load.
    • Test Upfront—With DevOps you need to start early and you will need to run performance tests frequently. The more you can run upfront, the more performance issues will be avoided in production.
    • Tools and Hardware—Performance engineering requires tools and hardware. It is important that your assessment evaluates what you currently have and what you will need to do it properly. There are many open-source tools on the market today that can help and other tools that are making a big difference in the performance space.
    • Budget—Developing a world-class performance engineering practice is going to take money. When you develop your performance road map, you might need two to three budget cycles to get the tools and infrastructure you need.
    • Load Performance Environment—Yes, you will need to have a performance environment that is scaled to match your production environment. If there isn’t enough budget, then you will have to know how much smaller the environment is so that you can be confident in your performance results.
    • Data—It is important to have data that is similar size to your production environment. This will help you understand how your system will handle performance within the production environment.

    Incorporating performance engineering within DevOps will require work, but the rewards will certainly put the organizations that embrace it at the top of the list. DevOps enables teams between Dev and Ops to produce a stable application that performs exceptionally well under load. Here are some benefits of DevOps performance engineering:

    • Predictable production performance results
    • Production deployment confidence
    • Eliminate catastrophic performance failures
    • Increased performance efficiency

    DevOps gives organizations a real opportunity to get performance engineering right; unfortunately, most organizations will carry over the same mistakes into DevOps. I hope you will take the opportunity to embrace the performance engineering challenge!

    About the Author / Ron Wilson

    Ron Wilson is a passionate champion of Agile, DevOps and Quality Engineering. He has managed large global Quality Engineering teams across multiple industries. He is currently the director of Quality Engineering at Arch MI. He regularly shares insights through social media where you can find him at www.qarevolution.com and on LinkedIn.

  • DevOps, Continuous Delivery – It’s All About Automation

    DevOps, Continuous Delivery – It’s All About Automation

    I spent a large portion of my career in the networking industry, marketing routers, switches and other gear that makes the internet work. Networking—like many industries—is currently undergoing a revolution that is in large part driven by software. Functions that used to be delivered as a standalone device can now run in software (not unlike the apps on your smartphone); and software is also being relied on more to make decisions about how traffic is routed through the network.

    I still read up on the networking industry occasionally, and if you’re looking for an expert opinion or just good journalism in this space, you’d be hard-pressed to find a better source than Craig Matsumoto. Craig wrote an interesting opinion piece called “NFV, SDN, Big Data – It’s All About Automation.” You don’t have to know what those acronyms mean to understand the premise.

    What’s interesting is that the same premise can be applied to many of the things that promise to make software development easier, faster and cheaper. DevOps, continuous delivery, continuous testing—all rely on automation to be successful.

    DevOps, of course, requires more than just automation—as it’s also a cultural shift—but automation plays a huge role. The three main components of successful DevOps are commonly cited as tools (or technology), process and people (or culture). Automation is the primary goal of the first two: tools that enable you to automate previously manual tasks, and then establishing the processes that allow you to take advantage of this automation.

    The same can be said of continuous delivery or continuous testing. Continuous testing relies on being able to automatically generate test cases, automatically create virtualized testing environments and create automated calls for testing.

    The premise of a modern software factory also relies on automation. Any modern factory, whether it’s producing cars or toothbrushes, strives to improve efficiencies by eliminating as many manual tasks as possible.

    The fact that software development and operations are lagging behind other industries in terms of automating the production of its goods (i.e., software) is maybe a bit ironic, given that software development is typically seen as more technologically advanced.

    However, it’s also a result of the maturity of the industry and that fact that coding, by its nature, cannot be automated. But the tasks that surround the actual writing of code certainly can be automated, and that’s what the modern software factory promises. The more you can automate, the more successful you will be in delivering quality software faster, with reduced risk and costs.

    Automation seems to bring with it concerns about employment—that is, there is a not-unfounded fear that it’s possible to automate oneself out of a job. However, there was an interesting study that came out, focusing on the automobile industry—one of the most well-known examples of factory automation. The study concluded that:

    In the last six years, (2010–2015), according to the IFR (International Federation of Robotics), US industry has installed around 135,000 new industrial robots. The principal driver is automation in the car industry. During this same period, (2010–2015), the number of employees in the automotive sector increased by 230,000.

    Essentially, automation not only improved production velocity and quality, it also resulted in a net increase in employment.

    The message is clear: Automate what you can and only rely on manual tasks for what you must. If you focus on automating your software development life cycle, not only will you be able to deliver software faster, cheaper and with better quality, but you also will be well on your way to achieving continuous delivery and DevOps.

    — Brendan Hayes