devopsy.kinsta.cloud

Tag: devops

  • DevOps’ Role in Developing a Robust and Secure IoT Framework

    DevOps’ Role in Developing a Robust and Secure IoT Framework

    Creating a secure IoT infrastructure is a challenge for developers, requiring a holistic approach. It requires a dynamic framework that is capable of collecting data from a cloud back end as well as directly from the embedded hardware.

    Supporting this secure environment is another challenge, requiring the development of complex systems capable of deployment in a timely and cost-effective manner. This is where the relationship between DevOps and IoT proves invaluable.

    In this article, we will take a close look at how DevOps adds value to the development of a robust IoT framework. We also will examine ways in which DevOps can help in creating a secure and efficient framework for IoT app development.

    DevOps and IoT Framework

    DevOps, with its focus on reducing delivery cycle times, can increase the speed at which IoT systems can be integrated and function in a seamless and secure manner. Seamless integration can be achieved through a combination of secure app development and operational quality assurance checks.

    The DevOps methodology is rooted in the agile philosophy, which attempts to bridge the gap between different technical roles that traditionally worked in silos within the same department in IoT app development.

    Using the right tools and practices, DevOps can help in creating a framework for IoT that can overcome challenges relating to the large-scale app deployment. It can help in infrastructure positioning and monitoring. Companies that have adopted DevOps methodology have reported a 38 percent increase in growth due to additional revenue sources.

    Let’s take a look at some of the ways the DevOps framework can create a secure and robust environment for IoT app development resulting in increased productivity and revenues.

    Application Reliability and Efficiency

    One of the main functions for DevOps is to improve application reliability and efficiency—there is less need for coding and increased focus on rule-based automated configuration of key connected components. With DevOps, the app development team can identify red flags and improve operational excellence. With monitoring tools such as Log-entries, Nagios and Icinga, the team can use DevOps processes to remove latency and develop an efficient and robust IoT infrastructure.

    DevOps comes with source code control of the software applications that execute in the data center. This means developers have access to the source code, which makes their job easier. The source code reduces the complexity caused by cloud connectivity, automation and virtualization.

    Applications deployed through DevOps recover 96 times faster from failures and reported to have 440 times faster lead time for changes. Deeper collaboration between operations, quality assurance and development teams result in an ongoing feedback that helps increase IoT application reliability.

    Security by Design

    An important component of DevOps is security by design, which helps improve the security of connected systems in a DevOps environment. In security by design, the system allows continuous security testing in real time, which helps improve the security of devices connected in a network.

    The security-by design approach also formalizes security controls and implements them at each stage of the IT operations. As a result, software developers can spend time efficiently developing security controls in a consistent way instead of manually building and patching individual scripts.

    The DevOps approach to IoT application development can also help in gaining insights into the usage and performance, which can help in further refinement of the design delivery pipelines. This will lead to improved security and quality architecture of IoT environment.

    Faster Release Cycles

    Around 21 percent of respondents in a survey of 300 senior managers had stated that DevOps integration increasingly lead to meeting application delivery deadlines. The DevOps metrics lead time is used to measure how quickly the app is deployed after starting work.

    DevOps integration with IoT will result in the following benefits:

    • Developers can work on different components of an application simultaneously, which allows faster completion of activities.
    • Quick release cycles allow IoT apps to be pushed live without much delay.
    • Fast release cycles enable the app development team to build IoT applications using separately deployable services that lead to quality app deployment.

    Continuous Integration and Continuous Delivery (CI/CD)

    DevOps framework helps overcome interoperability problem related to IoT app deployment and operations. Most systems required for IoT app integration are already running in the framework that helps in quick deployment. Businesses can choose the process to automate from staging to production using tools such as Jenkins, GitLab and TeamCity. CI/CD allows developers to respond more quickly to market demands by proactively monitoring the processes.

    The approach results in faster resolution time and release of quality IoT application. Research by CA Technologies found that organizations that adopt DevOps find 21 percent increase in software delivery and 18 percent reduction in time to market due to an efficient CI/CD pipeline.

    Infrastructure Automation

    Infrastructure automation save systems administrators and IT resources time on manual tasks and remove the possibilities for human error during IT configuration, patch management and performance tracking. By scripting almost every command, developers can significantly reduce input requirement of administrators. Increased automation also provides stability as the system is no longer prone to human error on input. Repetitive tasks are a prime candidate for automation, as these are the most likely areas where people make mistakes.

    DevOps methodology makes it possible to automate the infrastructure in multiple cloud platforms. For this purpose, developers can use tools such as Spinnaker and Terraform. Terraform is useful for defining and managing the DevOps infrastructure, while Spinnaker is more useful for managing the infrastructure after code release and deployment workflows.

    DevOps framework also helps automates allocation, de-allocation and monitoring of both cellular and cloud resources, which ensures quality of service (QoS) for applications that control connected devices.

    Effective Validation

    Using endpoint validation can help in evaluating how data flows between different components. Testing of the components in the DevOps environment is done using scripts written in Python, Go or Groovy. The infrastructure testing involves stimulating the environment to test security and other components of the IoT app.

    The app development team also can make use of infrastructure as code to test configuration and environment automation. The configuration is tested using tools such as Chef, Ansible and Puppet. In addition, virtualization tools such as Vagrant or container technology can be used to test the environment automation.

    A Comparison of IoT with and without DevOps

    IoT with DevOps

    IoT without DevOps
    1. Implementing DevOps framework to IoT app development results in 21 % reduced security risk.

    2. DevOps can lead to faster deployment of IoT apps. The updates to IoT apps can be deployed every few days, and even many times in a day.

    3. IoT apps with DevOps practices can result in shorter lead time due to the reduced complexity of the development systems.

    4. With DevOps, the IoT application development team has reported a 51% increase in productivity.

    		 1. 71 % of respondents have said that security breach is a large challenge for IoT development outside of DevOps framework.

    2. Lack of DevOps can prevent fast feedback mechanism.

    3. Without DevOps, lead times can be longer. This will result in time and cost inefficiencies.

    4. IoT app development without DevOps presents productivity issues. There are few standards for app development and deployment that results in decreased productivity of the team.

    Conclusion

    DevOps tools and practices help businesses achieve efficiency when it comes to IoT app development. DevOps improve the validation of automated processes across multiple cloud vendors in a secure environment.

    Looking forward, DevOps integration with IoT will become increasingly popular as it can reduce costs and improve revenues significantly for app developers.

    — Subramonian

  • Living in a DevOps World, Part 2

    Living in a DevOps World, Part 2

    Part one of this article focused on some of the more behind-the-scenes benefits of an Agile DevOps approach. In part two the focus turns to some of the other traditional problems that a well-executed DevOps approach can address, and how doing so can benefit an organization in more ways than just a technical perspective.

    By way of quick review, DevOps was born out of the Lean and Agile software development methodologies when it became clear that, while those methodologies did indeed speed up the development process, a bottleneck still occurred when push came to shove and new code had to be moved to quality assurance and production environments.

    DevOps was created to more seamlessly connect the development and operations processes, making code migration, testing and promotion to production a more efficient process. To accomplish this, the DevOps approach had to find solutions for some of the issues that caused operational delays, and create new ways to organize, implement and continuously optimize the operations process.

    Overproduction/Overprocessing

    For those who have been in development and/or operations for any length of time, it quickly becomes clear that there is a multitude of operational safety checks that serve to protect a production environment. While that is vitally important, it was also clear that there had grown an “over” problem around many operational procedures, and in many cases that manifested itself in the development process. That includes overproduction, when making or requesting more than was needed from requirements and/or operations perspective to clear arbitrary operations process hurdles.

    Alternatively, overprocessing, when development and operations resources do more work (as opposed to just enough, as Lean and Agile would suggest) than required to smooth the transition of code and functions from development to operations. This created waste regarding time, resources and budgets that were not proportional to the benefits derived from following the operations process.

    Motion and Transportation

    Similarly, DevOps also sought to solve the operational problems of both motion and transportation. That is, the amount of excess work required to deliver new code to meet the operational requirements for code migration. The friction caused by such requirements slowed the motion and momentum of the development process. The same is true of transportation, or the difficulty in moving code between environments such as testing, quality assurance and production.

    In both cases, development and project momentum was sacrificed for what often turned out to be a series of artificial hurdles that had long since become less effective or even obsolete parts of the operations process.

    Correction and Inventory

    In most instances, all of the above resulted in the final maladies of the pre-DevOps development and operational ways. The first was the number of in-flight corrections required when timelines were squeezed, and the rush was on to get to production. Unfortunately, this went hand in hand with the ultimate problem of good code being sacrificed for expedient delivery, often resulting in inadequate functionality, system outages and, in the end, lost market opportunity and revenue.

    3 Keys to DevOps Success

    Any successful DevOps implementation must address three critical factors in this order: culture, organization and tools.

    Culture

    It’s critically important to connect an organization’s values to the DevOps process. Valuing quality, timeliness and organizational alignment of goals and objectives is the first step toward DevOps success. Such cultural values translate directly into a DevOps organization.

    Providing empowerment and accountability to DevOps team members helps to build ownership among the team, and trust from their customers in the rest of the organization. It also helps to provide a physical environment that fosters collaboration, teamwork and continued learning. Co-working spaces and collaboration tools such as Slack are a good start. Attending external conferences to broaden perspectives and to bring new ideas back to the team is often beneficial. From there, brown bag lunch sessions where ideas and experiences can be shared, frequent post-mortems on implementations to hone best practices, and even internal mini-conferences where several departments come together for a day to discuss DevOps practices are all effective ways to build a strong DevOps culture.

    Organization

    Any good DevOps organization is two-sided; that is it has to work from the top down and from the bottom up at the same time.

    The top-down part is in the ability to “see the system” from a macro level, allowing for process understanding and insights from a business workflow perspective. This helps to identify the pain points and bottlenecks in the current process that can be optimized through the DevOps process.

    Once that’s accomplished, the bottom-up work begins. Identifying things such as inconsistencies in code deployment environments that cause delivery issues, elimination of manual and custom built deployment processes and quarantining inefficient and poorly written code until it can be redone or eliminated are all part of optimizing the time, quality, resources and success factors for deploying production systems on schedule. It’s also important here to continually audit the current processes with an eye toward eliminating the processes that are no longer required or useful but have been kept in place out of the fear of  “breaking something we don’t understand.” If nobody understands it, then it shouldn’t be in production software.

    Automation Tools

    The final factor for DevOps success is to have the right toolset.

    Communication: Any DevOps team requires the ability to quickly and directly communicate with other team members sans meetings. For this purpose, tools such Slack (real-time chat), Skype (video chat), and Confluence (for storing persistent information) are pretty good options.

    Planning, Monitoring & Consistency: For the team’s planning needs, a tool such as Trello that can provide Kanban board functionality is worth a look. For issue tracking and monitoring of any system’s overall health, tools such as Jira and NewRelic respectively provide some good functionality. Likewise, consistency is vital in a DevOps world, and using automation to ensure that all systems are configured as desired across different environments is a crucial best practice. For this, a tool such as Ansible is worth a review.

    Integration & Deployment: For continuous integration of systems in development and as a way to tighten the feedback loop for developers to determine if the central build used for deployment to production is working as intended, the Jenkins toolset might be a good fit. And finally, when it comes making any deployment process as painless as possible, a tool such as Docker that can handle created containers for an application that includes all dependencies, reducing the complexity of deployment to multiple environments, is a solid way to go.

    The point of all of this is to create an environment—culturally, technically and physically—where DevOps can succeed, grow and thrive. Organizations that can create an effective and efficient DevOps environment have also created a competitive advantage for themselves.

    — Dave Farinelli

  • DevOps Adoption: The Driving Force of the Industry

    DevOps Adoption: The Driving Force of the Industry

    DevOps has transformed from a mere idea on paper to the way processes function in the face of an immediate requirement. Development and Operations together make for one of the most successful collaborations today.

    As per a research report, people have now begun to understand what DevOps is. When asked how familiar they were with DevOps, 84 percent of respondents said that they well-versed with the concept, in comparison with 60 percent who agreed to the same the previous year.

    DevOps is writing the success stories of the world’s business giants. Facebook, Google, Twitter and Amazon are all working the “DevOps way.” All falls in place when these companies develop, test and successfully deploy code in a matter of one to 23,000 minutes per day. We agree that’s insane!

    How Have We Benefited from DevOps Adoption?

    To sum up the benefits of DevOps adoption in a nutshell, users see:

    • Drastic reduction in lead times—With the pressure to deliver software faster and meet the challenges of the growing market, there is always a need to look at progressive ways of getting the job done, DevOps became the solution. As per statistics, practitioners believe this to be the top benefit of DevOps in 2018. Sixty-nine percent of respondents agreed to this benefit, as compared to 39 percent in 2017.
    • Increased ability to release new features with improved reliability stability and security—The need to improve the user experience and keep the customer equipped with the necessary tools to meet the challenges of a growing business made IT personnel realize that it is always essential to introduce new features into the deployment process. DevOps made it easy. Figures tell us that the implementation of DevOps led to 73 percent of people believing in its benefit of enhanced user experience, this year.
    • Enhanced IT and Organizational Productivity, Higher Profitability, Revenue and Market Shares—Now with the best process having so much to offer, all businesses have gained a lot in terms of savings and earnings. DevOps drives revenue: 26 percent of people reported an increase in revenue in the year 2018 as compared with 20 percent in 2017.

    Why No DevOps?

    Although we’ve seen a major shift in trend from being skeptical about DevOps and its implementation to becoming “pro-DevOps,” over the years, not all of us are up for this big business move. The reasons being many.

    Listed below are the most commonly given reasons by industrialists around the world:

    • Other technology and business priorities—With newer technologies being developed every few weeks, companies are confused over which technology will give their business the required leverage. This leaves them to either becoming very choosy or just ignoring the prospects of adopting a technology at all. Some business owners think about fund allocations. They want to invest their hard-earned money into something they see as more viable or something that convinces them about profits. When it comes to innovative ideas such as DevOps, they think these things may not bring them immediate results or they’re not as important as the “conceived” business priorities. As per a study conducted in February 2018, as much as 38 percent of the sample population opted out of DevOps for the above reason.
    • Lack of resources—Most organizations lack the required resources in terms of personnel, tools and persons with expertise. This discourages organizations from taking up such practices. While it is great to have a workforce with work experience for the dev and ops teams individually, it is crucial to have employees with DevOps experience on board. Similarly, lacking the right tools can have your organizations whimpering in the wake of a growing market despite having an able workforce. Statistics show that 34 percent of people hesitate to adopt DevOps for the given reason.
    • Lack of cooperation between development and operations—Whenever there is a mention of DevOps, there is also a mention of the cultural indifference of Dev and Ops teams. Each team has their unique work culture and when they come together on common ground, they cannot help but fall apart. Most organizations do not encourage the DevOps practice on account of this cultural disparity.A significant 22 percent of the industries under this research shun the idea of DevOps owing to this disparity.
    • ‘We don’t see the value’—It is likely for some business to say this. It is difficult to assess the value of the process from an outsider point of view. If someone has heard about DevOps but don’t how it will profit their businesses, then it is difficult for them to adopt it. As per statistics as small as 6 percent of the sample population did not adopt DevOps for this reason.

    On an End Note

    Organizations have been opting for DevOps after seeing its contributions to the industry. This year has most businesses functioning with major dependence on the skill and agility of the DevOps processes, and this has been proven through the success of their business. We hope the remaining number of entrepreneurs come to realize the importance of DevOps for their business and implement its benefits to build their success and keep up with the challenges of the fast-growing market.

    — Veritis

  • Compuware Extends DevOps Push for Mainframes

    Compuware Extends DevOps Push for Mainframes

    Compuware has advanced its campaign to erase the DevOps divide between mainframes and distributed systems via an update to Topaz for Enterprise Data that makes it easier to generate data from mainframe applications for testing purposes without compromising the security of some of the most sensitive data in the enterprise.

    Company CEO Chris O’Malley said Topaz for Enterprise Data now includes a data masking capability that preserves essential data relationships and characteristics without letting developers see the data being copied.

    Designed to be compatible with Eclipse integrated development environments (IDEs), Topaz for Enterprise Data also automates the extract, transform and load (ETL) process by enabling developers to convert file types.

    O’Malley noted that, at a time when compliance regulations such as the European Union’s General Data Protection Rule (GDPR) are becoming more stringent, it has become more important for organizations to be able to demonstrate that personally identifiable information (PII) has not been exposed during any point in the application development cycle. That’s especially important when it comes to mainframe data, which typically is used to drive transaction processing applications.

    In general, O’Malley said the historic divide between mainframes and distributed systems is narrowing as organizations embrace DevOps processes. Instead of proponents of these platforms battling within organizations for control over specific classes of workloads, there’s more awareness of the need to leverage both platforms to drive a new generation of digital business applications, O’Malley said.

    The challenge is that most mainframe applications are built and updated using waterfall methodologies. Compuware has been developing a series of DevOps tools for mainframe environments that are intended to modernize those processes. As the IT organizations that have mainframes adopt those tools, it becomes easier to federate application development and deployment across mainframes and distributed systems, O’Malley said.

    Of course, this all assumes various teams within the IT organization can lay aside historic animosities. The shift to DevOps has always required as much process and cultural change as it does new technologies. But in the case of IT organizations that have mainframe teams, that shift may take years. The good news is a that a new generation of IT professionals doesn’t have much history with infighting within internal IT organizations. The simple fact is there are classes of workloads that run better on mainframes, O’Malley noted, but that doesn’t mean every workload running on mainframe is ideally suited for that platform—or conversely, that every workload running on a distributed system is ideally suited for that platform.

    Arguably, the rise of DevOps processes might create an opportunity to revisit many platform decisions that were made in the heat of a larger argument. In the meantime, it’s clear that applications integration spanning multiple platforms is the new order of the day. The question now is how best to go about achieving that goal in a way that erases the divide existing between all the platforms that now make up the extended enterprise.

    — Mike Vizard

  • GitLab Moves to Automate DevOps

    GitLab Moves to Automate DevOps

    GitLab is looking to eliminate the pain of setting up and maintaining DevOps processes by making available an option that automates DevOps processes end to end using a prescriptive approach defined by GitLab.

    The latest version of GitLab includes an Auto DevOps option that organizations can employ to automate the building, testing, code-quality scanning, security scanning, license scanning, packaging, performance testing, deploying and monitoring their applications.

    GitLab CEO Sid Sijbrandij said the company is now able to make this service available because GitLab 11.0 can now be deployed as a set of more integrated applications running on top of a cluster based on Kubernetes container orchestration software. That support for Kubernetes also makes it easier to deploy GitLab 11.0 in a public cloud or on-premises environment as needed. To facilitate that process, GitLab now includes templates that simplify the spinning up of Kubernetes clusters as needed, Sijbrandij said.

    To further its support for Kubernetes, GitLab also decided to start deploying the software it makes available as a cloud service on Google Cloud Platform. Previously, GitLab had been committed to Microsoft Azure. Microsoft recently announced plans to acquire GitHub.

    GitLab is all in favor of automating DevOps processes, but doesn’t think organizations should be forced to move everything into a public cloud to achieve that goal, explained Sijbrandij. Organizations can still elect to construct their own DevOps processes using GitLab as they best see fit.

    But as interest in DevOps continues to rise, more organizations now would prefer to rely on a set of best practices that are defined to make them much easier to consume, he said.

    GitLab 11.0 also adds security scanning capabilities for application built using .Net and Scala programming language in addition to existing support for C, C++, Java, Python, Ruby on Rails, Go and PHP.

    In general, Sijbrandij noted that organizations that embrace DevOps usually find themselves trying to manage various islands of automation with mixed success. Auto DevOps is intended to provide a more comprehensive approach that eliminates the friction that often exists between those various islands of automation. Sijbrandij acknowledged that a prescriptive approach may not appeal to every IT organization, but as traditional enterprise IT organizations start to embrace DevOps, many of them prefer a more platform-centric approach that’s easier to both set up and maintain, simply because they lack internal DevOps expertise.

    Sijbrandij also noted that as DevOps gains more traction in the enterprise, many of the decisions about what platforms to employ will be driven from the top down. Today, most acquisitions of DevOps tools and practices are driven from the bottom up, which accounts for why there are already so many silos of automation, he said.

    Over time many organizations will conclude that the opinionated approach enabled by Auto DevOps will free developers to spend more time on writing code instead of on managing the infrastructure employed to create it, Sijbrandij said. Most traditional enterprise IT organizations today are at least aware they should be employing DevOps practices, they just know exactly what such an effort fully entails.

    Thanks to advances in automation, many IT leaders may never know or care what specific DevOps processes are in place. After all, what most of them are primarily interested in how fast the next set of application code is going to be delivered.

    — Mike Vizard

  • How Important Is Open Source for DevOps, Really?

    How Important Is Open Source for DevOps, Really?

    Depending on your perspective, you might believe that DevOps and open source go hand in hand. Or you may think that, quite to the contrary, the two have little to do with each other. There are good arguments to be made for both interpretations.

    Let’s explore these two ways of thinking about the relationship between DevOps and open source.

    Match Made in Heaven?

    In certain respects, the two share a great deal of core philosophical overlap. Both movements emphasize collaboration and rapid innovation.

    That is part of the reason why many major DevOps tools are open source. Examples include Kubernetes, Docker, Git and Jenkins, to name just a few.

    Indeed, it’s hard to imagine any modern DevOps team or CI/CD pipeline that doesn’t rely to a significant extent on open source tools.

    The Tension between DevOps and Open Source

    On the other hand, it’s an undeniable fact that many other tools and resources that are commonly used by DevOps professionals are decidedly not open source. There are plenty of closed-source CI servers and release automation platforms, for example. (There are also many tools in this category that are built on an open source codebase but require commercial licenses and proprietary extensions for real-world use.)

    There is also the public cloud, without which it’s hard to imagine anyone doing DevOps today. Most public cloud services are very proprietary in nature. Not only can you not inspect the source code behind services such as Azure Functions or EC2, but users also have no control over how those services operate. And porting a workload from one cloud to another can be a big challenge.

    In these respects, DevOps is far removed from open source.

    How They Really Intersect

    What to make, then, of the relationship between DevOps and open source? What does it mean that they share important cultural affinities on the one hand, yet DevOps teams hardly shy away from closed-source tools and resources?

    The most obvious conclusion, I think, is that DevOps teams are committed to openness, but they balance that priority with an equal commitment to getting results. In cases where production-quality open source tools exist, such as Kubernetes, most DevOps engineers will eagerly adopt them in place of closed-source alternatives.

    In some cases, however, closed-source solutions are superior. And sometimes proprietary solutions are the only type of solution available, as in the case of the cloud. There is no open source alternative to AWS, unless you build your own cloud, but that requires having infrastructure.

    Some DevOps teams may wish that they could achieve the results they need using only open source tools. But they can’t, and that’s why the relationship between the two can seem contradictory and inconsistent at times.

    — Chris Tozzi

  • Why Serverless Needs DevOps

    Why Serverless Needs DevOps

    EDITOR’S NOTE: This article was updated July 2, 2018

    Serverless, like Kubernetes and microservices, is a new technology. Yet, despite the amount of blog, forum and news-related posts published about serverless, it represents only a fraction of all IT deployments.

    Aside from the new factor, serverless shares few similarities with Kubernetes and microservices. Organizations turn to Kubernetes and microservices platforms to add more agility and, in many cases, computing power to their IT operations. Serverless deployments, meanwhile, are seen mainly as a way to save resources by relegating server management to third parties.

    Ultimately, cost savings is the main benefit. A successful migration largely hinges on DevOps, especially at the early adoption stage.

    “There are ways to use serverless while delivering a very rich customer experience,” said Sanjay Challa, senior product marketing manager at Datical. “It is about putting the pieces together in a very right way to do that.”

    More specifically, Challa said DevOps is relevant to serverless deployments because:

    • DevOps promotes collaboration between development and operations — a core mantra is “you build it, you run it,” Challa said. “Serverless deployments essentially embody this central tenant: it’s frankly difficult to separate development from operations because any edit to a serverless function needs to be in the context of its operation in the cloud (the cost of the resources the function will incur and the net user experience),” Challa said.
    • With serverless architecture, there is no longer a line between cloud operations engineers and developers. This is because “the roles are collapsed into one, thus embodying the fundamental core of DevOps, to collapse and harmonize the development and operations functions.” Challa said.

    For data and database management, Challa said it’s also important for teams not to forget about the underlying database and data architecture, when moving to the cloud and to new serverless architectures. “If teams only focus on the application code and forget about the database, they are setup to fail as changes to the data/persistence layer simply will not be able to keep up and scale with application changes,” Challa said. “This potentially leads to quality issues, outages or even data loss in the event of a hastily made or erroneous database change.”

    Serverless Not 

    The definition of serverless can vary, but among the many offerings available, it usually refers to functions-as-a-service (FaaS) and backend-as-a-service (BaaS) services. Despite its name, serverless has a lot to do with servers—but organizations delegate the management of those servers and the associated infrastructure to third-party cloud providers. In this way, organizations upload code or applications and the third party maintains and runs the servers. Amazon Web Services’ (AWS) Lambda, Google’s Cloud Functions, IBM’s OpenWhisk and Docker’s Serverless Docker are among the leading offerings.

    When an organization decides to migrate some or all of its data to a serverless platform, it’s important to note the technology’s drawbacks, especially as they relate to DevOps. Common downsides include slower computing performance and less capacity to run code compared to what is available when using servers on-premises or on the cloud. In the case of AWS Lambda, for example, the maximum total of concurrent executions per account is set to 1,000 by default per region and memory capacity for applications is limited to just over 3GB.

    Also, by shifting server management to a third party, organizations also are delegating those management functions, including many associated server-related security and monitoring capabilities.

    Ultimately, it will be up to the DevOps group to weigh the advantages versus the disadvantages when deciding whether to migrate to serverless platforms. While serverless technologies could potentially frustrate some developers because of the performance constraints on computing, memory, etc.; the key issue is to choose the right tool to solve the right problem, Challa said.

    “Each tool has its place,” Challa said. “Serverless is not a cure-all – but is just another tool for enterprises to consider.”

    Job Description

    Migrating to serverless also means job responsibilities will be affected, which will require DevOps to manage. “When you move to serverless, a lot of roles are disrupted. You need your network and infrastructure guys less than you did before, for example,” Challa said. “So, your team will look very different, since serverless teams in general looks very different compared to traditional IT operations.”

    It is also necessary to solve any existing DevOps challenges you have first before you make the switch to serverless, Challa said. “Otherwise, you are setting yourself up to fail,” Challa said. “Without transparency and strong coordination, moving to the cloud and to a serverless architecture will only slow things down—or, worse, lead to outages and quality issues.”

    However, once a serverless platform has been successfully set up and is running, DevOps’ workloads should decrease, since there will be less server maintenance work to do, said Holger Mueller, an analyst for Constellation Research.

    “With FaaS, DevOps is less relevant as deployments take place within the FaaS framework,” Mueller said. “There is no need for DevOps to write scripts or manually do anything. Normally, everything should just run itself.”

    However, Challa disagreed that serverless deployments completely run on their own once deployed. “Someone had to set up and configure a pipeline to enable developers to validate, build and ultimately deploy their serverless code. That someone must also play a DevOps function in enabling a path from a developer’s workstation to the cloud that supports the serverless capabilities—not to mention the work that needs to be done to connect the serverless application code running in the cloud to meaningful data,” he said.

    — B. Cameron Gain

  • Integrating Blockchain and DevOps

    Integrating Blockchain and DevOps

    Blockchain is one of today’s hottest technology buzzwords. And at the same time, the concept of DevOps has achieved acceptance and is gaining momentum. Blockchain and DevOps already offer a number of benefits individually. Together, they can provide significant tactical and strategic advantages. However, some challenges must be overcome simply to adopt blockchain, as well as to successfully merge blockchain and DevOps.

    What is Blockchain?

    Let’s start with an overview of blockchain. Many people mistakenly conflate cryptocurrency and blockchain, because many of the key features of cryptocurrency are a function of blockchain. But blockchain is a separate and unique concept that has applications and benefits far beyond cryptocurrency.

    Blockchain is a distributed account ledger, the use of which enables data to be processed and stored securely. Blockchain relies on three core concepts—peer-to-peer networks, public-key cryptography and distributed consensus that is based on the resolution of a random mathematical challenge. Reliance on these three concepts allows blockchain transactions to be secure and relatively anonymous. Every transaction is recorded in the ledger as a data block that is appended to the end of the chain of previous transactions. No group or individual owns or controls the ledger, and all parties can view the chain of transactions, ensuring transparency and integrity.

    A number of key features make blockchain attractive. Data is validated across all participating nodes, which eliminates the need to reconcile transactions. The use of distributed consensus provides a high degree of confidence, but transactions in the chain also can be verified easily. Blockchains are immutable—new data can be appended, but blocks of data already in the chain cannot be modified or removed. Finally, blockchain reduces costs and improves the efficiency of processing transactions and storing data due to its decentralized architecture.

    The challenge facing organizations wanting to adopt blockchain, however, is how to incorporate it seamlessly with DevOps development processes and culture.

    DevOps for Blockchain Development

    Organizations need to address challenges in three major areas to take advantage of blockchain: culture, process and tools.

    On the culture front, the success of blockchain depends on support at the executive level as well as adequate budget and resources for proper implementation. If you can’t find champions within the organization to drive the blockchain cause, your project is doomed before it starts.

    Whether you’re building a blockchain within a single company or a blockchain network that includes nodes outside your organization, obtaining widespread agreement on the technology and blockchain processes can be an issue. The absence of widespread industry adoption or any sort of standardized tools or processes adds complexity.

    Finding the right talent can also be a problem—and once you find it, you may have to pay a premium to obtain it. According to a recent article in TechCrunch, “Blockchain-related jobs are the second-fastest growing in today’s labor market; there are now 14 job openings for every one blockchain developer.”

    From a process perspective, blockchain is an emerging concept that lacks mature management principles or design standards. The process framework, and the different roles involved, are still unclear. Most organizations lack expertise concerning the blockchain application stack and environment; in addition, the metrics that should be monitored to measure efficiency and performance are not yet well-defined.

    The absence of development tools is another major hurdle for blockchain. The developer tools currently available for the blockchain ecosystem are unreliable. The blockchain ecosystem still needs an integrated development environment with good linters and plug-ins, a build tool and compiler, a deployment tool, documentation, testing frameworks, debugging tools, logging tools, security auditing and analytics. While some versions of these tools exist, they are not yet adequate to meet the needs of blockchain developers.

    It is also important to understand that, although many blockchain implementations are based on open source technology, blockchain isn’t free. Keeping track of transactions requires hardware, networking bandwidth, electricity and processing power. Because a blockchain ledger is distributed, transactions aren’t just processed once—they are processed on every node in the blockchain network. As a trusted network grows, the associated costs skyrocket. These expenses can be a barrier that prevents smaller organizations from experimenting with the technology.

    Integrating Security into DevOps

    If you are thinking about implementing blockchain, some of the core principles of DevOps—specifically DevSecOps—come in handy. DevOps is built on the concept of breaking down silos within an organization and enabling each team and individual to take responsibility for the project as a whole, rather than separating tasks by role.

    To implement blockchain successfully, you need a system that rewards innovation and supports teams of individuals who work together to address any issues. Integrating DevOps and security fosters stronger application security throughout the software development life cycle and helps with blockchain development efforts as well.

    The shift-left philosophy and the push for automation are DevOps principles that also will serve you well with blockchain. Integrating security earlier in the life cycle—shifting left—and incorporating automated security testing and security bug tracking are essential. The blockchain network needs to be tested for performance and latency, which will vary depending on the size of the network, the expected transaction size, the consensus protocol used and the associated required latencies. It is also important to determine whether the integrity of the network and the shared blockchain ledger is maintained while testing throughput.

    To maximize efficiency and productivity, put processes in place that allow you to consistently monitor and adjust, based on feedback. Track individual and overall security vulnerabilities, the amount of time it takes to detect them, the amount of time it takes to remediate them, and other factors relevant to application security. These metrics should be visible to every individual so that each can understand and take ownership of the appropriate elements.

    Blockchain and DevOps

    Blockchain technology is poised to change the way business is conducted today due to its promises such as of secure, immutable, peer-to-peer decentralized trust networks with distributed consensus, shared account ledgers and smart contracts.

    These same advantages, however, introduce a host of additional considerations to blockchain-based applications and make for greater testing challenges. Not only is the change in technology significant, significant changes are also necessary from a business process standpoint. While blockchain applications will have to go through standard testing and validation, including functional testing, integration testing, security testing and performance testing, they also will require additional specialized testing for smart contracts and peer/node testing.

    DevOps practices can provide great value to organizations under pressure to become more customer-centric and innovation-focused. The efficiency, quality and consistency that adopting DevOps provides is exactly what is required for traditional organizations looking to embrace the leading-edge technology known as blockchain.

    — Tej Aulakh

  • Splunk Pushes Into the Realm of DevOps

    Splunk Pushes Into the Realm of DevOps

    On the heels of its recent acquisition of VictorOps, provider of incident management software, for $120 million, Splunk is gearing up to expand its foothold in DevOps significantly.

    Rick Fitz, senior vice president of IT Markets for Splunk, said the provider of widely used operational analytics software in the enterprise sees a significant opportunity to expand its presence across a range of DevOps processes.

    Incident management is a natural extension of operational analytics, said Fitz, noting Splunk chose to acquire VictorOps because the investments the company has made in applying machine learning algorithms to monitoring IT events. That approach significantly reduces the number of irrelevant alerts being generated and provides actionable intelligence into potential events that could adversely impact IT operations.

    Most IT operations teams suffer from alert fatigue, Fitz noted, and most alerts are ignored because of a lack of context. The combination of Splunk and VictorOps will result in alerts that more precisely identify the root cause of any IT issue, he said, and a better overall “systems of engagement” for IT operations teams in keeping with DevOps principles.

    Splunk has been steadily expanding its focus on DevOps. Earlier this year the company announced Splunk Insights for Infrastructure, which IT organizations can host on an Amazon Web Services (AWS) public cloud to monitor IT infrastructure deployed anywhere. The latest versions of Splunk Cloud and Splunk Enterprise also make use of machine learning algorithms on top of a revamped metrics engine that enables IT administrators to monitor everything from CPU speeds and available hard disk space to temperature readings in internet of things (IoT) devices and sensors. Splunk recently also added connectors for Docker and Kubernetes, as well as the Apache Kafka messaging platform for streaming data in real time.

    IT operations and cybersecurity teams already rely on Splunk operational analytics software extensively to identify anomalies. By investing in machine learning algorithms and acquiring incident management software, Splunk is moving to turn all the data it analyzes into actionable intelligence, which then could be shared in real time with developers to ensure application availability and optimize application performance.

    While adoption of DevOps processes in traditional enterprise IT organizations remains spotty, IT vendors that focus on IT operations such as Splunk have clearly seen the writing on the wall. Fitz noted that investments made in machine learning algorithms, for example, will enable IT operations teams to more respond to changing application requirements with greater agility at a time when the IT environment has become exceedingly complex. That agility is critical for any organization engaged in digital business transformation, Fitz said, because invariably those efforts lead to more applications being deployed.

    It’s not clear the degree to which incumbent providers of IT operations software will be able to remain relevant in the age of DevOps. But most of them certainly have the financial resources to at the very least acquire a portfolio of DevOps tools that will enable them to exert a lot more influence than many of them have to date.

    — Mike Vizard

  • Serverless Computing: Moving from DevOps to NoOps

    Serverless Computing: Moving from DevOps to NoOps

    Developers rejoice as serverless computing ushers in a new era of ‘NoOps’ to replace the ill-fitted DevOps model.

    While some organizations do better than others when it comes to implementing DevOps, almost everyone agrees it’s a concept easier penned to paper than pinned to reality. The idea of integrating development and operations is smart, but the implementation proves troublesome. A replacement option is thus largely welcomed by developers everywhere.

    Enter the new concept of NoOps, which means dropping operations from any consideration during the development cycle. It’s not that operations are being ignored; that would be a foolhardy step backward. Rather, NoOps means never having to worry about where the code runs and, thus, developers are freed from giving operations another thought.

    “Serverless computing allows users to run code without provisioning or managing any underlying system or application infrastructure, and they automatically scale to support increasing or decreasing loads,” explained Gartner serverless analyst, Arun Chandrasekaran, in a blog post.

    Amazon sparked interest in serverless computing when it announced AWS Lambda at its re:Invent conference in 2014.

    “I’d argue that the precursor to serverless computing is cloud IaaS, where the complexity of underlying resources—servers, storage and network—was hidden from the users, and with IaaS providing flexibility to provision unlimited compute, storage and network resources on-demand, Chandrasekaran said.

    Serverless: To Use a Server or Not? ‘Tis Not the Question

    Serverless computing is a developer’s dream not only because there is no longer a need to worry about where the code is going to run, but also because there is no need for direct access to the operating system to manage that code. The result: Developers can focus solely on coding—the very thing they do best!

    That statement would be easier for some to understand if it were not for the utterly confounding term “serverless.” Obviously, anything requiring computing involves servers, so saying this process is serverless is simply nonsensical. While the term is unfortunate and bordering on meaningless, the concept is solidly real and more than a little familiar.

    For all practical purposes, serverless is just another word for cloud but with a bit of a twist—pricing is based on the actual amount of resources consumed by an application, rather than on pre-purchased units of capacity.

    Deploying Code with NoOps in a Serverless Environment

    Additionally, the term “serverless code” can be used either in conjunction with code deployed in traditional styles such as microservices, e.g. loosely coupled/SOA. Or, code can be deployed in the form of full applications that can be written to require no provisioned servers.

    In serverless environments, jobs are scheduled on-demand, meaning as data comes in and is available. It can be automated so that a job runs once data is received. This means that jobs run only when data exists and otherwise sit idle, where they can’t burn resources.

    Economies of scale also completely change in serverless computing. That’s because serverless delivers the ability to put some intelligence behind when and where jobs run, so that they can be spun up and spun down as quickly and economically as possible.

    Bottom Line

    Opting for NoOps, meaning working in a serverless environment, is really a choice to be more intelligent about how resources—hardware, software and people—are being utilized. It is about cutting resource waste and associated costs, while enabling talent to focus on writing innovative code.

    No job will sit there burning resources and waiting for the data to come in. Rather, the job runs on demand, as you need it, in milliseconds and without distracting you from other tasks.

    “NoOps still requires connectivity so beef up your connectivity to your cloud providers, if you haven’t already. And you’ll eventually have to troubleshoot slow serverless functions as a ‘network problem,’” said Andrew Lerner, an analyst at Gartner in his post.

    To gain even more advantages in serverless computing, and perhaps to make up for time lost in network troubleshooting, no matter how small, look for and use reusable code everywhere you can. Drag-and-drop interfaces that enable fast use of reusable code frees developers further, so they can shorten both the development and testing cycles. Time economy is important in modern day continuous release cycles. A NoOps and reusable code strategy can dramatically improve your time and resource economies of scale.

    — Laurent Bride