Tag: devops

  • New to DevOps? Resources to Help You ‘Get With the Program’

    New to DevOps? Resources to Help You ‘Get With the Program’

    Resources to help put you on the right DevOps career path

    The demand for DevOps solutions is set to achieve an annual compound growth rate of 16.5 percent in North America for the 2017-2023 period, according to KBV Research. Globally, the DevOps market size is expected to reach nearly $9 billion in five years. That we’re facing a tsunami of skills shortages goes without saying.

    Yesterday’s view of IT operations tended toward the “Dev” side being the “makers” and the “Ops” side being the people that deal with the creation after its birth. The realization of the sheer amount of inefficiencies that occur when these siloed concerns work at cross purposes became a main core driver behind DevOps.

    So What is DevOps, Really?

    Jez Humble, co-author of “The DevOps Handbook,” defines DevOps elegantly as “a cross-disciplinary community of practice dedicated to the study of building, evolving and operating rapidly changing resilient systems at scale.”

    DevOps can be described in many ways. The Agile Admin blog claims it simply as:

    “… the practice of operations and development engineers participating together in the entire service life cycle, from design through the development process to production support. DevOps can be interpreted as an outgrowth of Agile – agile software development prescribes close collaboration of customers, product management, developers, and (sometimes) QA to fill in the gaps and rapidly iterate towards a better product.”

    Education and Training  

    There are plenty of resources available for wannabe practitioners to cut their teeth, starting with Lynda.com courses on the LinkedIn Learning channel.

    It is even possible to earn professional certification. For example, certification for a DevOps engineer from Cloud Academy comprises more than 32 hours of interactive content (including 16 video courses and 11 hands-on labs) to help you learn the DevOps ropes. Courses include content on networking, security and deployment.

    The DevOps Institute is another great resource, offering a range of courses to help DevOps practitioners of all competencies gain education and experience. The organization curates its members and the DevOps community in general to track and trend the latest and most innovative approaches to DevOps, and offers certification and “digital badging” to its members.

    But what if you don’t want to, or can’t take the steps needed to earn certification?

    Kaushal Amin, CTO at Atlanta-based KMS Technology, an outsourcing services provider with 500 developers in Vietnam, said it takes some studying, networking and, above all, people skills. Talk with IT professionals outside of your organization who are already established in DevOps, as experience can be one of the most valuable resources available. Amin proposes three helpful steps for IT professionals looking to learn about DevOps:

    • Gain a highly focused understanding of DevOps via low-cost online courses offered by Udemy such as Modern DevOps and Mastering DevOps.
    • Talk with friends and colleagues in different roles from your own. Devs should reach out to systems administrators to discuss job responsibilities and tools used. Attend conferences and training sessions to meet people in different DevOps roles.
    • Talk with QA test engineers. They’re the people at the end of the DevOps process responsible for executing quality assurance.

    Amin suggests that after you’ve gained familiarity with DevOps, try out popular cloud-based app development platforms such as Heroku. These providers often offer free versions that can help foster an easy learning path while showcasing cloud services that can help DevOp teams create business process apps and avoid headaches typically associated with assembling or managing platforms and IT infrastructures.

    Suggested Reads

    Tracking the entire software development life cycle is fraught with complexity. Rookies can take comfort in scanning through a glossary of commonly used DevOps terms and concepts.

    No collection of resources is complete without a reading list. At the top of most DevOps-related booklists is, “The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win,” by stalwart veterans Gene Kim, Kevin Behr and George Spafford. Other notable reads include:

    • “Building a DevOps Culture,” by Mandi Walls
    • O’Reilly Media’s “Effective DevOps: Building a Culture of Collaboration, Affinity, and Tooling at Scale,” by Jennifer Davis and Ryn Daniels
    • “The IT Manager’s Guide to Continuous Delivery: Delivering Software in Days,” by Andrew Phillips, Michiel Sens and Adriaan de Jonge

    And beyond this website, which offers a plethora of DevOps-related content ranging from articles to videos, podcasts to webinars, another helpful resource is DevOpsLinks.com, a learning community of professionals looking to share tips and best practices. As its founder Aymen El Amri, puts it, “For evangelists, DevOps is a culture and a transformation. For some engineers, DevOps is a set of agile tools and techniques. For managers, DevOps is probably a methodology. For other people it is just a buzzword and for recruiters, DevOps is a job.”

    — Victor Cruz

  • How DevOps Can Create Smarter, More Agile 5G Telecom Networks

    How DevOps Can Create Smarter, More Agile 5G Telecom Networks

    There is no doubt DevOps revolutionized software development with its introduction of lean and agile techniques. By applying continuous deployment and continuous integration methodologies into software development, enterprises can build, test and deploy applications and release new services more rapidly. In this article, we will see how the DevOps approach is being utilized in 5G telecom networks with the rise of software-defined systems.

    Emergence of Software in Legacy Telecom Networks

    In last few decades software has changed our lives in myriad ways—mostly for the better. Now, software is starting to disrupt the telecommunication industry with the eagerly anticipated 5th generation of telecom network, also known as 5G. It is expected to provide low latency and high bandwidth network connections to serve use cases including artificial intelligence, smart city, 3D video, industry automation and self-driving cars, and to provide end-to-end real-time experiences such as virtual reality and augmented reality. A variety of these use cases have different network requirements and dependencies, and 5G is expected to provide such flexibility along with agility.

    Software-defined networking (SDN) and network functions virtualization (NFV) form the backbone of the upcoming 5G network, enabling service providers to deliver lucrative services to their customers and realize a competitive advantage. SDN makes the network programmable and centrally controlled, while NFV replaces dedicated network appliances with software packaged in virtual machines on top of high-performance servers. SDN and NFV enables:

    • Better control over overall network.
    • Reduction of manual network configuration processes.
    • Lower CAPEX.
    • Network resiliency.
    • Orchestrate network dynamically.
    • Faster time to market, i.e. faster introduction of new services and features to customers.

    Application of SDN and NFV technologies in 5G telecom networks indicates the use of software in the legacy telecom network, wherein many network functions including firewall, content delivery network (CDN), network address translation (NAT), routers, deep packet inspection (DPI) and broadband remote access server (BRAS) can be virtualized and serve as virtual network functions (VNFs) in NFV architecture. This stands as the core feature of upcoming 5G networks, as most of hardware-dependent operations will be performed by software, providing higher performance with tons of benefits to end users.

    Cloud-Native Enables DevOps

    Virtual network functions (VNFs) are the software representation of network function appliances in NFV environment and a core element in 5G networks. Services can be provided by chaining different VNFs developed by different vendors. There are complexities associated with having multivendor VNFs, as each VNF has its own performance and operational issues including manual integration in NFV infrastructure, utilization of NFV hardware resources, lack of interoperability and multitenancy. For a 5G network to deploy successfully, each VNF must be configured and managed in automated way.

    As software becomes part of telecom network, the term “cloud native” becomes part of the vernacular for VNFs.

    By harnessing a cloud-native approach—including containerization, microservices-based architecture and dynamic orchestration—VNFs with their own life cycle enable:

    • Decomposition of sub processes into microservices—resources required will be automatically scaled and communicated with other services using APIs.
    • Management and simplified orchestration of VNFs.
    • Deployment within containers and connected by APIs.
    • Orchestration by container scheduler—manages provision into service and lifecycle management.
    • Portability to different NFV environments.

    More importantly, a cloud-native approach allows VNF vendors and service providers to impose DevOps (continuous deployment and integration) methods to automate process of building, validating and deploying VNFs into NFV environments.

    A DevOps approach in 5G networks brings benefits for overall network operations, enabling deployment automation and management of service chains and 5G network slices. Complete automation is required, especially for 5G network operations, which may increase operational efficiency and decrease human error, saving OPEX and decreasing expensive downtime.

    For 5G networks, requirement of DevOps methods in VNF can be for:

    • Automated onboarding and deployment of VNFs in NFV environment.
    • Automated performance testing and benchmarking for VNFs.
    • Enabling self-scaling of VNFs—automated resource provisioning for VNFs.
    • Rnabling self-healing after an error in VNFs residing in VMs or containers.
    • Real-time monitoring of VNF activities for analysis.

    More ‘Intelligent’ Network

    Cloud-native capabilities and a DevOps approach opens the possibility of incorporating artificial intelligence in 5G telecom network operations analysis and real-time monitoring. Cisco Systems has already introduced its solution based on machine learning, which it calls Intent Based Networking. We can imagine a network operation having “zero touch” feature by just providing an intent to complete network for any configuration or specific network-based instruction. The collected analysis data can be used in the management and organization of NFV for corrections of network errors, adjustments in VNF and NFV elements configuration and self-healing/scaling of network functions.

    Summary

    Development of 5G network is still in process. By taking into account a revolution in SDN, NFV, cloud native and DevOps, it is expected that we will have 5G networks by some of leading players Verizon, AT&T, Telefonica and Vodafone reasonably soon.

    — Sagar Nangare

  • DevOps is Old News. DevProd is the Future

    DevOps is Old News. DevProd is the Future

    DevOps has been widely embraced and isn’t going away. However, there’s a new software movement emerging that deserves headlines of its own. Let’s call it DevProd.

    In a recent Harvard Business Review article, “Agile at Scale,” the authors point out that “conditions are ripe for agile teams in any situation where problems are complex, solutions are at first unclear, project requirements are likely to change, close collaboration with end users is feasible, and creative teams will outperform command-and-control groups.”

    To understand why DevProd may be the next big thing, some perspective on the DevOps movement will first be helpful.

    DevOps Addressed a Huge Pain Point

    At one time, software was often created by a single person. By 2008, teams of hundreds or more were needed to meet the accelerating complexity of software development. We hit an inflection point, triggering an organizational need to streamline and fuse development (code) and operations (IT infrastructure).

    Companies were scrambling to “go agile,” but silos between these constituencies became roadblocks. In response, DevOps, which was coined at the 2008 Agile Toronto Conference, broke down those silos and had a profound impact on a company’s ability to move faster and more efficiently.

    Waves of companies and products soon enabled streamlined processes between developers and operations. Some became successful, such as AppDynamics (acquired by Cisco Systems), Splunk, Chef and Puppet.

    Today, while there remains room for improvement, this challenge is largely resolved. Companies have added DevOps experts, while DevOps tools are embedded in development and operations processes.

    History Repeats Itself

    As part of my job, I often speak with leaders of software development organizations. Once the conversation turns toward development speed, app deployment or even features management, executives will invariably mention the product manager—that is, if she isn’t already in the meeting.

    When I realized this was happening with startling regularity, I began to wonder, Why involve the product manager in a conversation around development and deployment?

    Just like with DevOps years ago, organizational leaders from various disciplines within the same enterprise are suddenly working closely together. There is a movement underway, with the purpose of creating feature-based teams that are organized around a single aspect of a product. The purpose of the teams is meant to break down silos by bringing together disciplines such as engineering and product management. A great example of this is Spotify, a company that created the ultimate #squadgoals approach to enterprise agility by breaking down core organizational units into smaller, autonomous product development team with no more than eight people. Each team or “squad” is accountable for a discrete aspect of the product, from cradle to grave. The company shared some fascinating videos on their engineering culture.

    Why DevProd? Why Now?

    Of course, organizations require speed and agility. Product management requests changes and technical staff implements them. It makes sense they would need to work in close proximity.

    Even though that’s been true for years, there’s still been a dramatic uptick in that collaboration.

    With DevOps, companies increased deployment agility and reduced version release times dramatically, allowing them to stay competitive and react more quickly to the market. Yet, the DevOps movement retains a focus on an engineering unit: “the version.”

    However, software versions are no longer as significant as they once were since full application distributions are no longer needed. Instead, every change to the application can now be considered a feature and can be rolled out separately and in more granularity. DevProd is to continuous software delivery what DevOps is to version updates.

    Developers and product managers have different priorities. Developers think about impact on the broader application and compliance with technical requirements. Product managers think about impact on business KPIs and how to determine that impact. If either group works in a silo, the other group can’t achieve its goals. Instead, they’ll need to work together much more often. That’s DevProd. I’m confident DevProd will have the same, if not more, impact as DevOps did. I know this because the pattern is the same:

    1. Companies try to solve a burning issue and spend resources on developing the solution in-house (e.g., what IT did with log management before Splunk).
    2. They continue to spend resources on evolving and maintaining that solution.
    3. The solution has a real impact on the company’s ability to meet its business goals
    4. The C-suite in that organization buys into the vision and is willing to take ownership of it and assign resources to it.

    It Won’t Be Easy and May Not Be Enough

    It’s an exciting time to work in software, and the fusion of development and product will deliver immense benefits for organizations. However, the DevProd movement is not without its challenges.

    Regardless of how closely they work together, project managers still need the development organization to create experiments and custom experiences. They cannot simply wish them into existence. Because developers will run experiments of their own (improving application performance, using new tools, etc.), it won’t be easy to juggle the business and development-focused experiments.

    All experiments need to be well-coordinated and carefully planned. That will become a necessity as organizations respond to a market that is feature-centric, not version-centric. Features need to be easily controlled and managed in production, by both developers and product managers, without overstepping each other’s actions. Moreover, the need for speed requires product managers to manage features with a click—without creating additional work for developers. At the same time, developers need to be assured that changes don’t break the application or hurt performance.

    Developers and project managers working closely together is necessary step toward solving these challenges. But it may not be enough. In fact, almost every company I talk to has either developed or is contemplating a single system that will allow both developers and project managers to run some level of experimentation.

    Larger companies such as Facebook, Uber and LinkedIn even have developed highly sophisticated experimentation systems in-house.

    What This Means for You

    It’s an exciting time. It is not that often that you witness a movement that gives rise to a new product category that will evolve over time and create a substantial impact.

    For companies to stay competitive, they will need to:

    • Create a DevProd virtual/actual team to align their feature deployment plans as well as share ongoing insights from current deployed features.
    • Implement a feature management platform that provides both control and visibility into the feature deployment and impact.
    • The platform should be implemented by engineering and used by both Engineering and Development.

    Thanks to DevProd, software development is about to take another quantum leap forward.

    — Ronit Belson

  • How DevOps Helped One Firm Get GDPR Right

    How DevOps Helped One Firm Get GDPR Right

    There has been a lot of press about how many, if not most, enterprises are still unprepared to comply with the European Union’s (EU) General Data Protection Regulation (GDPR) just days before it takes effect.

    However, executives from Cybric, a security platform provider, said their company has been ready for some time—and they thank DevOps for that. Their case study also shows how firms that are still struggling to get started before GDPR takes effect can do what is necessary as they integrate compliance with their operations.

    As soon as GDPR appeared on Cybric’s radar screen more than a year ago, the firm began to prepare. GDPR quickly became a compliance and data management process discussed among others on a regular basis and, as part of its DevOps, it involved the participation of all the stakeholders involved.

    “One core part of the culture of DevOps is about collaboration, sharing and communications and making sure you have automated processes in place,” said Mike D. Kail, chief innovation officer at Cybric. “That is, in part, how we were able to do what had to be done for GDPR.”

    Standard Procedure

    Before GDPR was announced, Cybric already had its data flow management processes in order, Kail said. This included keeping transparent records of data sources and the corresponding geographic locations of the individuals in the databases. “We were already analyzing how we processed our data and knew where it was going,” he noted. “We were also always looking at all the ways we could automate how we handled data flows.”

    The different infrastructure, development and business teams, of course, have also played a collaborative role in Cybric’s data flow management processes and resulting GDPR compliance. “Each team figures out how they how they can help, which goes back to the collaborative culture of DevOps,” Kail said. “It’s everybody’s responsibility and not just one team’s concern in the company. In the case of GDPR, the fines and levies can be significant, so everybody should take this seriously and not put it off and say ‘that’s not my problem.’”

    In this way, Cybric was in a position to respond to the demands of GDPR compliance long before the deadline.

    “One result of DevOps is you always take care of where the data is and how it’s being processed in an automated way,” Kail said. “The wrong way consists of manual, out-of-bound efforts for processes and tasks that people ultimately screw up or forget to do on a regular basis. And when that happens, you now have potential compliance issues.”

    GDPR’s Long Arm

    Some firms may be led to believe they are not concerned by GDPR if they do not store customer data from European customers. What they may not realize is that any data—whether it’s personal data for vendors, sales leads or general third-party business contacts for individuals based in a EU country—is protected by the GDPR mandate.

    Cybric, for example, outsourced its customers’ personal data. However, the firm still must comply for individuals who visit its website and submit personal information to download marketing collateral.

    “I think marketing is an area that a lot of people don’t think about too much,” Kail said. “They are like, ‘I don’t have any personal data directly,’ but indirectly, they do have personal data and must be GDPR-compliant.”

    Many organizations have not yet prepared for GDPR since they have not fully understood the regulation and whether it applies to them or not. But although some of GDPR’s terms are opaque, organizations still should not delay compliance just because they do not completely understand what is involved. “I think a lot of people are trying to delay their compliance even though the GDPR deadline is just three weeks away,” Kail said. “And sadly, their databases are just not ready since they have not thought about the complexity of GDPR compliance and certainly not in a DevOps way.”

    Small-to-medium-sized enterprises should also not be concerned about having to invest a disproportionate amount of resources to become compliant compared to what large firms will have to spend. Cybric’s small size—with fewer than 20 full time employees—has actually played to its advantage, Kail said. Additionally, Cybric’s story offers a look at how a small-to-medium-sized enterprise can use DevOps for GDPR.

    “As a small company, putting together a collaborative process with all of the different departments made it much easier for us,” Kail said. “That is also the approach people have to take.”

    — B. Cameron Gain

  • DevOps’ Contribution to UK’s Digital Sector

    DevOps’ Contribution to UK’s Digital Sector

    The UK’s G-Cloud is a national initiative that helps the public sector find and use cloud-based services. This is basically a DevOps initiative that combines cultural tools, practices and philosophies to help increase the ability of an organization to provide services and applications at high speeds, which helps them provide better services to their customers. Its high-security protocols, improved collaboration, reliability, rapid delivery and high speeds makes it better than traditional management processes.

    This technology seeks to breach the gap between private companies and the public sector—in particular, the government—by offering a wide range of digital services. So far, the adoption of DevOps in the UK’s public sector has benefited the UK’s digital sector in numerous ways—for instance, DevOps has enabled the government of the UK to ensure the digital-first experience for the public.

    DevOps enables marketers to establish clear marketing-related analytics. Using its cloud-based technology, marketers get a chance to influence how they can use available data in decision-making and be competitive. Besides, its interdepartmental communication platform offers a solution to companies to change how they operate to meet customer needs. This can easily be done by integrating customer feedback in quality improvement. The use of DevOps in making IT infrastructure, application operations and application development work as one reduced failure rates by more than 50 percent, reduced time to market by 50 percent to 75 percent and increased capacity creation by 25 percent to 30 percent.

    In the engineering digital sector, DevOps has played a crucial role in changing the focus of from software development life cycles (SDLCs) and software to engineering and services. The adoption of DevOps in different sectors—particularly engineering and governments—has shaped its demand and supply over the last decade.

    Before the invention of DevOps as a cloud service, the government was not digitally structured to handle the demands of the population of today. DevOps bridges the gap and offer digital-first services to the public. Since its implementation, DevOps has made operations that were once a dream become a reality for the public. The most benefiting organizations are public organizations; they procure digital cloud services, then move on to use the services to enhance the provision of services to the public through a faster delivery framework that enhances the competitive advantage of the involved organizations.

    The growing use of digital technology in the UK has led to an astronomic type of transformation that is bound to positively affect the infrastructure of the nation moving forward. It is undeniable that more companies using DevOps is spurring a revolution of digital transformation that is bound to change the way services are provided to the public. The public sector in the UK has now a more valuable competitive advantage in that it is able to provide its services in a digital-first kind of experience. DevOps has therefore not only made services available to the digital population of the UK but has enabled access to every single individual in the nation. 

    Conclusion

    The transformation brought about by DevOps will also increase the quality of services by ensuring the continuation of cloud services that provide faster service delivered. However, one unique aspect of the digital transformation related to the adoption of DevOps is that the rate at which IT skills will grow in the public sector will be faster than previously experienced in the public sector. This is bound to be the inevitable future of the use of DevOps in the digital sector. The growth of IT skills to match the use of the advanced technology will enable better use of the cloud services, creating the potential for continued growth in the future.

    — Subramonian

  • Despite DevOps, Firms Unprepared for GDPR

    Despite DevOps, Firms Unprepared for GDPR

    Most organizations are not fully compliant with the European Union’s (EU) General Data Protection Regulation (GDPR) before it takes effect in May, analysts report.

    While DevOps certainly gives organizations the mean to turbo charge the compliance process, stakeholders outside of IT departments often are failing to greenlight this necessary project ahead of the deadline, due largely to uncertainties about the regulation.

    Not being in compliance could hit organizations hard, with fines up to 4 percent of their annual revenues or €20 million.

    The Cloud Security Alliance (CSA) and Netskope, for example, recently released the results of a survey of more than 1,000 respondents about their organizations’ GPDR readiness. The survey revealed a full 83 percent were not very prepared for GDPR. While 85 percent had taken steps for compliancy, only 31 percent had well-defined plans for meeting the provisions of GDPR.

    The GDPR’s “right to erasure,” (53 percent) “data protection by design and by default,” (42 percent), and “records of processing activities” (39 percent) were deemed the biggest stumbling blocks for compliancy, according to the study.

    A separate survey of 700 network security and IT professionals by Enterprise Strategy Group (ESG) showed only 11 percent of firms were completely ready for GDPR ahead of the May 25 deadline. Thirty-three percent said they were prepared to publicly disclose data breaches within the GDPR-mandated 72-hour period, and 33 percent and 44 percent percent of the firms surveyed were mostly or somewhat prepared, respectively.

    Most organizations ESG surveyed also reported they are investing in GDPR, reflecting the seriousness in which they are regarding the mandate. While only 10 percent said they had made or were planning to make major investments in GDPR, 63 percent said they would make or had made incremental investments.

    Analyst firm Varonis made similar findings after completing a survey of 500 IT security experts in the United States and Europe in October, out of which 200 were in the United States and the rest of those surveyed were equally split between the UK, France and Germany.

    Varonis also found 60 percent in the EU and 50 percent in the United States faced serious challenges before they could be GDPR-compliant. However, 70 percent of the firms represented were also convinced GDPR compliancy would give them a competitive edge.

     

    IM000302.JPG

    Where Is the Data?

    Much of the uncertainty about GDPR reflects how organizations often fail to understand where exactly their data is located. While not specific to GDPR, Varonis found many organizations lacked fundamental data-protection protocols. Those surveyed revealed, for example, 21 percent of their organizations’ electronic folders were “open to everyone.” The survey also revealed 54 percent of the firms’ data was stale. Effectively tracking data extending from on-premises servers to across the cloud was often cited as an issue, making it difficult to track the exact origins of customer data and to determine whether it might be subject to GDPR protection or not.

    “We live in a hybrid world in which data isn’t stored entirely on-premises or completely in the cloud. The hybrid model creates additional challenges and potential risk from a security and data governance standpoint,” Ken Spinner, vice president of global field engineering at Varonis, said. “You can’t assume your data is safe in the cloud. If your cloud environment isn’t secure, your data won’t just be in danger of being exposed to your entire organization—it could be accessible to the world.”

    Uncertainties Abound

    A lack of understanding about what GDPR compliance involves is often cited an impediment to making necessary changes for compliancy. Much has been reported, for example, about whether EU citizens, residents, or simply those who are only temporarily on European soil are protected by GDPR. How the EU will impose fines against non-EU organizations also remains murky.  According to ESG data, less than one-third of the respondents of the organizations that took part in its survey fully understand all the GDPR’s requirements.

    These questions, of course, largely need to be resolved by legal counsel. Meanwhile, many legal departments are waiting for more guidance from the EU about uncertainties mentioned above, as well as other remaining questions about the GDPR’s scope.

    In other words, DevOps is ready to do the necessary, while legal and other stakeholders who fully understand what needs to be done must pull the trigger first. Only then can DevOps, consisting of IT operations, QA, InfoSec and development teams take over. When that happens, DevOps’ role will remain critical for making compliance more agile, automated and, ultimately, reliable.

    “DevOps can assist companies in complying with regulations like GDPR by ensuring that key regulatory requirements are part of the initial design criteria, and taken into account when applications are updated,” Varonis’ Spinner said. “GDPR, manual methods of enforcing data policy aren’t realistic solutions because large organizations could have millions of files. DevOps can support GDPR requests with processes and solutions that ensure personal data that falls under the new regulations is protected and used appropriately.”

    — B. Cameron Gain

  • Morpheus Data Extends DevOps Reach

    Morpheus Data Extends DevOps Reach

    Morpheus Data signaled it will pursue a more open strategy when it comes to enabling DevOps teams to manage multiple cloud platforms.

    Brad Parks, vice president of marketing and business development for Morpheus Data, said updates to the company’s namesake platform will make it possible for DevOps teams to swap in third-party modules and components even when Morpheus has already provided that equivalent functionality in the platform. This more open approach is being driven by a recognition that in many cases DevOps teams are being required to implement a specific corporate standard, Parks said.

    The latest edition of the Morpheus cloud management platform adds support for Kubernetes container orchestration software, Terraform configuration management software developed by HashiCorp and Microsoft Azure Resource Manager. Morpheus already has its own JSON/YAML scripting framework. But it’s clear more declarative approaches to managing cloud resources using, for example, Kubernetes or Terraform are starting to gain traction, Parks noted.

    At the same time, Morpheus is adding connectors to cloud services from Oracle and T-Systems.

    At its core, the Morpheus cloud management platform enables DevOps teams to consolidate analytics, reporting, governance, monitoring, logging and data protection within a single platform. Most organizations today either already are or expect to soon employ multiple clouds. Each of those clouds will be running different classes of workloads as IT organizations move to keep their cloud service provider options open. At the same time, Morpheus Data is betting that, rather than mastering separate management consoles for each cloud, organizations will move to centralize the management of multiple clouds as much as possible, Parks said.

    Parks noted Morpheus Data is also starting to see DevOps teams hooking up their preferred continuous integration/continuous deployment (CI/CD) platform to the Morpheus cloud platform. That approach provides them with a platform to manage applications starting the minute a CI/CD pipeline finishes pushing them into production, he said.

    CI/CD platforms and other DevOps tools are great when it comes to building and deploying applications, but currently a gap exists between how those applications are built and deployed versus managed when deployed in a production environment. Morpheus is designed to provide an open flexible approach to closing that gap, Parks said.

    Interest in various approaches to automating the management of IT is clearly on the rise. Most organizations simply don’t have enough IT staff to manage large numbers of applications being deployed at scale. In fact, Parks said Morpheus Data is seeing a marked rise in the number of managed service providers focused on DevOps and cloud management services.

    Regardless of how the integration of DevOps and cloud management is achieved, it’s clear IT processes are being transformed. The day when silos within IT organizations implemented various levels of IT automation based on various capabilities of platforms is starting to give way to a more comprehensive approach. The real challenge now is figuring out how to achieve that goal with the least amount of reasonable disruption to the culture of the IT organization as possible.

    — Mike Vizard

  • Trust and Safety in the Age of Automation

    Trust and Safety in the Age of Automation

    “As with any new technology, it’s really important that we be thinking now about how to do [it] ethically and responsibly.” —Robert High, Vice President and Chief Technology Officer, IBM Watson

    Here’s something to think about: According to an article in The Wall Street Journal, more than 30 countries have defensive weapons run by automation. South Korea has robotic sentries along the DMZ, while Israeli robots patrol the border at Gaza. Russia has robotic vehicles armed with 30mm cannons and anti-tank guided missiles that are well-equipped to battle it out with NATO forces should the need arise.

    Got your attention? Wait, there’s more.

    The SeaHunter is a U.S. military vessel that can travel the high seas on its own for months without a crew to feed or protect. It’s completely unmanned. And, it costs only $20 million per boat, less than half of the $49 million it costs to fire 59 Tomahawk missiles. At that price, the SeaHunter is practically throwaway.

    The SeaHunter, an unmanned oceangoing vessel, can remain at sea for months at a time.

    The implications are profound. The value proposition is hard to resist, Why spend billions on building and manning traditional warships when, for the same expense or less, you can have a navy of predominantly robotic ships patrolling the oceans of the world? And, you can have robotic aircraft, capable of automated takeoff and landing, using these ships as mobile bases. An air force untethered to any land mass is a game changer, particularly when non-state actors can get their hands on a few of them. It sort of brings new meaning to the word “pirate”—no need to board a vessel by force, just get the access credentials.

    And, at the core of it all this modern weaponry is automation and growing capabilities in AI. That should give you something to think about.

    We in DevOps are all about automation. It’s our raison d’être. Our fondness for, and dedication to, automation is admirable. We really have changed the way tech works. Yet, many of us think of our activities as mundane. It’s akin to the nonchalance of a Major League baseball player. They’ve hit the ball so much that many players have forgotten how skillful one needs to be just to foul off a bad pitch. Most people on the planet can’t hit a Big League pitch, let alone make it go foul by intention. But for the pro, it’s no big deal—it’s just part of a day’s work.

    It’s the same for us. We’re so accustomed to automating our world that it’s become no big deal. But, it is a big deal. It’s a very big deal.

    The easiest thing in the world is to trivialize the incredible impact our work has on others. After all, we’re “just working on” an ecommerce site, a dating app, or IoT endpoint for wired bicycles, right? Those of us working in the defense industry might grasp the significance of the work, but for most of us, it’s business as usual.

    Here’s the raw truth: Those of us doing the automation are changing the world. Each line of code we write is intelligence that alters the digital infrastructure of some part of the planet. The code we write will live on well after we’re gone, whether it’s from a company or from the planet. Are we really clear about the impact we have? Yes, some of our tech superstars are asking us to pay attention, but how many of us really are?

    Every day that passes we give automation more power. It might be a little piece of power, such as letting only authorized users access to an application. It might be significant power, such as determining whether a person qualifies for a loan to buy a house. It might be enormous power, such as manipulating political forces toward a desired behavior. Or, it might be the ultimate power: Deciding who gets to live and who gets to die. But, regardless of degree, it is power and we have it. And, we continue to make automation more powerful. As surprising as it may sound, despite appearances otherwise, we’re the ones the world trusts intrinsically to do our work safely and responsibly. A lot of the world’s safety depends on how we do our work. Political leaders don’t write the code. We do.

    Originally,  I was going to call this piece “Death and Destruction in the Age of Automation” and write about how we’ve taken automation way beyond commercial IT into the realm of life-and-death decision-making. I decided otherwise. The world does not need another gloom-and-doom piece, nor does it need another piece of sensational journalism. We’d do better to focus more on the positive things we’ve achieved. Technology is doing a whole lot more good than bad.

    But the fact remains that technology has accompanied death and destruction since Roman catapults assaulted the frontiers of ancient Europe. The thing that’s really different now is that the technology thinks—or, at least, it’s doing a really good job of emulating thinking. And, we’re the ones making it think better. Thus, we should ask, Are we making technology think in ways that are safe and trustworthy? Are we acting in ways that are safe and trustworthy?

    These are big questions. Do most of us really care about them? Or, is it easier to accept our work as mundane, with little consequence in the Big Picture? The nice thing about mundane is that it doesn’t require a lot of ethical contemplation. But, when you consider that the script that deploys an updated container to a Kubernetes cluster is not that far from the one that deploys an unmanned warship to a regional hotspot, things look different. And, contemplating the implications thereof becomes more than an academic exercise in a computer science ethics class.

    More than 2,000 years ago the philosopher Socrates said, “The unexamined life is not worth living.” Given the immense role that technology plays in today’s world and the power it wields—power that continues to move further away from our direct control—we might do well to say, “The unexamined technology is not worth doing.” It’s something to think about, while we still can.

    — Bob Reselman

  • How Python is Transforming the DevOps Landscape

    How Python is Transforming the DevOps Landscape

    DevOps is the combination of operations and software development which reduces or almost eliminates the disconnect between systems administrators who run the infrastructure and software developers who develop applications.

    During software development, when the Agile methodology is utilized properly, a bottleneck often occurs during the frequent operations and deployment phases. New fixes and updates are produced so rapidly in every sprint that the infrastructure teams can be swamped with deployments. To eliminate few of these issues, the operations personnel and application developers are asked to work closely to automate the code delivery to production from deployment.

    As DevOps is an approach for handling change and agility, engineers must master multiple languages. The Python programming language is one of the most crucial components of the DevOps toolchain. Many DevOps teams utilize it for building web applications for visualizing custom data, building custom utilities and more.

    Ansible and other tools are written in Python, which means you can build custom scripts or modules to automate your tasks, for example, or do other things.

    Reasons for Using Python for DevOps

    The accessibility and flexibility of Python are reasons why it is a preferred language for the DevOps toolset. If you are working in the role of DevOps, you must require an adaptable skillset; you must know how to work with multiple languages. For anyone looking to explore new tools and languages, and are curious about technology, the language can form a solid foundation, as it doesn’t require the commitment levels that a specialist language requires.

    In addition, it is a great scripting language, and scripting refers to automation. A few major configuration management tools, including SaltStack and Ansible, are written in Python underscores. This shows how useful the language is when it comes to orchestration and infrastructure automation.

    Python and Ruby are compared often, and with good reason: Both are highly accessible and utilized in applications developed by many organizations. They also feature in the DevOps toolset. We can do almost all the things with Ruby that we do with Python. But, when it comes to syntax, Python gains the upper hand, as it is much more direct than the syntax of Ruby.

    In the realm of learning to debug problems and code, Python has the advantage over Ruby, as it takes a much more direct approach to programming. Its directness and simplicity are invaluable if you are working in DevOps environment and agility is highly important.

    Conclusion

    As a growing number of engineers and developers take on decision-making responsibilities, Python can be the stabilizing language. And not only for DevOps—the language also can be used for developing blockchain, which is the underlying technology for various cryptocurrencies such as bitcoin wallet. As it can be utilized in various ways, it enables you to be open to broad range of technical possibilities. Python is the go-to programming language for the DevOps landscape.

    — savaramravindra

  • Monitoring Your Data Center Like a Google SRE

    Monitoring Your Data Center Like a Google SRE

    Back in the early ’00s, when Google was beginning to expand its portfolio of services beyond search, it encountered a combination of challenges. Some of these emerged from familiar, classic disconnects between developers and operations folks or between IT services and line-of-business owners. Others were brand new, never-before-seen failure modes that arose from providing services on novel cloud platforms—and doing so at planetary scales. 

    To confront these challenges, Google began evolving a discipline called Site Reliability Engineering, about which the company published a very useful and fascinating book in 2016. SRE and DevOps (at least the contemporary version of DevOps that’s expanded into a vision for how IT operations should work in the era of cloud) share a lot of conceptual and an increasing amount of practical DNA; particularly true since cloud software and tooling have now evolved to enable ambitious folks to begin emulating parts of Google’s infrastructure using open source software such as Kubernetes. Google has used the statement, “Class SRE implements DevOps” to title a new (and growing) video playlist by Liz Fong-Jones and Seth Vargo of Google Cloud Platform, showing how and where these disciplines connect and nudging DevOps to consider some key SRE insights.   

    Understanding Google SRE Principles

    First, some basic principles: 

    Failure is normal – Achieving 100 percent uptime for a service is impossible, expensive or pointless (e.g., given the existence of masking error rates among your service’s dependencies). 

    Agree on SLIs and SLOs across your organization – Since failure is normal, you need to agree across your entire organization what availability means, what specific metrics are relevant in determining availability (called SLIs, or service level indicators) and what acceptable availability looks like, numerically, in terms of these metrics (called the SLO, or service level objective). 

    Use agreed-upon SLOs to calculate an “error budget” – SLO is used to define what SREs call the “error budget,” which is a numeric line in the sand (e.g., minutes of service downtime acceptable per month). The error budget is used to encourage collective ownership of service availability and blamelessly resolve disputes about balancing risk and stability. For example, if programmers are releasing risky new features too frequently and compromising availability, this will deplete the error budget. SREs can point to the at-risk error budget, and argue for halting releases and refocusing coders on efforts to improve system resilience. 

    This approach lets the organization as a whole balance speed and risk with stability effectively. Paying attention to this economy encourages investment in strategies that accelerate the business while minimizing risk: writing error- and chaos-tolerant apps, automating away pointless toil, advancing by means of small changes and evaluating “canary” deployments before proceeding with full releases.     

    Monitoring systems are key to making this whole, elegant tranche of DevOps/SRE discipline work. It’s important to note (because, remember: Google isn’t running your data center) that this has nothing to do with what kind of technologies you’re monitoring, the processes you’re wrangling or the specific techniques you might apply to stay above your SLOs. In short, it makes just as much sense to apply SRE metrics discipline to conventional enterprise systems as it does to 12-factor apps running on container orchestration. 

    Applying Google SRE Principles

    So here are a few things Google SRE can tell you about monitoring, specifically: 

    Alert only on failure, or on incipient failure – Alert exhaustion is a real thing, and “paging a human is an expensive use of an employee’s time.” 

    Monitoring is a significant engineering endeavor – Google SRE teams with a dozen or so members typically employ one or two monitoring specialists. But they don’t busy these experts by having them stare at realtimecharts and graphs to spot problems: that’s a kind of work SREs call “toil”—they think it’s ineffective and they know it doesn’t scale. 

    Post-hoc analysis, no magic – Google SREs like simple, fast monitoring systems that help them quickly figure out why problems occurred, after they occurred. They don’t trust magic solutions that try to automate root-cause analysis, and they try to keep alerting rules in general as simple as possible, without complex dependency hierarchies, except for (rare) parts of their systems that are in very stable, unambiguous states (their example of “stable”: when they’ve redirected end-user traffic away from a downed data center, systems can stop reporting on that data center’s latency). Elsewhere, their systems are in constant flux, which causes complex rule-sets to produce excessive alerts. One exception to this general rule about simplicity: Google SREs do build alerts that react to anomalous patterns in end-user request rates, since these affect usability and/or reflect external dependency failures (e.g., carrier failures). 

    Heavy use of “white box” monitoring – Google likes to perform deeply introspective monitoring of target systems grouped by application (called business service monitoring in Opsview Monitor). Viewing related metrics from all systems (e.g., databases, web servers) supporting an application lets them identify root causes with less ambiguity (e.g., is the database really slow, or is there a problem on the network link between the DB and the web host?) 

    Four golden signals – Because part of the point of monitoring is communication, Google SREs strongly favor building SLOs (and SLAs) on small groups of related, easily understood SLI metrics. As has been widely discussed, they believe that measuring “four golden signals”—latency, traffic/demand, errors and saturation—can pinpoint most problems, even in complex systems such as carrier orchestrators with limited workload visibility. It’s important to note, however, that this austere schematic doesn’t automatically confer simplicity, as some monitoring makers have suggested. Google notes that “errors” are intrinsically hugely diverse and range from easy to almost impossible to trap, and “saturation” often depends on monitoring constrained resources (e.g., CPU capacity, RAM, etc.) and carefully testing hypotheses about the levels at which utilization becomes problematic. 

    Conclusion

    The bottom line is that good DevOps monitoring systems need to be more than do-it-yourself toolkits. Though flexibility and configurability are important, more so is the ability of a mature monitoring solution to offer distilled operational intelligence about specific systems and services under observation, along with the ability to group and visualize these systems collectively, as business services. 

    — John Jainschigg