devopsy.kinsta.cloud

Tag: open source

  • OpenStack and Open Source MANO: Technologies for NFV Deployment

    OpenStack and Open Source MANO: Technologies for NFV Deployment

    We have experienced how open source software technologies revolutionized the application development process, which ultimately resulted in digital transformation across various industry verticals. Open source technologies now are disrupting the telecom sector for building 5G internet, which will be powered by network functions virtualization (NFV) and software defined networking (SDN). With NFV and SDN, multiple network functions and control and management operations in telecom networks will be software-driven; enabling the cloud-native and DevOps approach.

    There are various communities emerged to innovate the software stack behind NFV and SDN technologies such as OpenStack, ONAP, ETSI Open Source MANO (OSM), OPNFV, OpenDaylight, Open Networking Foundation (ONF), Open Contrail, Open Baron, On.Lab, CORD, Kubernetes, ONOS and Open-O. In this article, we will take a look at how a combination of data center management platform management platform, OpenStack and ETSI Open Source MANO (OSM) will be useful for NFV deployment.

    OpenStack for NFV

    As we know, OpenStack is mainly known to be the largest pool of open source projects that collectively form the software platform for cloud computing infrastructure. This infrastructure is used widely in private cloud use cases by many enterprises. After an introduction of NFV by ETSI, OpenStack has emerged as a key infrastructure platform for NFV. In most of the NFV deployments, OpenStack is used at the VIM (virtual infrastructure manager) layer to give a standardized interface for managing, monitoring and assessing all resources within the NFV infrastructure.

    Various OpenStack projects (including Tacker, Neutron, Nova, Astara, Congress, Mistral and Senlin) are capable of managing virtualized infrastructure components of NFV environments. As an example, Tacker is utilized to build a generic VNF manager (VNFM) and NFV Orchestrator (NFVO), which help in the deployment and operation of VNFs within NFV infrastructure. Additionally, integrating OpenStack projects introduces various features to NFV infrastructure, including performance features such as huge pages, CPU Pinning, NUMA topology and SR-IOV; service function chaining, networking slicing, scalability, high availability, resiliency and multisite enablement.

    Telecom service providers and enterprises that have implemented their NFV environment with OpenStack include AT&T, China Mobile, SK Telecom, Ericsson, Deutsche Telekom, Comcast and Bloomberg.

    Open Source Mano (OSM) for NFV

    The management and orchestration (MANO) layer is responsible for orchestration and complete lifecycle management of hardware resources and virtual network functions (VNFs). In other words, the MANO layer coordinates NFV infrastructure (NFVI) resources and maps them efficiently to various VNFs. There are various options available as three-dimensional software stacks for MANO. But ETSI-hosted OSM is largely preferred due to large activity at the community level as well as its highly mature framework, production readiness, ease of initiation and constant feeding of use cases by members.

    Virtual network functions (VNFs) forms network services and may need updates for feature addition or patch for functionalities. OSM provides a method to invoke the VNF upgrade operation with minimal impact in the running network service.

    With the continuous community support and involvement for feature innovation, OSM has now evolved to bring a continuous integration and continuous delivery (CI/CD) framework at MANO layer.

    The latest release 4 of OSM has brought a large set of features and enhancements to the OSM framework, impacting functionality, user experience and maturity, which enables various enhancements for NFV MANO from usability and interoperability perspective.

    OSM has steadily adopted the cloud-native principles and can be easily deployed in the cloud, as installation is container-based and runs with the help of container orchestration engine. A new northbound interface aligns with ETSI NFV specification SOL005 and provides dedicated control of the OSM system. Monitoring and closed‐loop capabilities have also been enhanced.

    The next version of OSM release 5 is expected to be launched in November and will be bundled with more 5G-related features such as network slicing and container-based VNFs.

    Why OpenStack and Open Source MANO for MANO layer in NFV?

    Both OpenStack and OSM have a large community known for its rapid pace for innovating NFV and higher contribution by all scale companies to enhance current features and develop new capabilities for core projects under it.

    For NFV, OpenStack standardizes interfaces between NFV elements and infrastructure. OpenStack is used for commercial solutions offerings by companies including Canonical/Ubuntu, Cisco Systems, Ericsson, Huawei, IBM, Juniper, Mirantis, Red Hat, SUSE, VMware and Wind River. A large percentage of VIM deployments are based on OpenStack due to the simplicity in handling and operating various projects targeted toward providing full potential storage, compute and networking for NFV.

    With last two releases (3 & 4), OSM has evolved to support integration for cloud-native approach by enabling CI/CD frameworks into orchestration layers. Cloud readiness involvement of OSM is the key benefit along with OpenStack, which has proven architecture for private as well as public clouds. OSM deployment into NFV infrastructure has become very lean, where one can start with importing docker containers into production. On the other hand, OpenStack is known for enabling simplicity to manage virtualized and containerized infrastructure. Organizations can realize the full benefits out of integration as NFV MANO using OSM and OpenStack due to lean and simple management and deployment.

    Conclusion

    Since the introduction of NFV and SDN, telecom service providers and vendors were evaluating the use case offered for telecom networks, and due to this application of both, were in under maturity stage. Service providers and vendors were concerned about dynamic orchestration and automation of infrastructure deployment and constantly upgrading virtual network functions (VNFs). But due to contribution by open source communities such as OpenStack and Open Source MANO (OSM), they are now at the stage to apply cloud-native principles to 5G network implementation and operations. Furthermore, with contributions by communities to OpenStack and OSM projects, we will see more enhancement to 5G network enhancement technologies including network slicing, mobile edge computing (MEC) and cloud radio access network (cRAN), which will benefit emerging technologies including IoT, AI/machine learning, real-time analytics, augmented reality (AR) and virtual reality (VR) with low latency and high-bandwidth features.

    — Sagar Nangare

  • Selecting the Right Service Virtualization Tool

    Selecting the Right Service Virtualization Tool

    Once you have established that implementing service virtualization (SV) as a means to remove the bottlenecks formed by critical yet hard-to-access dependencies in your test environments is the way to go forward, one important decision to make is which SV tools to use for the process. Most major testing solution vendors (including CA, Micro Focus, Parasoft, Tricentis and more) offer an SV tool as part of their product range. Next to that, smaller companies have emerged for which an SV tool is their primary offering (TrafficParrot, for example). Finally, similar to the test automation ecosystem, many open source offerings have seen the light of day, most notably including WireMock and Hoverfly.

    Similar to what has been going on in the test automation world for a while now, open source SV tools seem to have become the default choice for many development teams when looking for a solution that satisfies their SV needs. In this blog post, I’d like to take a closer look at the pros and cons of this bias toward open source products, as well as provide some tips on how to decide what type of tool works best for your organization and situation.

    Benefits of Open Source and Commercial Solutions

    There are several benefits associated with the use of open source service virtualization tools in general, and these benefits apply equally to SV tools:

    • There’s no risk of vendor lock-in when you’re using an open source tool.
    • There are no license costs associated with open source tools, whereas commercial off-the shelf (COTS) tools obviously do.
    • Open source tools can be extended with specific features in case you need them (and you’ve got the development skills to do so). With COTS tools, you can file a feature request with a vendor, but that’s no guarantee when your request will be addressed (and if it will be addressed at all).

    COTS solutions, on the other hand, have benefits of their own:

    • They come with a richer set of features out of the box, such as support for a wide variety of transport protocols and message formats, etc., whereas open source tools often focus on a specific protocol and/or message format.
    • They generally have a lower level of entry, for example because they offer graphical user interfaces that make them accessible for a larger audience, whereas open source tools may require you to write code or other forms of technical specifications to get them running.
    • Professional support, both online and in house, may be included in the license purchase price, whereas open source tools offer support on a best-effort basis only, most of the time.

    This is by no means a complete list of the pros of both types of tools. Also, some of the benefits mentioned here come with a caveat of their own. For example, the lower initial cost of open source tools (no need to buy a license) is often being made undone by the additional effort it might require to build simulations that are as powerful as those that can be created by commercially licensed tools out of the box. Always consider the total cost of ownership when you’re looking to implement an SV tool, and don’t forget that this TCO extends beyond licensing fees.

    One other important aspect to understand when comparing open source and commercial tools is that they’re typically sold to different audiences and therefore, their creators must appeal to these different audiences accordingly. Open source tools are generally adopted in a bottom-up manner: they’re downloaded and experimented with by a developer or development team and, when successful, might slowly be adopted by a larger audience within the organization. Commercial tools, on the other hand, are often sold to development managers or C-level executives instead, since they hold the budget that is needed to make the purchase of licenses possible. Note that this is a generalization, and it might not work this way in your organization, but in general, I feel the above holds true. (Footnote: I have to say thanks to Tom Akehurst, the creator of the open source SV tool WireMock, for pointing out this important distinction to me.)

    So, now that we’ve seen some of the benefits of both open source and COTS SV tools, I’d like to give you some advice on how to choose the right tool for your organization and situation.

    First of all, understand that no tool is inherently better than another one (even though some tool creators and vendors seem to think otherwise, judging by their promotional and social media outings). Tool A, however, might under certain circumstances be a better fit for a given situation compared to tools B, C and D. What tool works best for a given situation depends on many factors including, but definitely not limited to:

    • The extent to which the required transport protocols and message formats are supported out of the box (better support makes for easier implementation).
    • The level of experience with SV of the people that are going to wield the SV tool.
    • The complexity of the behavior that is to be simulated.

    When you’re looking for the best fit for your SV needs, it is wise to take into account a range of tools, both open source and commercially licensed. Don’t discard the latter category because of the seemingly high initial investment due to licensing costs. As I’ve pointed out before, it might just be that the ease of implementation and the richer set of features provided by a commercially licensed tool give you a return on investment that warrants the initial expense many times over. I’ve seen this happen in practice multiple times, and that’s the reason I now prefer to include a range of tools in my test automation and service virtualization training courses, rather than focusing on a specific tool. I sincerely believe there is no one-stop solution for every situation and I’d rather show people what’s available on the market and help them see what would be the best fit for their own organization than train them in a single solution (with the risk of giving them a hammer and making them see everything as a nail).

    Also, don’t simply go with the first tool promoted by a Google search or a friendly sales representative, as there’s a good chance that it might not be the most efficient choice for your specific situation. Take the time to select a tool carefully. Conduct a proof of concept (or more than one, ideally) that demonstrates the actual value that a tool can bring you, and that proves that a tool fits your development and testing requirements and is a good match for the skill set of your development team (or anybody that is to be made responsible for creating and maintaining the simulations). Consider hiring an independent expert that can help you make the right choice.

    The field of service virtualization has rapidly become mature in recent years, going through the same maturing process we’ve seen happen to test automation in the past. That means that it’s time to take the implementation of SV seriously as well. With software development becoming more distributed, more complex and delivery cycles becoming ever shorter, you can’t afford not to at least consider SV seriously if you want to be able to keep up.

    Hopefully, this blog post has given you valuable information that can help you choose the best SV tool for your situation, no matter if that’s an open source or a commercially licensed tool.

    — Bas Dijkstra

  • How Important Is Open Source for DevOps, Really?

    How Important Is Open Source for DevOps, Really?

    Depending on your perspective, you might believe that DevOps and open source go hand in hand. Or you may think that, quite to the contrary, the two have little to do with each other. There are good arguments to be made for both interpretations.

    Let’s explore these two ways of thinking about the relationship between DevOps and open source.

    Match Made in Heaven?

    In certain respects, the two share a great deal of core philosophical overlap. Both movements emphasize collaboration and rapid innovation.

    That is part of the reason why many major DevOps tools are open source. Examples include Kubernetes, Docker, Git and Jenkins, to name just a few.

    Indeed, it’s hard to imagine any modern DevOps team or CI/CD pipeline that doesn’t rely to a significant extent on open source tools.

    The Tension between DevOps and Open Source

    On the other hand, it’s an undeniable fact that many other tools and resources that are commonly used by DevOps professionals are decidedly not open source. There are plenty of closed-source CI servers and release automation platforms, for example. (There are also many tools in this category that are built on an open source codebase but require commercial licenses and proprietary extensions for real-world use.)

    There is also the public cloud, without which it’s hard to imagine anyone doing DevOps today. Most public cloud services are very proprietary in nature. Not only can you not inspect the source code behind services such as Azure Functions or EC2, but users also have no control over how those services operate. And porting a workload from one cloud to another can be a big challenge.

    In these respects, DevOps is far removed from open source.

    How They Really Intersect

    What to make, then, of the relationship between DevOps and open source? What does it mean that they share important cultural affinities on the one hand, yet DevOps teams hardly shy away from closed-source tools and resources?

    The most obvious conclusion, I think, is that DevOps teams are committed to openness, but they balance that priority with an equal commitment to getting results. In cases where production-quality open source tools exist, such as Kubernetes, most DevOps engineers will eagerly adopt them in place of closed-source alternatives.

    In some cases, however, closed-source solutions are superior. And sometimes proprietary solutions are the only type of solution available, as in the case of the cloud. There is no open source alternative to AWS, unless you build your own cloud, but that requires having infrastructure.

    Some DevOps teams may wish that they could achieve the results they need using only open source tools. But they can’t, and that’s why the relationship between the two can seem contradictory and inconsistent at times.

    — Chris Tozzi

  • Securing Third-Party and Open Source Code Components: A Primer

    Securing Third-Party and Open Source Code Components: A Primer

    The increasing popularity of open source code continues to be a boon for developers across the industry, allowing them to increase efficiency and streamline delivery. But there are security risks to be considered when leveraging open source and commercial code components, as each carries with it a significant risk of becoming the enemy within, creating a vulnerability in the program it helps build.

    Sacrificing Security for Ease of Use

    While the many benefits of open source have been well-touted, the sheer prevalence of code components (open source and commercial) across industries is underestimated. CA Veracode recently found that 83 percent of developers use code components to build web applications, with the average number of components per application at a whopping 73.

    Considering the fact that the vast majority of developers build applications with code components, the number of those familiar with the risks of doing so is startlingly low. A mere 52 percent of companies reported they provide security fixes to components when new security vulnerabilities are discovered. Despite an average of 71 vulnerabilities per application introduced through the use of third-party components, only 23 percent reported testing for vulnerabilities in components at every release.

    Furthermore, only 43 percent of developers reported being aware of the industry-accepted OWASP recommendations for preventing the use of components with known vulnerabilities.

    Where Does Responsibility Lie?

    One clear issue in this software development practice is that responsibility for securing code seems to fall on everyone—and, sometimes, no one. Forty-four percent of respondents claimed that developers are responsible for the continued security of code components, while 31 percent reported that they expect security to handle this task.

    This lack of preparedness and miscommunication leaves the door wide open for bad actors to swoop in and take advantage of vulnerabilities that could so easily be fixed.

    How, then, can teams ensure the security of applications built with code components? The solution lies in three key steps:

    • Start with education: Software development teams must have the tools and training necessary to ensure the code they write is secure. Educating developers on the risks associated with open source and commercial code components is a start, but prioritizing security training and ongoing education is the key to long-term success. Tapping the security team to lead trainings for their developer colleagues is a good way to embrace peer knowledge-sharing and break down silos between the two teams.
    • Introduce clear processes: To alleviate miscommunication and confusion, organizations must implement clear processes for secure software development. Knowing what an application is built of is the first step, as teams can’t secure what they don’t understand. This may seem obvious, but CA Veracode found that only half (53 percent) of organizations keep an inventory of components. Code components should be thoroughly vetted before being worked into an application and then should be added to a working inventory. Managers should be sure each part of the maintenance process has a clear owner on the team so responsibility is always clear. Consistent testing and patching are also best practices when it comes to security.
    • Drive innovation with DevSecOps: All of this comes back to integrating security into the entire software development life cycle (SDLC) and implementing DevSecOps, because a team that operates in silos will never be able to push out secure code. The development, operations and security teams each provide their own unique and critical contribution to the software development process, and combining forces will result in the best possible product.

    While taking the time to secure applications from the perils of code components may slow down development in the short term, safeguarding your organization and its software assets is invaluable in the long term.

    — Pete Chestna

  • Open Source: Is Your DevOps Org Vulnerable to an Equifax-Style Hack?

    Open Source: Is Your DevOps Org Vulnerable to an Equifax-Style Hack?

    More than half of the Fortune 100 could be at risk of falling prey to the same kind of hack that caused devastation at Equifax last year, and it all comes down to poor open source component governance.

    A new report out from Fortune reports that in the year after Apache Software Foundation issued an update to its Apache Struts framework to fix a well-known and critical deserialization vulnerability, more than 10,000 organizations downloaded the vulnerable version instead. Among those were 57 percent of the Fortune 100, according to numbers provided to Fortune by Sonatype.

    The vulnerability is best known to have triggered the catastrophic breach at Equifax in 2017 that exposed the records of 148 million people. It’s a doozy of a flaw—a command injection vulnerability that allows for unauthenticated remote code execution—the kind of opening that gives attackers to gain a powerful foothold in affected systems.

    “Seven months should be enough time for organizations to install the necessary patches and it’s unfortunate that so many still choose to download the older vulnerable versions,” said Nick Bilogorskiy. “There is really no excuse for this.”

    This news should be sobering to DevOps organizations. Increasingly, they’re turning to an assembly-focused mode of software delivery that relies heavily on open source components rather than developers reinventing the wheel with new code. It offers a wake-up call for DevOps organizations to get serious about tracking and securing the third-party code they use.

    In a recent DevSecOps survey conducted by 451 Research on behalf of Synopsys, 40 percent of organizations don’t perform any kind of software composition analysis on their software. That’s problematic, because it means that organizations often don’t know when vulnerable open source components such as Apache Struts are woven into their code base.

    More disconcertingly, even fewer organizations have systems in place that can actually control when, where and what open source components are used within their software development. According to a different DevSecOps survey conducted by Sonatype, 62 percent of organizations don’t have any meaningful controls over which components are in their applications.

    It’s a disconcerting statistic, considering the sheer volume of open source component use in development today—whether in DevOps shops or not. In a talk titled, “We are ALL Equifax” at the DevOps Connect event held at RSA conference last month, Derek Weeks highlighted the fact that open source component download requests have increased 87 times in the last 10 years.

    “The growth rate and consumption volume is massive. And it’s coming into your organization,” Weeks told the audience. “And as a result of this behavior the applications your developers are building are now 80 percent to 90 percent open source components—built from code you did not write from scratch.”

    Sonatype’s survey shows that almost 1 in 3 organizations this year suspect or have verified that they’ve suffered from a breach related to open source components in the last 12 years.

    — Ericka Chickowski

  • Google Looks to Simplify Building of Workflows

    Google Looks to Simplify Building of Workflows

    Google this week launched the beta of Cloud Composer, a managed Apache Airflow service for building workflows that promises to simplify management of integrated DevOps processes. Apache Airflow is an incubation project being developed under the auspices of the Apache Software Foundation that enables IT teams to programmatically author, schedule and monitor workflows using directed acyclic graphs (DAGs) that visualize pipelines running in production, monitor their progress and can be used to troubleshoot issues.

    James Malone, a product manager for Google Cloud, said Cloud Composer leverages Apache Airflow to not only make it simpler to manage workflows, but also port them between various public and private clouds, including on-premises IT environments.

    Cloud Composer also includes the Google Developer Console and Cloud software development kit (SDK), a cloud identity-aware proxy, Stackdriver logging and monitoring, identity access management (IAM) and Python support, along with simplified DAG management and streamlined Airflow runtime.

    Google plans to add support for autoscaling and availability in additional Google Cloud regions over time, Malone said, as well as integrate Google Composer with Kubernetes to extend the reach of Apache Airflow workflows to containers.

    Pricing for Cloud Composer is consumption-based as measured by vCPU/hour, GB/month and GB transferred/month.

    Malone said Cloud Composer will play a critical role in enabling DevOps teams to construct pipelines spanning hybrid cloud computing environments. Today IT administrators regularly make use of text editors or scripts they write to automate a process. But it’s difficult for those scripts to be shared between multiple IT administrators. Cloud Composer and Apache Airflow make it easier to share workflows that automate processes across DevOps teams. Just as importantly, those workflows will be accessible to IT administrators who don’t have the programming skills required to develop a script.

    Those workflows will also play a critical role in unifying DevOps and DataOps processes around a common set of continuous integration/continuous deployment (CI/CD) processes, Malone added.

    Most organizations today are managing multiple clouds in isolation from one another. But as organizations look to build applications faster, they want to be able to deploy those applications on the most optimal platform available without having to develop a unique set of pipelines for each cloud environment. Having an application locked into a specific cloud platform not only is economically unacceptable, it’s also fundamentally inefficient from an application development and deployment perspective.

    Apache Airflow is not the only tool that can be employed to construct pipelines. But Malone noted that because it is an open source Apache project, organizations can be assured that workflows they develop can run anywhere now and into the future. Google chose Apache Airflow to build Cloud Composer largely because an active development community is emerging around this open source project, he said.

    The way IT platforms are managed is transforming rapidly. The days when servers and applications where managed in isolation are over. IT administrators are being asked to manage fleets of server and storage systems running more applications than ever. The tools required to achieve that goal are thankfully now becoming a lot more accessible to everyone in IT.

    — Mike Vizard

  • Eclipse Foundation Charts Cloud-Native Path Forward for Jakarta

    Eclipse Foundation Charts Cloud-Native Path Forward for Jakarta

    A new era begins for the Java community today with a commitment to a Jakarta EE specification that the Eclipse Foundation expects will become cloud-native in short order.

    Jakarta EE formally marks the end of a Java era. While the term Java will still apply to any technologies developed prior to Oracle transferring stewardship of the venerable programming language to the Eclipse Foundation, all future innovations will be delivered under the Jakarta brand.

    A survey of 1,800 Java developers shows that in addition to increasing the overall pace of innovation, the top two priorities of the developer community is to see more emphasis being placed on microservices and integration with the emerging de facto Kubernetes container orchestration platform.

    A full 95 percent of the respondents building microservices said they are employing Java as part of those efforts, while nearly one-third of respondents are already working with Kubernetes. Mike Milinkovich, executive director of the Eclipse Foundation, said that although it’s yet to be determined precisely how Java and Kubernetes will be more tightly coupled, it’s clear the Jakarta community wants to be able to deploy cloud-native applications based on Jakarta. Half of survey respondents said they ran only one-fifth of their Java applications today in a cloud. But more than 30 percent said they expect to be running 60 percent of more of their applications in the cloud within the next two years.

    Milinkovich said with Jakarta under the auspices of the Eclipse Foundation, many more developers from different vendors will be contributing code to advance it. Previous advances to Java have been slow in coming over the years because all contributions to the code base were controlled by Oracle. One of the first things to change under the stewardship of the Eclipse Foundation, he noted, is that Oracle has committed to open-sourcing the Test Compatibility Kits (TCKs), which vendors previously were required to license at significant expense.

    Also expected to witness a faster innovation cycle, Milinkovich said, is Eclipse MicroProfile, a Java framework developed by IBM and Red Hat optimized for microservices, which the Eclipse Foundation expects to be as widely embraced as the Spring framework based on Java that Pivotal Software helped popularize. In fact, there will be a lot more cross-pollination of ideas and concepts across all the major frameworks from writing applications, he added.

    Tighter integration between Jakarta and Kubernetes and a general simplification of the effort required to develop microservices is likely to have a significant impact on DevOps for years to come. Many organizations today are contemplating the degree to which they may want to lift-and-shift existing Java application into the cloud versus rewriting them completely in a new language. With the Eclipse Foundation’s commitment to a cloud-native implementation of Jakarta EE, it’s clear that organizations can have greater confidence that existing investments in Java applications likely will remain relevant in the age of the cloud. In fact, the only decision left for many of them to make is the degree to which they might want to turn monolithic applications written in Java into highly portable microservices based on Jakarta EE.

    — Mike Vizard

  • Study Traces Cybersecurity Issues to Lack of App Supervision

    Study Traces Cybersecurity Issues to Lack of App Supervision

    A global survey of 400 application developers conducted by Vanson Bourne on behalf of Veracode, a unit of CA Technologies that provides software development lifecycle tools, shines a light on how big an issue cybersecurity has become because of organizations failing to update components. The survey finds only 52 percent of developers using commercial or open source components in their applications update those components when a new security vulnerability is announced.

    Pete Chestna, director of developer engagement for the Veracode, says the root cause of the many cybersecurity issues is that developers routinely incorporate libraries into their applications. But, after completing that task, there is no bill of materials that enables development teams or cybersecurity professionals to keep track of when those libraries need to be updated and for what reason. Without that bill of materials, it’s easy to miss a critical update addressing a newly discovered cybersecurity vulnerability, Chestna says.

    The survey finds 83 percent of developers are using commercial and/or open source components, with an average of 73 components being used per application. But only 23 percent test for vulnerabilities in components as they are updated, even though there are an average of 71 vulnerabilities per application built using third-party components. Furthermore, the survey finds only 53 percent of organizations keep an inventory of all components in their applications.

    The reasons organizations rely so much on open source and commercial software versus writing their own code span everything from best practices (53 percent) and an ability to work faster (51 percent) to being able to access advanced capabilities (47 percent) and company policy (47 percent). Whatever the reason, it’s apparent that more applications are being assembled rather than being developed completely from the ground up.

    The top factors considered when selecting modules to be included in an application include functionality (62 percent), performance (48 percent) cost (44 percent), known security vulnerabilities (42 percent) and reliability (40 percent).

    Unfortunately, there’s not much in the way of DevSecOps processes spanning the development process. This report shows that development (44 percent) or security (31 percent) teams are most likely to be responsible for the maintenance of third-party commercial and open source components. But the number of organizations that enable developers and cybersecurity teams to work together remains relatively small. A quarter admit (25 percent) admit they don’t even have a formal application security (AppSec) program in place.

    On the plus side, more organizations (41 percent) say they are likely to choose DevOps as their IT methodology, which, longer term, should put them in a better position to embrace DevSecOps.

    When it comes to open source software, Chestna says that IT organizations would do well to remember that free software is much more akin to a free puppy versus say a traditional gift. Puppies require ongoing food and maintenance that comes at a cost to the owner. When IT organizations elect to employ open source software, they are assuming a set of responsibilities that many of them still don’t fully appreciate, notes Chestna.

    Of course, the good news is that it’s lot easier to swap out a module of code than it is to return a puppy.

    — Mike Vizard

  • Using Inspec to Validate Infrastructure

    Using Inspec to Validate Infrastructure

    So, here we were, at one of the bigger financial corporations in New Zealand, working on an elaborate infrastructure built on Docker containers. It had configurations, SSL certificates, DNS entries and all the other condiments required to spice up a tasty infrastructure curry. Though we were able to stand up our complete infrastructure environments from scratch using automated configuration management tools, we weren’t really sure if it was indeed working correctly, or rather the way it was supposed to. The question staring at us was, Is there a way we can declare the desired state of our infrastructure and then validate our infrastructure against it?

    Inspec to the Rescue

    The open source tool Inspec has been used with considerable success over the last few years to achieve security compliance for infrastructure. The syntax/constructs happens to be unbelievably simple. For instance, let’s consider the code snippet below. It doesn’t really require a lot of programming knowledge to make sense of it. Isn’t it?

    This made us wonder, what if we use Inspec to validate our infrastructure setup? What if we map our infrastructure configurations in an easy-to-read declarative syntax? What if this actually solves our problem?

    In line with the best practices of DevOps, we started with a simple experiment; an experiment that could solve a tangible problem. We had a Docker host which was supposed to have a particular docker container running inside it, but it used to fail quite often, for some reason. We decided to write a test in Inspec to check whether this container was running on this host:

    … and tried executing the test.

    This was just what we wanted!

    Excited with this success, we started writing a battery of tests that validated each and every component of our infrastructure. Within a single day, we churned out 192 tests. The number of tests wasn’t as important as the scope and breadth of the test coverage. Once the tests were there, we plugged them on to our infrastructure pipeline (which was built using Jenkins) to ensure the components align with the desired state. We called them ‘component-readiness’ tests.

    So far so good.

    Now that our components match the desired state, are we still confident that our platform would work the way we intended? Perhaps we need some “platform readiness tests” as well. Can we use Inspec for that, too? The answer: Yes!

    Now that the platform readiness tests were ready, we extended our infrastructure pipeline to incorporate them as well.

    Setting our component and platform readiness tests using Inspec had one more fantastic offering: live documentation for infrastructure. Inspec allows the audit results to be displayed in a variety of report formats, including JUnit, JSON, HTML and others. We used the JUnit report format and displayed them using the Jenkins JUnit Plugin. With this, we were able to have human-readable JUnit reports that doubled as live documentation for our infrastructure.

    Needless to say, this infrastructure testing with Inspec provided that (much needed) confidence on our setup and readiness. Our pipeline is designed in such a way that whenever any infrastructure component changes in our ecosystem, the pipeline is triggered which reconfigures and (if required) rebuilds the ecosystem. These Inspec tests are triggered with this infrastructure pipeline and validates the newly configured infrastructure setup. Thanks to this, we are never in a situation in which the client application running on this infrastructure complains or aborts because of incorrect configurations. Moreover, as described above, the living documentation helps document the expected configuration of the environment.

    So far, the Inspec implementation I have referred to is using the freely available Inspec tool and the Jenkins platform. Inspec can also be combined with Chef Compliance to produce more descriptive reports and charts. Chef Compliance also comes with out-of-the-box CIS profiles that can be used to jump start an Inspec implementation for security compliance and audits.

    Inspec is an easy tool to master with simple and intuitive constructs and one could get up and working quite easily. However, this simplicity has a downside. It’s very easy to speed ahead and gloss over the documentation while missing out on the best practices and tips/techniques to implement the solution in a better way. One way to master this is to have a crack at the newly released Auditing with Inspec certification, which provides the impetus to go through the documentation in detail while preparing for the certification. And, you also get to show off a shiny badge for your efforts!

    In Summary

    Infrastructure validation is quite an important step in any DevOps implementation. Apart from being very useful for security compliance testing, Inspec can be used to validate infrastructure using a simple and easy-to-understand declarative syntax. Moreover, it has a small footprint, a vibrant community looking after it and up-to-date and clear documentation. It definitely is a tool with the potential to provide quick and effective results.

    — Mrinal Mukherjee

  • Rundeck Gains Funding to Support Open Source IT Automation Framework

    Rundeck Gains Funding to Support Open Source IT Automation Framework

    Rundeck has emerged as a popular open source job scheduler and orchestration option for IT automation that enables IT teams to build runbooks that can be easily shared with others. The company behind it—Rundeck Inc.—has garnered $3 million in seed funding to provide enterprise-class support for instances in the form of the Rundeck Operations as a Service Platform.

    Company CEO Stephanie Fohn says the Rundeck framework has gained popularity because it provides an agentless approach to creating runbooks capable of spanning multiple platforms, tools and scripting languages. That approach doesn’t require that software be on each endpoint to be automated or that the IT operations team standardize on one set of tools and languages to start integrating various DevOps processes, said Fohn.

    In fact, the goal is to reduce resistance to DevOps in enterprise IT organizations by making it easier for IT professionals to employ their existing skills to create runbooks that can shared and executed by others, said Fohn.

    Rundeck Inc. claims there already are more than 40,000 IT professionals participating in the open source project, which is already being used by more than 100 commercial organizations.

    IT automation frameworks are not a new idea, but resistance to change traditionally has been high. Many IT professionals have been developing their own scripts for years, which they consider to be intellectual property they take with them from job to job. Frameworks that require them to learn new tools or languages are not readily adopted unless the organization they work for mandates them. But the number of IT organizations that have embraced IT automation frameworks as a percentage of the whole remains less than 30 percent.

    Fohn said commercial support provided by Rundeck Inc. should make more organizations more comfortable adopting an IT automation framework that extends the shell scripts many IT professionals have already developed.

    In the meantime, pressure on IT operations teams to manage IT infrastructure more adroitly at scale only continues to mount. Organizations are deploying and updating more applications at a faster rate than ever. Add to that a mix of emerging microservices technologies based on containers, and it’s apparent the complexity of the IT environment will only increase in the months and years ahead. Fohn said that means it’s almost a sure thing organizations will embrace IT automation and DevOps.

    Rundeck offers an alternative agentless approach to managing IT at scale with the benefits of being free and having a growing population of IT professionals that know how to employ it. The degree to which Rundeck can move beyond a grass-roots initiative to become a bona fide movement remains to be seen. But given the total percentage of organizations that have yet to embrace an IT automation framework, clearly there’s plenty of opportunity.

    — Mike Vizard