Category: DevOps Practice

  • Puppet Adds CIS Benchmark Compliance Service

    Puppet Adds CIS Benchmark Compliance Service

    Puppet this week announced it has added a service that makes it easier to achieve compliance with benchmarks defined by the Center for Internet Security (CIS).

    Yasmin Rajabi, global services strategy manager at Puppet, said the CIS Service offering from Puppet extends the ability to manage infrastructure as code into the realm of compliance. Via the service, analysts hired by Puppet will now scan infrastructure on behalf of customers and then generate a report identifying which machines do not meet CIS benchmarks.

    The service also provides a list of the controls that pass or fail per node as well as scores intended to help IT teams triage issues, said Rajabi. If any drift from a previous level of compliance is detected, IT teams can then use Puppet tools to return those machines to a previous state, she noted.

    The goal is to make available a service that automates what would otherwise be a time-consuming monotonous task internal IT teams normally would have to do themselves using scripts they would have to develop, she added.

    Rajabi said Puppet decided to focus on CIS benchmarks because they are often considered foundational for any number of compliance mandates. Achieving CIS benchmark compliance is about 60% to 70% of the work that might be required to achieve mandates that are required in various vertical industry segments.

    As IT environments become more complex many IT teams don’t have the time to manually assess their compliance with multiple mandates. However, because of audits many IT organizations wind up devoting a significant amount of time to compliance assessments. The CIS Service from Puppet offloads that tasks from IT departments in a way that produces reports that can be shared easily with an auditor, said Rajabi.

    It’s not clear to what degree compliance is about to shift left along with cybersecurity. What is certain is in the wake of the COVID-19 pandemic IT teams will be looking to automate every process imaginable simply because there may be no other way to accomplish a task. Many IT organizations have already either frozen headcount or been forced to reduce the size of their existing IT staff. A service that outsources a compliance process to specialists that are better equipped to handle that task might be a timely alternative.

    Whatever the motivation, most IT teams have no trouble finding ways to contribute to their organization that add a lot more value than achieving compliance. In fact, because most IT teams tend to give the compliance process short shrift because of other pressing demands on their time it’s likely mistakes will be made. Specialists who spend all their time on compliance assessments are likely to do a much better job in a fraction of the time. The challenge is making sure the IT staff doesn’t view that service as an existential threat to its existence. Of course, if they do, chances are high that the IT team in question has much bigger issues to address than simply making sure the right compliance report is being generated at the right time.

  • Site Reliability Engineering 101: DevOps Versus SRE

    Site Reliability Engineering 101: DevOps Versus SRE

    Consider the following scenario.

    An Independent Software Provider (ISV) developed a financial application for a global investment firm that serves global conglomerates, leading central banks, asset managers, brokerage firms and governmental bodies. The development strategy for the application encompassed a thought through DevOps plan with cutting-edge agile tools. This has ensured zero downtime deployment at maximum productivity. The app now handles financial transactions in real-time at an enormous scale, while safeguarding sensitive customer data and facilitating uninterrupted workflow. One unfortunate day, the application crashed and this investment firm suffered a severe backlash (monetarily and morally) from its customers.  

    Here is the backstory: Application’s workflow exchange had crossed its transactional threshold limit, and lack of responsive remedial action crippled the infrastructure. The intelligent automation brought forth by DevOps was confined mainly to the development and deployment environment. The IT operations, thus, remained susceptible to challenges. 

    Decoupling DevOps and RunOps — The Genesis of Site Reliability Engineering (SRE)

    A decade or two ago, companies operated with a legacy IT mindset. IT operations consisted mostly of administrative jobs without automation. This was the time when the code writing, application testing and deploying was done manually. Around 2008 to 2010, automation started getting prominence. Now developers and ops teams worked concurrently toward continuous integration and continuous deployment — backed by the agile software movement. The production team was mainly in charge of the runtime environment. However, they lacked skillsets to manage IT operations, which resulted in application instability, as depicted in the scenario above. 

    Thus, DevOps and RunOps were decoupled, paving the way for SRE — a preventive technique to infuse stability in IT operations.

    Software-First Approach: Brain Stem of SRE

    “SRE is what happens when you ask a software engineer to design an operations team,” said Benjamin Treynor Sloss, VP of engineering at Google.

    This means an SRE function is run by IT operational specialists who code. These engineers implement a software-first approach to automate IT operations and preempt failures. They apply cutting edge software practices to integrate dev and Oops on a single platform, and execute test codes across the continuous environment. Therefore, they carry advanced software skills, including DNS configuration, remediating server, network and infrastructure problems, and fixing application glitches. 

    The software approach codifies every aspect of IT operations to build resiliency within infrastructure and applications. Thus, changes are managed via version control tools and checked for issues leveraging test frameworks, while following the principle of observability.

    SRE, DevOps

    The Principle of Error Budget

    SRE engineers verify the code quality of changes in the application by asking the development team to produce evidence via automated test results. SRE managers can fix service level objectives (SLOs) to gauge the performance of changes in the application. They should set a threshold for permissible minimum application downtime, also known as error budget. If a downtime during any changes in the application is within the scale of the error budget, then SRE teams can approve it. If not, then the changes should be rolled back for improvements to fall within the error budget formula.

    Calculating Error Budget

    A simple formula to calculate error budget is: (system availability percentage) minus (SLO benchmark percentage). Please refer to the System Availability Calculator cheat sheet below.

    Suppose the system availability is 95% and your SLO threshold is 80%. 

    Error budget: 95% – 80% = 15%

    Error budget/month: 108 hours. (At 5% downtime, per day downtime hours is 1.2 hours. Therefore, for 15% it is 1.2 x 3 = 3.6. So, for 30 days it will be 30 x 3.6 = 108 hours).

    Error budget/quarter: 108*3 = 324 hours.

    Quick trivia: Breaking monolithic applications lets us derive SLOs at a granular level.

    Cultural Shift: A Right Step Toward Reliability and Scalability 

    Popular SRE engagement models such as Kitchen Sink (aka Everything SRE), infrastructure teams and embedded SREs, tend to build dedicated teams that lead to a “Silo” SRE environment. The problem with the Silo environment is that it promotes a hands-off approach, which results in a lack of standardization and coordination between teams. So, a sensible approach is shelving off a project-oriented mindset and allowing SRE to grow organically within the whole organization. It starts by appraising the teams of customer principles and instilling a data-driven method for ensuring application reliability and scalability. 

    Organizations must identify a change agent who would create and promote a culture of maximum system availability. The agent can champion this change by practicing the principle of observability, where monitoring is a subset. Observability essentially requires engineering teams to be vigilant of common and complex problems hindering the attendance of reliability and scalability in the application. See the principles of observability below.

    The principles of observability follow a cyclical approach, which ensures maximum application uptime.

    Step Zero: Unlocking Potential of Pyramid of Observability

    Step zero is making employees aware of end-to-end product detail — technical and functional. Until an operational specialist knows what to observe, the subsequent steps in the pyramid of observability remain futile. 

    Also, remember that this culture shift isn’t achievable overnight. It will be successful when practiced sincerely over a few months. 

    DevOps Versus SRE

    People often confuse SRE with DevOps. SRE and DevOps are complementary practices that drive quality in the software development process and maintain application stability. 

    Let’s analyze four key fundamental differences between DevOps and SRE. 

    SRE, DevOps

    Conclusion – SRE Teams as Value Center

    A software product is expected to deliver uninterrupted services. The ideal and optimal condition is maximum uptime with 24/7 service availability. This requires unmatched reliability and ultra-scalability. 

    Therefore, the right mindset will be to treat SRE teams as a value center, which carries a combination of customer-facing skills and sharp technical acumen. Lastly, for SRE to be successful, it is necessary to create SLI driven SLOs, augment capabilities around cloud infrastructure, a smooth inter-team coordination and thrust automation and AI within IT operations.

  • Driverless Cars May Become an Integral Part of Our Lives Soon

    Driverless Cars May Become an Integral Part of Our Lives Soon

    If you live in the downtown areas of Pittsburgh, Palo Alto, Miami or other AV hot spots, you’ve probably been missing the Ford Fusion Hybrids with the bulky lidar sensors in your neighborhoods. The COVID-19 pandemic has resulted in many job cuts as well as work from home orders from the government, and this has forced big driverless car technology companies such as Argo AI, Waymo, Aurora, etc., to halt on-road testing of their vehicles and work remotely.

    This would impact many automobile companies partnered with these firms that had plans to mass produce and release their fully autonomous vehicles by the end of 2021, including the Ford Motor Company, Volkswagen Group, BMW, etc. The only way to bring back the accelerated development and testing of these technologies is by realizing their current value in the consumer market, especially when humans need resources to be delivered to them without spreading the virus further.

    This article will shed insight on some perks that autonomous vehicles provide in mitigating the widespread impact of this virus, as well as whether these advantages make us realize their value in everyday life.

    Home Delivery Without a Human Delivery Worker

    We’ve all tried delivery when in a rush or simply when we’re too lazy to go out and buy food or groceries. This used to be an occasional thing for the most of us; something we wouldn’t do every time we purchased an item because the delivery charge was an additional expense. But COVID-19 has changed our entire lifestyle dramatically, and now most of us order our groceries online or through shopping apps such as Instacart or Amazon, so that we can avoid having to physically go to the store. While this seems like a better alternative, it also puts millions of delivery workers at risk of contracting and spreading the virus, making the outbreak worse than it originally seemed.

    Autonomous delivery vehicles are a potential solution to this issue, since they allow for driverless home delivery of items that customers have purchased online, limiting physical interactions with the items to two point-contact (store shopper and customer). Nuro, a leading self-driving vehicle startup in the U.S. founded by former Google engineers Jiajun Zhu and Dave Ferguson, recently became the first driverless car manufacturer to obtain permission from the Department of Transportation (DOT) to produce and put driverless cars into action for home delivery. The R2, Nuro’s second generation driverless car after the original R1, is a low-speed vehicle that can interpret and adapt to a wide variety of weather and road conditions, while not posing considerable risks for pedestrian safety. Its maximum speed is sealed at 25 mph with self-adjusting acceleration and braking control in response to traffic flow and obstacle detection.

    Like most driverless cars, R2 does not possess a steering wheel, rear-view mirrors and actuator pedals, placing it in the exemption category for vehicle production according to the DOT’s Federal Motor Safety Standards (FMVSS). This is an exemption that is not easily approved by the DOT, with many automobile companies trying their luck at getting their mass driverless car production proposals approved without success, until Nuro recently hit the jackpot.

    Although this exemption has only been approved “with regulatory certainty” as of now, due to the low speeds that R2 can travel, it has enabled Nuro to get more R2s out working in Houston, delivering customer orders from Walmart, Domino’s and Kroger. The R2 is a start to transitioning to a fully driverless delivery experience, which in our current state, is a great way to prevent the spread of the virus from exacerbating further. 

    Could Autonomous Vehicles Be Our New Mail Trucks?

    Every week, thousands of mail workers across the U.S. hop into their mail trucks and drive to multiple neighborhoods and communities that they serve, in order to deliver mail to residents. Receiving mail in a timely manner is very important for the most of us, since we need to pay our bills for water, electricity, gas, etc., every month and most of these statements arrive by mail. During this pandemic, mail workers have been working long hours to get all of our mail delivered to us without delay. Through this process, they haven’t been able to take many days off to spend with their own families or rest at home if they get sick, making them even more vulnerable to catching the coronavirus.

    Autonomous mail trucks are a solution that USPS has been considering since late 2017, and to fulfill this dream the Postal Service has partnered with the University of Michigan and TuSimple, a self-driving truck startup. The plans to put an autonomous mail truck to work have only been extended as test runs to a couple of cities across the U.S., such as Phoenix and Dallas, and about 10 rural routes in Michigan as of May 2019. Although these remain a viable solution to be used in conjunction with traditional mail delivery by mail workers, efforts to develop autonomous mail trucks has declined over the COVID-19 crisis due to other priorities that have sprung up.

    Counter to this thought, introducing autonomous mail delivery vehicles during this time to assist mail workers (as originally planned by USPS) or operate independently with robotic assistance in at least a few areas that have already been tested and approved, would highly facilitate the mail delivery process and reduce the stress and workload of mail workers across the United States.

    The latter option would allow mail to be delivered directly to customers, with robots inside the vehicles serving as mail workers that sort and place mail in each resident’s mailbox. Residents can pick up their mail directly from their mailboxes without needing to worry about social distancing, spreading the virus to the workers or, counteractively, getting the virus from them. Even if mail workers did accompany the autonomous mail trucks, their work would be halved in intensity, since they would only have to sort and place mail in each resident’s mailbox, without needing to worry anymore about physically driving to every neighborhood that they serve. 

    Shifting Supplies from One Location to Another on Pedestrian-Free Routes 

    We’ve long considered autonomous vehicles to be an option to use in factories, truck yards and other less congested routes, primarily to move inventory to different locations. This is a great use for AVs since it involves minimal, if any, contact with humans or pedestrians, so it doesn’t pose a high risk for safety.

    The pandemic has raised a lot of new challenges to implementing this idea across the board, in terms of the inability to commit much time at all to these efforts at the current moment, as well as the difficulty in monitoring test vehicles within facilities with very few workers on site to overview the actual process.

    Despite these challenges, Phantom Auto, an automotive safety solutions company based in California, recently announced their decision to deploy their new driverless trucks — which they call productivity vehicles — across their truck yards to load/unload inventory on their own, without needing a worker to drive the trucks. Phantom Auto calls these trucks “almost” autonomous in terms of their navigational capabilities, since they may still need a little input from remote workers to guide them to the right docking/undocking locations within the yard, but the workers do not have to be physically present on site or inside the vehicle to drive it over themselves.

    This is possible due to new technology called HILT (Human in the Loop Technology), a telecommunication software advancement that allows telemonitoring, remote assistance and, in emergency scenarios, teledriving performed by a worker remotely. Performing these labor-intensive tasks autonomously during normal times would be a great relief on workload for workers employed at truck yards, but during this crisis, this is an absolutely essential step that more factories and truck yards should invest time and efforts on, in order to prevent the spread of the virus amongst their workers.

    Mayo Clinic, an American non-profit organization and academic medical center, has also been working to get coronavirus tests and medical supplies delivered to hospitals on pedestrian-free routes in Jacksonville, FL, to prevent workers from getting exposed to the virus. The driverless vehicles being used to transport these supplies are autonomous shuttles from Beep, Mayo Clinic’s new corporate partner from early April of this year.

    Currently, about four of these shuttles are working independently on different routes in the Jacksonville area, while being monitored by Beep, Mayo Clinic and the Jacksonville Transport Authority Officials, to ensure that the vehicles don’t steer off route. With these constraints and monitoring methods in place, autonomous vehicles can be deployed across a variety of platforms to perform simple, supply shifting tasks that human workers have been undertaking in addition to their primary responsibilities for years, during the crisis and post-pandemic.

    Everything Considered…

    With their day to day work and responsibilities increasing multifold during the pandemic, frontline workers have been doing a lot of overtime, not being able to spend time with their families during and further exposing themselves and their families to the virus, so that the majority of us can stay safe at home and still get our necessities delivered to us. It’s now time for technology to take over, and one possible solution to help ease their work is automating tasks that are laborious, such as grocery and mail delivery, supply shifting, etc.

    Autonomous vehicles that have already been approved for deployment, or in the final stages of testing, can help with performing these tasks since they don’t require a driver. These vehicles can be supervised by a physically present or remote worker to ensure that they follow the right routes and operate at the appropriate speeds

    A few firms have already started taking the initiative of deploying these vehicles to help prevent the spread of the virus amongst their workers and customers, and are witnesses to the overall simplification and ease of the automated process over the manual labor that they had originally employed. The shift to a more automated future that we had originally planned at the end of 2021 should start when we need it most, which is during this time of crisis. So, at this time, are we willing to make the change and invest more efforts in developing and testing autonomous vehicles to perform our everyday tasks?

  • Don’t Tool Up a Broken Value Stream

    Don’t Tool Up a Broken Value Stream

    A well-known failure mode for business process automation is to automate a broken process. The same goes for tooling up value stream management (VSM). If you have an overly complex value stream – immobilized with dependencies, a clumsy way to move work from ideation to engineering or disagreements over who has power over the plan – then trying to tool up your value stream to make it visible is a waste of time and money.

    First, form engineering teams capable of delivering business value, and form program or product owner teams to feed backlog to those engineering teams. You may need a portfolio or strategy team to set prioritization across all of the programs. If you must matrix manage your teams, ensure that the business (not engineering) is the strong side of the matrix.

    Such an engineering team should be able to take a chunk of business value, a small epic, and develop, test and deliver it with very few, if any, dependencies on other teams. How you define business value in this case should be very close to how the customer would define value. It’s okay to have some component teams or service teams provide value to multiple other teams, but organize to have few of those because dependencies kill agile.

    Done well, your portfolio team can have a kanban of epics in which the status of those epics is clear, instead of complicated. This is because there are few teams involved in building those epics. The portfolio team will be able to see epics move frequently. They should all slide over a column every month or three. You don’t need a sophisticated VSM console because it’s so simple. Whatever tool they have their epics in will most likely have a cumulative flow diagram (CFD) and info on lead time, flow time and cycle time for the epics. That’s the level at which they need to manage, and all the info they need is (more often) able to be in a single tool.

    It’s the same all the way down, at the program and delivery team level as well. When features map to sprint goals and when velocity is stable and dependencies are few, each engineering team can understand their own process. They can manage their own value stream with whatever simple tool they happen to be using at that level. 

    To achieve higher levels of velocity, agility and predictability, companies need to connect people, process and tools effectively. Integration has its place, even in messed up situations. It can be a useful tactic to help deal with a complex and hard to understand value stream, but integration and VSM platforms will not automatically fix your situation. That’s because VSM is human.

  • Unlocking Enterprise Blockchain Potential with Low-Code Capabilities

    Unlocking Enterprise Blockchain Potential with Low-Code Capabilities

    Since the rise of Bitcoin more than a decade ago, companies have been trying to harness the potential of blockchain technology for enterprise applications. Industries that used to operate in silos are becoming increasingly connected, and the lines of where one ecosystem ends and another begins are blurring. In this connected age, blockchain can provide organizations with new opportunities to share and collaborate with their partners, other industry-players, government organizations and even academia. 

    On a more tangible front, integration would ultimately help establish new end-customer experience possibilities and establish new revenue streams. Especially, as we continue to navigate forward in the midst of COVID-19, enterprises are looking for innovative ways to adapt their business to best operate in the new world. Beyond that, supply chain operations are more critical than ever before. Across industries, maintaining a steady flow of essential goods, protective equipment and much more is critical to the health and safety of our frontline workers. Blockchain can help manage this flow and mitigate risks, helping deliver on that need for instant agile flexibility.

    Understanding the Opportunities

    Use cases are innumerable and it’s impossible to ignore the ongoing COVID-19 crisis as we discuss the potential applications for any technology. In the short-term, blockchain provides an added layer of transparency and data accuracy — minimizing the risk for any potential fallout from supply and value chain disruption during this time. With this transparency and data accuracy comes improved tracking of important supplies and materials such as PPE and ventilators, streamlined patient consent and triage capabilities for overwhelmed hospital staff and much more.

    With that, the applications will only continue to expand as industry operations return to normal in the hopefully-near future. Government agencies can develop a seamless international credit bureau, for example. As it stands, different countries have different processes and standards for credit scores or distribution of stimulus funding. With blockchain, governments could potentially develop a universal credit system that allows for international travelers and frequent movers to reap the benefits of their positive credit scores, no matter where they are. 

    Manufacturers can use blockchain to truly optimize their supply chain operations — understanding as much information about a purchase order throughout the lifecycle of a product or part. Take a heavy equipment manufacturer, for example. Blockchain provides access to data that will allow them to understand how their equipment is being used by their customers, when each particular piece of machinery will need maintenance, how to optimize predictive and emergency repair and replacement services, and more — all while understanding and contextualizing what the impact of these external operations will have on its own internal supply chain as well. In today’s world, this information is especially valuable for manufacturers who are shifting their focus to help support the development of PPE and other COVID-19 response essentials.

    Within banking and financial services, cross-bank and cross-border transfers are exponentially streamlined with blockchain integration. Instead of requiring validation and authentication from intermediaries, blockchain allows for instant transfers, with the same level of security and assurance. From a functionality standpoint, this also brings an added layer of convenience for both the end-user and the banks involved. These same conveniences provide a crucial layer of service during times of crisis as well. Through blockchain integration, stimulus allocation can be instantly approved and deposited into bank accounts for all citizens, regardless of existing banking preferences like direct deposit.

    Similarly, in the education sector, blockchain can make it easier for student records to be transferred between universities and other academic institutions — whether a student plans to transfer a few credits, transfer full-time or study abroad. 

    These are just a few of the many use cases for enterprise blockchain. So, what’s keeping them from becoming mainstream?

    Acknowledging the Barriers of Entry

    While the benefits are undeniable, adoption has been slow. But there is definitely interest, and adoption can soon surge. Gartner reported back in 2018 that business value added by blockchain adoption is projected to reach a whopping $3.1 trillion by 2030. But, only 10% of businesses will undergo radical blockchain-driven transformation by 2022. 

    Why is adoption slow to start? Companies are dipping their toes into the water, but studies have shown that a majority of enterprises are holding back due to regulatory uncertainty (48%), trust issues (45%) and an inability to bring partners together to build a network (44%). Other studies have reported that the “very high” difficulty in platform selection, a lack of in-house expertise on use case development and questions about privacy are the top barriers of entry. 

    What will change between now and 2030? How will these concerns be directly addressed? To answer this, it is important to note that these barriers of entry derive from a lack of foundational understanding of blockchain. More education will come, and more experts will rise. That said, integration doesn’t happen overnight. Blockchain, like any other emerging technology, takes time to integrate at scale.

    It’s understandable why companies are hesitant to go all in at the moment — as organizations are often strapped for resources, and the talent that is needed to build a blockchain platform from scratch is hard to come by. That said, those who don’t invest early are likely to be left behind by thriving early-adopters. Luckily, there is a solution to the challenges that are holding adoption back — one that takes away the need for resource expenditure and dedicated developer teams.

    Leveraging Low-Code

    Low-code development platforms allow enterprises to reap the benefits of complex code, without the need to dedicate valuable time and resources toward development from the ground-up. “Plug and play” customization allows them to address specific needs within their organization, and prioritize implementation on a smaller scale without the stress of diving head-first into an infrastructural overhaul. 

    Especially during our ongoing COVID-19 crisis, low-code eliminates the need for large dev teams to develop new software applications, allowing for a streamlined, timely transition as organizations dedicate their valuable resources elsewhere to help minimize the negative impact of COVID-19 on their workforce and their surrounding communities.

    Beyond this epidemic, these benefits provide risk-averse C-level decision makers with an easy and confident investment opportunity, as well as disruptive tools that deliver on the growing need for constant innovation — in an era where agility and digital transformation are now a necessity. In fact, the main reasoning behind an organization’s low-code adoption is to accelerate said digital transformation and increase responsiveness to the business (66%, respectively).  

    Low-code directly addresses the aforementioned barriers of entry that come with blockchain investment — or any complex, resource-dependent software investment for that matter. With low-code, even non-developers can build high-tech applications. Devs are often expensive and good talent is often hard to come by. Because of this, the ability to get the same quality output with minimal input is extremely valuable.

    Beyond the simple cost and scarcity of good talent, dev teams often have significant backlogs, which make it hard for them to keep up with industry and business demands. There are benefits of having a non-developer heavily ingrained in the development process as well, as they will have an overall better understanding of the function that the application will serve, and how it fits into the overall business structure.

    Bridging Together Low-Code and Blockchain

    Upon evaluation of the barriers of entry that keep blockchain from widespread adoption, as well as the glaring benefits that low-code programming brings to enterprises with limited resources and talent, the potential for low-code blockchain to minimize enterprise concerns and justify investment is obvious.

    By reducing the need for dedicated network developers that would originally be required to build the blockchain platform from the ground-up, enterprises can build and maintain a trusted network with clicks, not code. Users will be able to create and share blockchain objects with familiar functionality — essentially interacting with the platform in the same way they would interact with any CRM data object.

    The simplicity and ease of use of low-code blockchain makes it so that trusted networks can grow organically, faster. Integration across the entire partner network is simple and can be completed with just a few clicks — making it easier to bring new third-parties on board to join the trusted network, ultimately making the blockchain platform and user network exponentially more valuable in the long-term.

    Blockchain adoption will undoubtedly pick up steam, aided by the low-code capabilities that now exist. The minimized risk and resource expenditure that comes with low-code will make blockchain adoption much more appealing to those who are currently skeptical. Additionally, once low-code serves as the entry-point for enterprise adoption, organizations will begin to expand upon the capabilities of their own blockchain networks — and ultimately drive further innovation. 

    Eventually, we’ll see new ecosystem partnerships forming, and end-customer experiences improving. Early leaders will become more comfortable with the idea of experimenting with new, previously unimaginable use cases of their own.

    Looking Ahead to 2030

    We are approaching a turning point with blockchain in the enterprise. Low-code development platforms will provide enterprises with the opportunity to adopt blockchain technology with minimal risk and proven ROI. As more enterprises build trusted networks through low-code blockchain, industry-agnostic innovation will elevate all participating organizations and deliver truly exceptional results — for both, the network partners and their end-customers as well. 

    The blockchain hype is justified. We have been searching for a way to manifest its potential without over-complicating the process of bringing it to life. Low-code does just this, without fault.

  • GitLab Survey Sees DevOps Gains as Developers Exercise More Control

    GitLab Survey Sees DevOps Gains as Developers Exercise More Control

    A global survey of 3,650 software professionals published by GitLab respondents indicates developers are exercising more direct control over IT environments.

    More than a third of respondents (35%) said developers define and/or create the infrastructure their applications run on, with 14% actually monitoring and responding to infrastructure events. Additionally, 18% of developers instrument code for production monitoring, while 12% serve as an escalation point when there are incidents.

    More than 25% of developers reported feeling solely responsible for security, while 33% of security team members say they own security. A total of 29% said they believe everyone should be responsible for security.

    A quarter of respondents said their organizations have been practicing DevOps for three to five years, while 37% said they have been practicing for one to three years. Nearly 60% of respondents said their organizations deploy code either multiple times a day, once a day or once every few days, a 15% increase over the previous year.

    Brendan O’Leary, senior developer evangelist for GitLab, said that massive jump in the number of organizations capable of deploying code at faster rates indicates many more organizations are starting to move further along the DevOps maturity curve. For example, the survey finds 70% of developers can now provision their own IT environments.

    Well over a third of respondents (38%) said their DevOps implementations include a continuous integration/continuous delivery (CI/CD) platform. Almost 59% of those surveyed use GitLab, while 23% use GitHub and 11% use BitBucket. For builds, 60% use GitLab and 38% use Jenkins. A total of 29% said their DevOps implementations include test automation, while 16% said they have embraced DevSecOps.

    Almost 40% of survey takers said they “partially” use microservices, while a quarter (26%) said they fully use them. Well over a third (38%) have also adopted Kubernetes.

    Despite that progress, however, the survey makes it clear there is still much work to be done. More than 42% said testing still happens too late in the life cycle. In fact, 47% identified testing as the top DevOps bottleneck for the second year in a row. Only 12% said they have completely automated testing.

    When it comes to security, well over a third (36%) find it hard to understand, process and fix any discovered vulnerabilities. About a third (31%) said prioritizing vulnerability remediation is still an uphill battle.

    O’Leary said when it comes to security the most challenging issue remains determining what vulnerabilities to prioritize alongside a long list of other bugs and issues that often need to be addressed. Cybersecurity teams all too often continue to send developers a list of vulnerabilities they have discovered with no context, he noted. Many developers are only made aware of an issue after their code has been merged, which O’Leary said makes it more difficult for them to address a specific security issue.

    It’s not clear any organization will ever achieve complete DevOps nirvana. However, as organizations become more proficient it’s clear the return on investment in DevOps is starting to be achieved as more organizations deliver code faster. The challenge now, of course, is making sure the right secure code is delivered at the right time.

  • Report: Software Delivery Slowed During Pandemic

    Report: Software Delivery Slowed During Pandemic

    A survey of more than 500 IT and development professionals conducted by Atlassian finds two-thirds of respondents have slowed down the frequency of software delivery since the advent of the COVID-19 pandemic.

    Patrick Hill, head of reliability process at Atlassian, said while developers may be creating more code than ever while they work from home, IT operations teams working from home are struggling to keep pace.

    In fact, the survey finds more than half of respondents (51%) reported incident response times have been slower since they began working remotely. At the same time, there’s been a 73% increase in demand for the digital services being provided, the survey finds.

    It’s too early to say to what degree slowing down the application delivery process might be a byproduct of the “new normal.” It’s more likely that IT operations teams need more time to re-engineer processes to return to the software release levels typically achieved prior to the pandemic, said Hill.

    In many cases, IT teams may also need more time to shift toward cloud-based frameworks for managing application release cycles that can be accessed from anywhere. Hill said Atlassian has seen a significant increase in the use of its IT management platforms since March. New signups have increased by approximately 125%, he noted.

    It’s also unclear to what degree an application release backlog may be building inside organizations. While developers may be launching more projects, Hill said many of those projects may be only to address bug fixes. As such, it may not take that long for IT operations teams to recover from the disruption caused by the pandemic.

    Nevertheless, Hill said most organizations are still coming to terms with the human impact of the pandemic. Many organizations, for example, might not decide to bring IT teams back into the office at all. In fact, there may not be any office to return to—many organizations might conclude there is no need, especially if productivity levels have not declined substantially. No doubt many organizations consequently will be closely monitoring application release cycles in the weeks and months ahead to determine to what impact working from home is having on IT productivity.

    Of course, not every member of the IT team is cut out to work from home. While many IT professionals may be introverts by nature, there are also a great many distractions at home that adversely impact productivity. Conversely, some IT professionals might even discover they have begun to work longer hours.

    In the meantime, Hill said IT managers would be well-advised to focus more on outcomes than metrics. The amount of code being released in a given week is not likely to be as important as making sure the right code is being released at the right time. With more demand than ever for application services, the greater the need there is for DevOps teams to focus their efforts on the components of an application that make the most critical difference to the business.

  • How to Scale Microservices CI/CD Pipelines

    How to Scale Microservices CI/CD Pipelines

    As opposed to a traditional monolith, microservices architecture can theoretically bring many benefits. Microservices decouple software elements, enable reusable components and allow independent development cycles. However, in practice, microservices are prone to many issues.

    For example, teams accustomed to a traditional monolithic build process may find it difficult to reshape their established practices. Furthermore, as the number of microservices increases into the hundreds, managing separate CI/CD pipelines for each service becomes untenable, especially as DevOps components aren’t always optimized to fit every pipeline precisely.

    These are some of the real-world problems that Dan Garfield, chief technology evangelist at Codefresh, brought up in episode five of the DevNetwork Dev Professional Series. In this article, we follow his talk to examine the benefits of microservices, their hangups in a real-world CI/CD case study and solutions for scaling delivery as complexity and numbers increase.

    Monoliths in Space

    To get a better idea of why we need microservices, Garfield uses an analogy from space—the Apollo 13 mission. Apollo 13 was made of two ships: the lunar module and the command module. In software terms, these were essentially two giant monoliths.

    When the command module experienced an oxygen failure, its crew needed to repurpose oxygen filters from the lunar module. However, the lunar module engineers designed their oxygen filters differently, causing a major headache to reassemble and retrofit these parts.

    This analogy goes to show how monolithic-style development can create tunnel vision. When teams adopt guidelines that work only for the monolith, they are usually incompatible externally. This stifles the possibility of reusable components, thus decreasing code value and extending overall development effort.

    “Being able to reuse components and scale independently is the benefit of microservices,” said Garfield.

    Expedia’s Microservices Journey

    To see real-world microservices architecture in action, take Expedia. Previously, Expedia services had existed within monoliths, which meant much duplication. Services were detrimentally tethered, meaning if a service broke you had to fix the entire monolith.

    Garfield describes how Expedia isolated its architecture into individual microservices to represent abilities such as car search, car sorting, bookings, airline search and others. While this aided elasticity and usability, the microservices journey came with issues.

    Problems in Microservice Heaven

    Expedia ran into multiple problems within their microservices journey. Though Expedia had valid business reasons to adopt microservices, “they didn’t have a plan for how to scale their CI/CD that was realistic,” expressed Garfield.

    Foremost, as the number of microservices expanded into the hundreds and thousands, standardizing CI/CD became quite difficult. The organization was using one CI/CD pipeline per service, meaning they now had to support thousands of pipelines, each connected to an individual Git repository. A microservice may also have different pipelines for production, internal use or testing environments, leading to even more duplication.

    When you scale microservices, you also encounter a new set of roadblocks related to network and storage. This became extremely evident when Expedia realized the sheer network bandwidth and related costs from running validation and integration testing on thousands of separate CI/CD pipelines.

    “The scale of automation itself is where most people have the biggest bottlenecks,” said Garfield.

    Consolidating codebases with plugins and shared libraries became a common issue. Expedia developers experienced issues with Jenkins and were forced to programmatically or manually edit each pipeline, copy and pasting code from one to the next. In short, the flow was not easy, and differing approaches between teams led to shadow IT.

    Solutions For Scaling Microservices

    To help ease large-scale microservice environments and CI/CD pipelines, Garfield recommended taking a few approaches:

    • Use a container-based pipeline: The first principle Garfield suggested is using a container-based pipeline. If a pipeline is containerized, you can run it independently with different language versions. According to Garfield, “shared libraries are not the solution,” as they create version-specific conflicts. Using a Docker image is better than shared libraries because users can self-serve images with whatever version they want.

    There are many resources on Docker Hub to ease common CI/CD steps such as building a Docker image, git clone, running unit tests, linting projects and security scanning. Using these images can help avoid wasted development effort.

    • Consolidate to a single pipeline that operates with context: Instead of managing thousands of pipelines for all your microservices, Garfield suggested you use a single malleable pipeline for all integrations and deployments. His method involves Triggers, which carry information to direct actions.

    In this setup, Triggers carry context, or metadata, allowing a pipeline to change its behavior accordingly. These Triggers could be time-based or centered around actions like a repo Git commit, or pushing a new image. When a trigger is initiated, the trigger brings in relevant dependencies to perform an action, such as cloning a repository or cloning a codebase. Further steps can then consume these unique criteria.

    Garfield demonstrated how this is accomplished in practice within the Codefresh CI platform with native Kubernetes integrations. The platform also enables you to filter and view by certain criteria; a window into commits and changes.

    Garfield acknowledged this overall strategy does mean making microservices a bit more uniform, but perhaps the ease of deployment could outweigh the development constraints.

    • Adopt canary release testing: If an organization has hundreds or thousands of microservices, spinning up each microservice to run independent tests can quickly become cost-prohibitive. As Garfield noted, “testing early becomes less useful as infrastructure complexity arises.”

    To solve this issue, teams can adopt a canary release strategy. Canary release is a practice in which new software is first released and tested among a small subgroup of users. It’s a good way to “limit the blast radius of a change that goes wrong,” said Garfield.

    Lasting Image: Pipeline Reusability

    Adopting a single container-based CI pipeline could help resolve the expanding set of issues with large-scale microservices testing and deployment. By templatizing pipelines, many microservices could abstract complexity and reuse the same modular components. Essentially, make CI/CD reusable in the same way microservices are.

  • LogicMonitor Releases IT Downtime Detection and Mitigation Study

    LogicMonitor Releases IT Downtime Detection and Mitigation Study

    Although 99.999% availability may be IT’s ambition, it’s far from reality. In March 2019, Google’s Gmail experienced a 4.5-hour global outage. In the same month, Facebook suffered a 14-hour outage, its most massive outage to date, crippling app-accessibility worldwide.

    Downtime and low availability will likely occur to all systems eventually, especially when introducing code changes. When outages occur, it’s truly the detection systems and smart mitigation processes that separate quick rebounds from long recoveries.

    LogicMonitor has released its IT Downtime Detection and Mitigation Report, a survey of IT professionals to uncover trends and tactics impacting availability in 2020. The report surveyed over 300 IT design makers at organizations with 2,500 or more employees within the U.S., U.K., Australia and New Zealand.

    Below, we examine the LogicMonitor report’s key findings. We’ll discover, on average, what sort of projects are causing the most outages and brownouts. We’ll also see what strategies top IT teams are using to prevent, detect and mitigate disruptions.

    Which Transformation Initiatives Hurt Availability the Most?

    Ninety-six percent of organizations surveyed had experienced at least one outage in the past three years. But why? One reason LogicMonitor conducted this report was to pinpoint which digital transformation initiatives and IT trends are the leading contributors to high-profile outages and brownouts.

    When determining causation, answers revolve around novel cloud IT initiatives. Fifty-nine percent of respondents felt that mobile computing was causing more brownouts/outages. Fifty-seven percent found AI and edge computing were making outages more common. Other high-ranking reasons included digital transformation (57%) and IoT (53%).

    According to the report, “LogicMonitor’s research suggests that IT decision-makers hold IT transformation initiatives responsible for increasingly frequent outages and brownouts.”

    Though we generally view digital transformation initiatives in a positive light, they do come with caveats. The move from private to cloud infrastructure, for example, can bring unforeseen costs. Also, some authorities note a lack of talent specializing in cloud and hybrid environments. The report also suggested that accelerated cloud technologies may require time to mend before they realize positive business returns (and higher availability).

    Critical Strategies for Preventing Outages

    Another goal of the study was to discover what sort of tactics IT professionals are actively using to prevent outages.

    The study exposed many strategies that IT folks are currently undertaking. The top three tactics were performing preventative maintenance (75%), reviewing system logs (71%) and increasing the capacity of systems (71%). Other top strategies included designing redundancy into systems, and keeping an eye on customer support tickets.

    With the rise of a completely remote workforce, some IT executives do anticipate higher expenditures due to AI and cloud architecture, which could bring added pressure. However, it’s worthy to note the cloud hasn’t negatively affected all groups, or even reached all companies. According to another study conducted by Adobe and Fortune, only one in three organizations store their data in a public cloud. Of course, one could argue that in doing so, such companies substitute innovation for less breaking change.

    Another necessary aspect to keep a vigilant eye on the digital ecosystem is proactive monitoring. According to the report, 74% of teams rely on proactive monitoring to detect outages. Continuous monitoring has been a philosophy trumpeted time and time again by DevOps professionals to enable a better release frequency and a window to detrimental interruptions.

    Careful analysis ahead of time, active monitoring systems to spot problems and continuously reviewing system logs were the most important tactics found being applied by IT leaders to mitigate found outages and brownouts.

    Best Practices: Prevent, Detect, Mitigate

    In terms of takeaways (other than the ultimate LogicMonitor product sell), the IT Downtime Detection and Mitigation Report does reveal some interesting trends on what IT leaders view to be leading causes of system outages. It will be interesting to see how this evolves and what technologies mature (or remain headaches) into the future.

    The report leaves IT professionals with some common, albeit helpful, advice to consider going forward:

    • Actively monitor IT systems.
    • Optimize IT systems with forecasting.
    • Prepare for tomorrow’s IT norms today.
    • Use intelligent monitoring.
    • Design redundancy into your systems.
  • Atlassian Acquires Halp to Gain Help Desk Based on Slack

    Atlassian Acquires Halp to Gain Help Desk Based on Slack

    Atlassian this week announced it has acquired Halp, a provider of a help desk platform that employs Slack as its user interface.

    Steve Goldsmith, head of product integrations at Atlassian, said Halp will provide organizations with a way to manage lightweight processes via an interface that is already commonly used within many organizations. The goal is to provide a level of integration between Halp and the JIRA Service Desk platform from Atlassian to handle more complex requests, he said.

    The Halp platform has already been integrated with Confluence, a knowledge base platform from Atlassian that enables help desk staff to share best practices for resolving known issues. Deeper integrations with JIRA and Confluence will be forthcoming, added Goldsmith.

    At the same time, Atlassian is working on tighter integration between its service desk platform and Microsoft Teams, which has emerged as a popular alternative to Slack, he noted.

    Halp CEO Fletcher Richman said that approach has gained traction because the biggest challenge when it comes to help desks is making it as easy as possible for an employee to generate a help ticket. Halp addresses that issue by enabling end users to simply drag and drop on an emoji to generate a help desk ticket.

    The help desk staff is then provided a web interface through which they can remotely manage tickets and respond to requests via Slack. That approach eliminates the need for the help desk staff to be physically located in an office or employ a virtual private network (VPN) to access a help desk platform running on a local area network, he said.

    Richman noted too often employees wind up supporting each other because the process of generating a ticket is too complex. At a time when many employees are working from home to help combat the COVID-19 pandemic, it’s not really possible for employees to currently provide that mutual support.

    It’s not really clear to how many employees might ever be heading back into an office at this point, so Halp also provides a way for remote employees to more easily remain productive, he added.

    Slack and JIRA, of course, are already de facto standards among many DevOps teams. Halp should provide developers with a means to instantly resolve a wide range of low-level issues in near real-time by generating a help desk ticket via Slack.

    Richman noted that for any IT service management (ITSM) platform to be successful, it needs to meet users where they are. If the platform is too complex or intimidating, users will look to their fellow employees for help first. Before long, however, all the mutual support saps productivity. By making help just a simple Slack message away, many more issues will be resolved by help desk personnel who already know the right answer, he said.

    It’s too early to say how many more help desk tickets might be generated because of Slack. However, in the wake of the rush to remotely enable the entire enterprise, the need for an easier way to provide support is now more apparent than ever.