devopsy.kinsta.cloud

Author: Ericka Chickowski

  • 8 Hot Takes: Will We See a Monolithic Renaissance?

    8 Hot Takes: Will We See a Monolithic Renaissance?

    Is the DevOps world slowly backing away from microservices and to a monolithic renaissance? After years of proselytization about the benefits (Flexibility! Resilience! Support faster release speeds!) of microservices and serverless architecture, there are signs of a potential backlash against the singularly fragmented microservices architectural approach.

    The argument in favor of rolling back to monolithic architectures feels similar to the cloud repatriation debate. Just as with a lot of cloud-first transformations—often in lockstep with them, actually—many organizations moved headlong toward microservices without a case-by-case business or technical rationalization for the change. As the shiny newness of microservices has dulled under that approach’s daily exposure to real-world deployment and business challenges, the monolith isn’t looking so bad in a lot of situations.

    The monolithic versus microservices debate boiled over the last few weeks in the wake of the publication of a case study earlier this month from the Amazon Prime Video team. The analysis from Marcin Kolny, senior software development engineer for Prime Video, reported that his team was experiencing scaling bottlenecks in its audio/video quality inspection tooling, which was architected as a distributed system using serverless components. By rearchitecting the tool as a monolith, they were not only able to increase scaling capabilities but drastically reduced the infrastructure costs around that tool.

    “Moving our service to a monolith reduced our infrastructure cost by over 90%,” Kolny wrote.

    The announcement elicited a fair degree of hyperbole on the topic, but it also spurred a very organic airing of the pros and cons of monolithic, microservices and serverless architectures.

    The following are 10 different perspectives spurred on by this online debate from a range of CTO and engineering experts. Some of them are diametrically opposed to one another, many are overlapping in their opinions. But as a body, they point to the general (and very pragmatic) consensus that there simply will never be a one-size-fits-all approach to software architectures.

    Microservices a Siren Song for Complication

    “It’s clear that in practice, microservices pose perhaps the biggest siren song for needlessly complicating your system. And serverless only makes it worse. Replacing method calls and module separations with network invocations and service partitioning within a single, coherent team and application is madness in almost all cases.”

    —David Heinemeier Hansson, creator of Ruby on Rails and CTO at 37signals (maker of Basecamp and HEY)

    Architectures Aren’t Good or Bad

    “Application architectures are neither good nor bad, nor are they appropriate for every use case. Organizations need to step back and consider carefully what it is they’re trying to achieve and which architecture might best serve that need instead of choosing the most “modern” architecture because, well, it’s in vogue.”

    —Lori MacVittie, distinguished engineer in office of CTO at F5Networks

    Distributed Architectures Will Suit AI-Driven Future

    “In the ongoing debate between monolithic and serverless architectures, it is important to acknowledge that both approaches have their merits and considerations. Monolithic architectures have been prevalent in large human-centric organizations due to their management advantages and historical use cases. However, as the world continues its rapid progression towards a more AI-driven future, the landscape is shifting towards architectures that embrace fluidity, autonomy, and distribution.”

    — Reuven Cohen, veteran CTO and entrepreneur, mentor for TechStars

    Monolithic and Serverless Each Have Pros And Cons

    “Over the last 15 years, everyone has wanted to move to serverless microservices-based horizontally-scaled architecture – but it is not ideal for every application’s specific needs.  As with any choice in life and in business, there are always tradeoffs;  you must evaluate and choose the appropriate architecture based on your business and technology needs. Another important dimension to this is workforce skills for application architecture.

    —Hemang Davé, client technical leader and CTO at Kyndryl

    The Amazon Rearchitect Wasn’t Even Truly a Monolith

    “The problem is that they called this refactoring a microservice to monolith transition when it’s clearly a microservice refactoring step, and it is exactly what I recommend people do in my talks about serverless first. I don’t advocate “serverless only”, and I recommended that if you need sustained high traffic, low latency and higher efficiency, then you should re-implement your rapid prototype as a continuously running autoscaled container as part of a larger serverless event-driven architecture, which is what they did. If you built it as a microservice to start with, it would probably take longer (especially as you have to make lots of decisions about how to build and run it) and be less able to iterate as you figure out exactly what you are trying to build.”

    —Adrian Cockcroft, longtime DevOps advocate and veteran architect with stints at eBay, Netflix, Amazon, and Battery Ventures, among others

    One Tool as a Monolith, Maybe, But Not All of Amazon

    “Microservices, while duplicative and inefficient from a compute standpoint, allow you to release stuff really quickly, which is a tradeoff that most organizations at scale are willing to make.

    Microservices exist to reduce communication pathways—not to solve technical scaling challenges. At a certain point, teams stop functioning due to the communication overhead. A seven-person team has 21 communication pathways, whereas a 100-person team has 4,950. I work with large retailers selling billions of dollars a year online, and putting everything into a single monolith just doesn’t work. Could the entirety of Amazon.com be deployed as a single monolith? No way.”

    —Kelly Goetsch, chief strategy officer at commercetools

    Architectural Tension Always Exists Between Speed and Maintainability

    “Speed to market and speed to develop does not always equate to long-term scalability and maintainability. You must actively balance your investments across these two critical pillars to build a viable product. Active rebalancing and reconsideration of the stack and technology mix for your FinOps and BizOps context is both art and science.

    Cloud-native distributed systems paradigms are here to stay. The Prime Video folks have mistakenly labeled a properly factored, highly coupled data-intensive processing step a monolith—it’s at best a rocky outcropping in their distributed microservices forest. “

    —Ajish George, managing director of cyber data & analytics for State Street

    The Shiny Thing is Not Always the Best Thing

    “The news that AWS discovered that serverless and microservices aren’t for everything is surprising. Not because it didn’t work out for that use case; but because the wrong use cases have been so well-known for the serverless/microservices pattern and should have been obvious to Prime Video. It’s worth remembering: That shiny new thing isn’t necessarily a better way of trapping mice. Right tool for the right job.”

    —Doug Vanderweide, director of cloud architecture for Avanade

  • 4 Risk Factors Devs Should Consider for ChatGPT Integrations

    4 Risk Factors Devs Should Consider for ChatGPT Integrations

    It’s only been a couple of months since OpenAI introduced its latest low-cost API for developers to leverage ChatGPT in their applications and already many engineering teams have jumped headlong into new integrations. But developers experimenting with the possibilities of how ChatGPT can boost their software should temper that with some risk analysis and threat modeling before deployment. So say a range of cybersecurity and risk experts, who warn organizations that leverage the ChatGPT API in their software that they’ll need to navigate a minefield of privacy, data governance, and security operational risks.

    “I use ChatGPT Pro and think it’s an exciting tool for holiday cards, poems for my kid, writing inspiration, first drafts and search,” says Patrick Hall, data scientist and co-founder of BNH.AI, a boutique law firm focused on minimizing legal and technical risks of AI. He’s also the author of the upcoming book Machine Learning for High-Risk Applications. “I get concerned about generative AI used as a consumer product in high-risk domains, such as consumer finance or the law.”

    With that framing in mind, the following are a few of the areas that experts like Hall suggest that developers, DevOps teams and CTOs start thinking about to avoid some potentially costly unintended consequences of ChatGPT integration in their software.

    Data Privacy and Governance Concerns

    One of the most immediate and obvious risk concerns IT leadership, DevOps teams and privacy experts are worried about is what happens to sensitive data when users enter it into a ChatGPT-backed prompt—whether directly or through an app using the API. Be it a customer entering personally identifiable information (PII) into an external-facing app or an employee entering proprietary company information in an internal tool using ChatGPT’s API, these use patterns open up a world of data exposure and regulatory compliance concerns. The good news is that OpenAI does not use data submitted through the API to train its models, but this shouldn’t automatically give devs peace of mind. They’ll need to be sure that using generative AI in an app truly aligns with their company’s privacy policies and with any local, federal or international data privacy laws their firm is beholden to, said Hall.

    “Although OpenAI appears to maintain a robust security and privacy program and there are steps you can take on your own to limit data sharing, some people are still worried,” said Walter Haydock, founder and CEO of StackAware.

    Among Haydock’s side projects, he is currently developing a tool for sanitizing data before sending it to OpenAI. Called GPT-Guard, it’s still in beta and it would need to be run on a local machine or in the cloud to sanitize data before sending it via API. But it provides some rapid prototyping of the kind of methods and tools developers may need to think about as they design their applications to interact with ChatGPT.

    Intellectual Property Infringement

    Developers should also be thinking through intellectual property infringement concerns when it comes to the responses generated by an app tapping into ChatGPT, Hall said.

    “Devs need to consider whether a generative AI system may be coughing up other people’s licensed or copyrighted intellectual property as responses,” he said.

    This is likely going to be a tricky situation that will likely be tested by the courts in the coming years. For example, will applications using ChatGPT be required to compensate licensed copyright holders for certain kinds of responses using their material? These are the kinds of questions that developers and the organizations they work for should be asking themselves, said Frank Huerta, CEO of Curtail Security and a longtime DevOps expert.

    “For example, what’s the mechanism for resolving IP and copyright disputes?” he asked. “Is ChatGPT going to have its own IP? Is it going to generate it—and will it put that in the public domain? Who owns the copyright to a generated response? None of that has been considered yet, and it’s going to come up fast.”

    Adding New Cybersecurity Threat Vectors

    In addition to all of the concerns about the data, developers integrating ChatGPT into their applications also need to consider what that does to the software’s attack surface. ‘Never trust user input’ has always been a good rule of thumb for developers, and that advice is just as true for large language model (LLM) prompts as anything else.

    A recent example scenario laid out by a data scientist on Twitter showed one of the myriad ways that malicious actors could carry out prompt injections in the same way they’d carry out a SQL injection. In that scenario, if a recruiter used ChatGPT to crawl LinkedIn profiles for a certain keyword and send an email to the people whose profiles included that keyword, it’d be possible to add text to a profile that sends an email chiding the recruiter for using an automated tool to send out emails. That’s a fairly low-stakes example, but it shows the possibility for injection attacks, said Andy Patel, a security researcher for WithSecure who has been digging into AI risks.

    “So, if you are integrating it, you should think about things like that. You should think about the fact that people will determine that you are using a large language model to do something, to automate something, and then they’ll essentially SQL injection attack it,” he said.

    Problems With Bias

    Finally, developers and software design teams should be cognizant of the potential problems that AI bias can introduce into their software if they lean on the generated output from ChatGPT to trigger actions or to inform the user.

    “Generative AI systems continue to make wrong, offensive or otherwise problematic content. Being wrong can give rise to negligence issues and biased content can lead to discrimination issues,” Hall warned. “Don’t embarrass yourself, offend others or run afoul of the law by not testing and further protect yourself by screening the heck out of generated content.”

  • 5 Ways DevSecOps Can Manage Software Supply Chains

    5 Ways DevSecOps Can Manage Software Supply Chains

    The unbridled use of open source components within the software supply chain is on a major uptick, according to new research. Even as this surge in open source dependencies fuels faster innovation, the study shows that it comes with high cybersecurity costs, as the number of breaches related to these components is similarly on the rise.

    However, the numbers also indicate there are some clear practices used by the best DevSecOps teams to minimize the risk while maximizing the speed and agility that third-party components offer.

    Sonatype and research partners from IT Revolution and Galois made a thorough analysis of download patterns, software component makeup and DevSecOps behaviors around open source components and third-party libraries for the “2019 State of the Software Supply Chain Report.” Their analysis found that the rate of open source component releases has picked up by 75% in the last two years and that download requests from teams using these components have risen 68% year over year.

    Most tellingly, the study showed that among modern applications today, 85% of the average code base is comprised of open source componentry. Not only are the majority of applications using open source components and libraries, but also they depend on them heavily.

    “The practice of assembling open source component releases into the form of an application screams of efficiency. Developers no longer need to code every line from scratch,” according to the report. “Developers can download component releases in seconds that deliver new capabilities, built by experts outside of their organizations who make their code freely available to others.”

    Perhaps not coincidentally, the study noted a parallel rise in breaches associated with open source component use. Last year 1 in 4 organizations suspected or verified they experienced a breach related to open source components in the last 12 months. That’s a slight improvement over last year’s stats, which had it pinned at 1 in 3 organizations. But since 2014, that rate of breaches has risen by 71%.

    The trouble is, many of the components developers download and integrate into their code are full of dangerous vulnerabilities. For example, just over half of all JavaScript package downloads contain known vulnerabilities, with 1 in 3 rated as high vulnerabilities. One in 10 of those flaws were rated as critical.

    The point the report hammered home is that not all components are created equally. There exists a certain class of well-run open source projects that release updates more frequently, update problematic vulnerabilities faster and remediate flaws more quickly. It is up to DevSecOps teams to systematically manage components, so they leverage more of these high-quality open source projects and try to avoid the riskier open source dependencies.

    “Management of software supply chains is not simply ensuring quality at velocity. Our supply chains are being attacked by adversaries in new and creative ways,” according to the report. It found that organizations that actively manage their supply chains are able to reduce the proportion of vulnerable components by 55% compared to those who do not.

    The report found several software supply chain practices contributed to this reduction of risk, with the following five standing out as the most impactful:

    Develop a governance program: Foundational to this is having some sort of open source governance or standards policies in place, according to the report, which noted that among the best DevSecOps organizations at managing risky components, nearly 3 in 4 have such policies in place.

    Have a process for evaluating and adding new components: As a part of that open source policy, the best teams have standardized processes for how developers add new open source components. This includes how these components are evaluated, approved, standardized and so on. The exemplary organizations are 11 times more likely to use this kind of process for new dependencies.

    Proactively remove problematic components: New vulnerabilities are discovered every day, so simply evaluating a dependency upon its first inclusion in an application can only take an organization so far. The best organizations also take the initiative to look for potential sources of open source vulnerabilities in existing code. That includes proactively seeking out and removing known vulnerable dependencies, as well as reducing the attack surface of the code by eliminating open source dependencies that aren’t being used. The best organizations are 9.3 times more likely to do so.

    Aim to always use the latest versions of components: It should go without saying that the latest version of any given component has the least vulnerabilities. The report showed that component releases that are 3 years or older have a 65% higher defect rate compared to newer components. Exemplary organizations strive to use the newest updates of all of their open source components and are 6.2 times more likely to do so than other organizations. That doesn’t happen accidentally—these teams are 10 times more likely to schedule update dependencies as part of the daily work of DevSecOps teams.

    Use Enforcement Automation: Just having a policy in place is not enough—organizations need to enforce the open source standards to reduce risk meaningfully. The study showed that DevOps practices combined with automated enforcement of policies increase developer adherence to standards by 150%. The exemplary teams with the lowest open source defect rates are 12 times more likely to have automated tools to track, manage and/or ensure policy compliance of open source dependencies.

    All of these best practices add up to less risk from components, as well as less pain for DevSecOps teams, who are three times less likely to consider the process of updating components as painful.

    — Ericka Chickowski

  • 4 Traits of High-Performance Digital Leaders

    4 Traits of High-Performance Digital Leaders

    Businesses that have retooled their culture, their hierarchy and their business mindset toward the use of technology are consistently outperforming their more traditional peers. According to the newly released Harvey Nash/KPMG CIO Survey 2019, digital leaders are the top 30% of businesses that are extremely effective at leveraging digital capabilities to advance their business strategy. The survey showed that digital leaders are beating the competition more handily than their cohorts on some of the most important business metrics followed by CEOs, including time to market, customer experience, response to change and profitability.

    As the report explained, organizations don’t spontaneously become digital leaders and they don’t create their success by “pedaling harder in their traditional IT function; in fact, for many organizations the concept of a traditional IT department is an anathema to them.”

    The study showed that organizations that have established themselves as digital leaders tend to have incredible support and commitment from the top of the food chain. Whereas only about 58% of CEOs at most other firms want their technology projects to make money versus save money, 78% of CEOs at digital leaders see technology capabilities as their proverbial goose laying golden eggs. In the same vein, digital leaders are also much more likely to have a top technology leader—be it CIO or chief digital officer—reporting directly to the CEO and sitting on the executive team.

    “There is a chicken and egg question–is the technology leader influential in these enterprises because the board supports them, or does the board support them because they are influential in their own right?,” ruminated report authors on the board access and influence that CIOs have at digital leader organizations. “The truth is, it’s probably a mixture of both.”

    Regardless of how that relationship evolved, the tone from the top is helping digital leaders to hone themselves through some common winning practices and modes of operating. The report found that these organizations tend to exhibit four common traits, many of which have significant cross-over with the practices and principles of technology organizations that have embraced DevOps transform their IT delivery models:

    An Emphasis on Modern IT Delivery Speeds

    In fact, one of the most stark differences between digital leaders and everyone else is that they are far more likely to have leveraged DevOps to accelerate innovation. The study showed that these organizations are more than three times as likely to use agile or DevOps methodologies enterprisewide to speed up feature delivery. What’s more, digital leaders are also fully buying into the old DevOps ethos of innovating fast and failing faster. Digital leaders are similarly almost three times as likely to pivot quickly from small experiments, scaling up quickly on success or stopping quickly on failure of these small pilots.

    Strong Business-to-IT Collaboration

    “Digital leaders are swapping control for influence and investing time in business relationships,” according to the report.

    The report showed that whereas only about 18% of most traditional technology functions work collaboratively with business leaders to deliver technology change, that spirit of cooperation is alive and flourishing at 54% of digital leaders. In fact, technology executives at digital leaders are less likely to look down on business-driven spend as rogue or shadow IT, but instead as business-managed IT that’s sanctioned and even encouraged. At the same time, technology decision-makers are more likely to influence and govern these business-managed IT decisions than at other organizations.

    Focus on Value Rather than Technology

    The report showed that digital leaders have an “expansive mindset,” meaning that they’re much more likely to think of technology in a big-picture way. So they look at technology delivered as long-term products rather than short-term projects. And success is not viewed through traditional technical SLAs, but instead “… through the lens of business performance, often tied to measures like customer or employee experience, lifetime value, loyalty, uptake and time to market.”

    Additionally, this expansive mindset combined with high levels of collaboration with the business has most digital leaders putting together cross-functional digital teams that combine IT and business staff. These organizations are three times as likely to invest in training non-IT people in IT skills.

    Fanatical About Data

    Finally, all of that experimentation and work is driven by data. The report showed that “data is at the heart” of digital leader organizations. Digital leaders are more than three times as likely to maximize the value of all of the data they hold, and almost four times as likely to maintain an enterprisewide data management strategy than other organizations.

    This use of data is helping them provide business leaders with actionable customer insights, to improve operational efficiency and to get a better line on risks and opportunities the business will face in the future.

    “Digital solutions are the oxygen that allow a business to breathe and run at market speed,” said Steve Bates, global lead of the CIO Advisory Center of Excellence at KPMG. “This year’s survey provides compelling evidence of the transformational progress digital leaders are making. It is up to the rest to catch up and join them.”

    — Ericka Chickowski

  • Are Self-Service Machine Learning Models the Future of AI Integration?

    Are Self-Service Machine Learning Models the Future of AI Integration?

    DevOps teams seeking to step up their mojo in developing cutting-edge artificial intelligence (AI) features are facing a big skills bottleneck when it comes to data analytics and machine learning modeling. As a result, the market is seeing an influx of self-service machine learning models and machine learning-as-a-service offerings designed to help development teams more easily integrate AI capabilities into their software.

    This is coming in direct response to an explosion in demand for AI capabilities in the enterprise. According to Gartner analysts, AI adoption in the enterprise tripled in the past year. A report last fall from MIT Sloan Management Review and Boston Consulting Group found that 91% of enterprises believe that AI will deliver new business growth to them by 2023.

    The trouble is that folding AI functions and predictive analytics into software requires a whole new level of expertise in data science and machine learning from cross-functional DevOps teams. They need added skills to choose the right algorithmic approaches, acquire and manage data, train the models and integrate them into the code base and underlying infrastructure so that everything works properly under the hood.

    “We have heard from customers everywhere that they want to adopt machine learning but struggle to actually get models into production,” said Eric Boyd, vice president of cognitive and AI for Microsoft.

    Ali Farhadi Xnor
    Ali Farhadi, Xnor.ai CEO

    DevOps leaders today already face an uphill battle to keep their developer ranks staffed with well-trained software engineers. Piling on additional requirements for very specialized machine learning and data science expertise further strains those recruitment efforts. According to the most recent Harvey Nash/KPMG CIO Survey, the top one and two technical realms that suffer the biggest skills shortages today are in data science and AI, with 46% and 38% of CIOs respectively reporting recruiting pain in those areas.

    “There’s a huge imbalance between the demand in the market and the supply of the very best AI experience to actually train these models,” said Ali Farhadi, CEO of Xnor.ai, a Seattle firm that just last week released a self-service platform for developers. Called AI2Go, the platform provides pre-trained deep learning models to quickly integrate AI features such as facial recognition and object classification directly into their software.

    “We want to enable anyone who can code to benefit from AI,” Farhadi said.

    The Xnor release comes just a few weeks on the heels of a big launch from Microsoft of several self-service machine learning model tools aimed firmly at MLOps teams—those DevOps teams implementing machine learning capabilities. The new services from Microsoft roll out capabilities to manage the code, data and environments used by MLOps teams throughout the machine learning deployment life cycle.

    “Azure Machine Learning service’s MLOps capabilities provide customers with asset management and orchestration services, enabling effective ML lifecycle management,” explained Jordan Edwards, senior program manager for Azure MLOps, in a blog post on the topic. “With this announcement, Azure is reaffirming its commitment to help customers safely bring their machine learning models to production and solve their business’s key problems faster and more accurately than ever before.”

    Tools such as these are not always going to obviate the need for data scientists and machine learning/AI specialists on DevOps teams. As Farhadi noted, even a simple service like AI2Go requires development teams to choose the right models for the right situations. Nevertheless, this burgeoning market could take the pressure off teams by minimizing the scale of specialized recruiting, while reducing bottlenecks and dreaded re-work.

    “Machine learning projects usually involve several people—data engineers, data scientists, DevOps and others. Conventionally, at each stage of the machine learning life cycle, most, if not all, of these people end up doing an excess of custom coding, redundant work and manual trial and error,” said Craig Stewart, senior vice president of product management and product marketing for SnapLogic, which last fall introduced a self-service machine learning add-on for its integration platform. He noted that one of the primary goals is speed, to help cross-functional teams “build, train, validate and deploy high-performing models faster.”

    — Ericka Chickowski

  • Open Innovation Labs: How Lockheed Brought Better Security to Fighter Jets

    Open Innovation Labs: How Lockheed Brought Better Security to Fighter Jets

    How Red Hat’s Open Innovation Labs have helped accelerate Lockheed’s software development for the F-22 Raptor

    As the key government contractor charged with helping the U.S. Air Force keep its F-22 Raptor jets in fighting trim, Lockheed Martin Aeronautics increasingly values software delivery speed as much as engine speed. So much of the continued evolution of these fighter jets depends upon continual upgrades of the software code powering all of the physical systems built into them, from tip to tail.

    Until a year ago, these jets, which first rolled out of the manufacturing plant in the 1990s, were supported by the waterfall development methods of that same era. Lockheed executives and Air Force brass recognized that the F-22 needed a DevSecOps transformation, and fast.

    “When you have a world-renowned platform like the F-22 Raptor, adversaries are constantly looking for ways to counter it,” said Michael Cawood, vice president of F-16/F-22 product development at Lockheed Martin. “This means we must constantly add capabilities and improve the F-22. And we must do it faster than ever before. We needed to transform our own organization and how we did things.”

    To accelerate that process, Lockheed last year sent a small contingent of engineers through a residency program at the Red Hat Open Innovation Labs to fast-track the process of implementing an open source architecture onboard the F-22 while adopting agile development methodologies and DevSecOps practices. Since bringing back their learning to the organization, these engineers helped to catalyze a faster development process that has helped Lockheed introduce software features for the F-22 years ahead of schedule.

    How Open Innovation Labs Works

    Now in their third year of operations, the Open Innovation Labs were developed by Red Hat as immersive training grounds to help its enterprise customers figure out how to deliver better software, faster. A standard engagement has companies such as Lockheed pay for residency for anywhere between three to six developers and operations staff to get paired one to one with Red Hat employees—engineers, consultants and others—with deep knowledge in DevOps and agile development. They work with them in a time-boxed four- to 12-week engagement to build out application prototypes using DevSecOps practices along the way. The residents are also teamed with a designer and an engagement lead who doubles as an agile coach, typically working with the customer’s product owner for the prototype.

    Obviously, Red Hat has a vested interest in training these people in technology such as its OpenShift Container Platform, getting them up to speed on how to orchestrate and manage containers and microservices. But the labs approach is in a different league than the standard vendor training program. It more closely follows the DevOps dojo model that enterprises like Target and Capital One are running internally to scale up cultural and process transformations across their IT organizations. Like those dojos, the working model is to build real software through a guided learning process that helps participants get hands-on experience in how to change the way they deliver software.

    Red Hat runs the labs either out of a handful of dedicated spaces it runs around the world or within pop-up labs of the customer’s choosing. Besides Lockheed, other notable alumni include the likes of Volkswagen, UNICEF and Heritage Bank. According to a recent analysis by Forrester Consulting, participants in the Open Innovation Labs program are accelerating application development by more than 60%.

    Lockheed’s Accelerated DevSecOps Transformation

    In Lockheed’s engagement with the Open Innovation Labs, the organization sent out a cross-functional team of five developers, two operators and a product owner. They were tasked with building out a new application for the F-22 on OpenShift, using that process as a model for future development workflows. As a part of the process, Lockheed also worked with the labs to help it build out an internal dojo that could facilitate the scale-out of the knowledge gained by its small team across the entire F-22 development team. Six months after the initial residency, Lockheed managed to scale its OpenShift deployment and DevSecOps transformation across its entire 100-person team.

    “By working with the Red Hat Open Innovation Labs team, we changed everything—our toolchain, our process and, most importantly, our culture,” explained Cawood.

    As a result, the F-22 development organization has been able improve its ability to forecast for future sprints by 40%. More importantly, it has cut what was once an onerous five- to seven-year process of rolling out new software capabilities for the aircraft. Lockheed is on track to deliver new communications capabilities for the F-22 this summer a full three years ahead of schedule.

    — Ericka Chickowski

  • AIOps: Supporting Reliability at DevOps Speeds

    AIOps: Supporting Reliability at DevOps Speeds

    AIOps looms large as a way to help push the DevOps envelope. How can organizations prepare?

    As organizations journey down the path of DevOps maturation, sustainable IT operations and IT service management remains a challenge for many. Even advanced organizations that have managed to speed up deployment rates and improve software quality struggle to maintain the resilience of the infrastructure that supports those applications. To support DevOps speeds, a growing number of organizations are turning to AIOps—the use of artificial intelligence (AI) and machine learning in IT ops—to speed up analysis of IT problems and better automate incident handling.

    A new study out this week from OpsRamp shows that ops pros are able to reduce mean-time-to-resolution of incidents by as much as 50% through the use of AIOps. The use of AIOps is still infrequent, but analysts say its prevalence will grow quickly. Gartner says about 5% of organizations were using AIOps tools last year, but that by 2023 the deployment rate will shoot up to 23%.

    But as DevOps teams lean on AIOps tools, management will need to harness a new set of skills to get the most out of the types of operational automation AIOps will afford. In the coming years, experts believe AIOps will change the complexion of the IT ops workforce. Fewer people will be needed in the network operations center (NOC) to manually sift through alerts and respond immediately to incidents. More people will be needed to curate data, scrub it and train the algorithms that will do the heavy lifting of event correlation and determining the root cause of problems.

    As organizations and individuals plan out how they’re going to future-proof themselves for the changes in IT ops wrought by AIOps, experts say they should increasingly see the need for the following three roles in IT organizations.

    AIOps Architects

    Expect to see more job postings in the near future looking for AIOps architects and automation path designers, said Will Cappelli, CTO of Moogsoft and the former Gartner analyst who’s known for coining the term AIOps.

    “(AIOps architect is) essentially a software development job involving the construction of meta-algorithms that guide the flow of control of the AIOps systems,” Cappelli said.

    Potentially falling in this umbrella could be roles such as machine learning and AI designers that focus on designing the algorithms themselves, and interaction designers that focus on designing the automated workflows. According to Brendan Caulfield, co-founder of ServerCentral Turing Group, these roles already exist in other business capacities. With AIOps, these design jobs will start to be applied specifically to IT operations as well.

    To prepare, veteran IT ops experts should start thinking about how they can bolster their skills, and management need to recruit and train accordingly.

    “Mathematics and an end-to-end understanding of how modern IT systems behave will be critical,” Cappelli said.

    ITOps Data Scientists

    AI depends upon good, clean data and sound modeling to work effectively.

    “The most sophisticated and expensive AI tools are useless if a company isn’t able to feed them with a high volume of high-quality data, in a format the system can digest,” said Rajesh Kalidindi, CEO of Leva Data.

    This was corroborated by the OpsRamp report released this week that showed maintaining data accuracy is the No. 1 challenge cited by organizations deploying AIOps today. Many organizations will need ITOps specific data scientists to attend to this problem.

    “Building high-volume, high-quality data sets will be increasingly important, and it requires a lot of human effort,” Kalidindi said. “AI has a long way to go before catching up with human reasoning (let alone surpassing it), and it can’t make any progress without people developing data models for AI to work with.”

    Augmented Intelligence Analysts

    According to Caulfield, algorithm training and support by domain experts in operations will become increasingly important at most organizations.

    “No matter how ‘smart’ an algorithm is, or how well its designed, there will be exceptions that are unique to a business that will need oversight and tuning,” he said.

    Kalidindi concurred that human intervention will be needed for a long time to augment and validate the insights and actions driven by AIOps, noting that this will lead to the rise of a new role for ops analysts.

    “Especially in the early days of machine learning, humans are needed to judge whether an algorithm is leading to the right judgment,” he said. “For example, if we were to apply AI to analyzing the problems the IT Help Desk encounters, a human would be needed to judge whether the algorithm made an appropriate recommendation. If not, IT and data science needs to work with the human’s recommendation to better train the AI model on how to handle that problem in the future.”

    One thing that most experts agree about is that the rise of AIOps will not diminish the need for IT pros. In fact, they say that domain expertise will be more important than ever. However, it will need to be bolstered by additional knowledge of data science, how AI works, and systems architecture.

    “They need to retool themselves from admin-oriented activities to scripting/reporting-oriented tasks, and they need to be more application aware than ever before as infrastructure becomes a commodity,” said Ravi Raghavenderrao, vice president of service delivery at NetEnrich, of IT ops professionals in an AI-enabled future.

    To those who have adjusted to DevOps transformations, this should all sound very familiar.

    — Ericka Chickowski

  • The State of Web and Mobile DevSecOps

    The State of Web and Mobile DevSecOps

    As the leaders in coverage of the DevOps world, the reporters here at devopsy.kinsta.cloud are constantly pressed upon to convey the true state of DevSecOps at any given moment in time. We try to dig up the state of the union by asking important questions of IT executives, DevOps champions, security leaders and other pundits. The answers we get offer illuminating narratives of the DevSecOps journeys in global organizations across a wide range of maturity levels. But, at the end of the day, these are anecdotal at best.

    We know that nothing beats hard data, which is why we’re partnering with NowSecure to field a new survey that we hope will offer solid numbers around secure DevOps maturity and adoption practices.

    With this survey, we hope to do a couple of things: First, we obviously want to provide insight into how well organizations are automating their software delivery and application security (appsec) testing practices across their entire software portfolio. We’d like to understand the perceived benefits organizations have for folding security into DevOps and the inhibitors.

    Second, though, we want to take a deeper dive into where DevSecOps is specifically when it comes to delivering mobile applications. There’s not a lot of visibility into whether there’s a different set of standards, practices or benchmarks for mobile software, but we have a hunch there might well be a gap. We can’t be sure until we’ve got some data to show where the industry stands when it comes to mobile DevSecOps.

    Which is where our esteemed readers come into the picture.

    We’re asking executives, managers and practitioners to take our new survey and let us know how things stand in their organizations. We’re looking for respondents from a range of backgrounds—be they DevOps engineers, developers, architects, security pros, operations gurus or QA professionals. Help us out so we can provide important data back to the industry.

    — Ericka Chickowski

  • DevOps Kids Holiday Gift Guide: Giving the Gift of Code

    DevOps Kids Holiday Gift Guide: Giving the Gift of Code

    Any DevOps professional worth their salt inherently understands how inextricably tied the future of work is with coding, scripts and automation. It’s already a development-oriented world out there and is increasingly growing so. As a result, today’s kids need to at least understand the fundamentals of programming if they want to succeed later on in life—even if they have no interest in becoming full-fledged developers themselves. This knowledge is what educational professionals call “computational thinking.”

    Learning this mode of thinking doesn’t have to be a drudge, though! Fortunately, there are tons of awesome STEM toys and games out there to get kids into the coding mindset from a very early age—all while having fun in the process. We’ve gone through the trouble of sifting through the offerings to give the DevOps community a good head start on their holiday gift giving, no matter the age of kids they’re hoping to bring into the fold.

    Preschool

    Code-a-Pillar

    Price: $47

    What’s not to love about a programmable caterpillar? Very young children pick up planning and sequencing skills by experimenting with the way the Code-a-Pillar moves. Each segment of the toy controls its movements such that different segment sequences create a route for it to move.

    Cubetto

    Price: $224

    Inspired by the Montessori emphasis on the accessible simplicity of wooden toys, Cubetto gives little ones the chance to physically manipulate a programming language via a wooden robot. Putting wooden blocks on the Cubetto tells it where to go.

    Robot Turtles

    Price: $20

    Built from a massively popular Kickstarter campaign, Robot Turtles distills the basics of coding into a fun board game tailored for preschoolers on up through grade school kids. Kids learn sequencing and even subroutines by moving the robot turtles around on the board. It’s what the creators call ‘stealth learning.’

    Grade School

    Let’s Go Code

    Price: $27

    Believe it or not, the creators of Let’s Go Code have managed to figure out a way to teach programming principles in a kinetic activity that doesn’t even require a computer. This is a game that has kids use coding precepts to build mazes that they’ve got to hop, skip and jump through—perfect for bubbly young grade schoolers.

     

    Botley

    Price: $52

    Much like the Code-a-Pillar does for younger kids, Botley teaches screen-free coding by having kids create more advanced routing programming for a little robot. This adds more complicated challenges such as creating obstacle courses, looping commands and more to require a higher level of critical thinking for the grade school set.

     

    Kano Computer Kit

    Price: $100

    Get kids well and truly pumped to code with the Kano Computer Kit, a complete box kit that comes with everything they need—including detailed instructions—to build their very own computer and then code on it. The software platform comes built with over 100 coding challenges and teaching tools for kids to create their own games, websites, art and music.

    Middle and High School

    Circuit Playground

    Price: $34

    This handy starter kit comes built in with multicolored LEDs, environmental sensors and more to get kids into experimental embedded programming. The platform runs MakeCode, Circuit Python, Adruino and Code.org CS Discoveries, giving tons of runway to create unique projects. The Playground’s creator has a very extensive online knowledge base and maker community to give teens some ideas to get started.

     

    Space Chem

    Price: $2

    Pick up this design-based puzzle game on Steam for cheap to get older kids into programming in a fun and engrossing format. The game has players acting as a “reactor engineer” in a sci-fi space colony building out chemical synthesizers to transform raw materials into chemical products. As they solve puzzles they start to learn about basics like sequencing, loops, branching and subroutines.

     

    MakeBlock Ultimate 2.0

    Price: $350

    One of the most versatile robot kits for teens on the market, this one comes with more than 160 parts that can be combined into more than 10+ forms that are then programmable via a number of methods. Beginners can use drag-and-drop function blocks, and as they learn more kids can program it using Python, Arduin IDE and NodeJS.

    — Ericka Chickowski

  • When DevOps and ITSM Collide: Mistakes to Avoid

    When DevOps and ITSM Collide: Mistakes to Avoid

    In many ways the DevOps movement evolved as a reaction against the worst elements of IT service management (ITSM). IT leaders wanted to get from under the yoke of overly complex processes, bureaucracy and red tape to speed up software delivery, and to just get stuff done. Now that DevOps has kicked into full gear at so many organizations, the question remains: What do we do about ITSM?

    While the instinct in some may be to completely ditch ITSM or to just let the ITSM team operate in a separate orbit from DevOps practitioners, experts say that would be a big mistake.

    “DevOps aims to deliver high-quality software to the business faster and in shorter cycles,” explained Krishna Mohan Reddy, vice president and deputy global head of Cognitive Business Operations at Tata Consultancy Services. “However, it still does not eliminate the need for controls.”

    In many ways ITSM is the yin to DevOps’ yang—the two operating philosophies do offer contrasting viewpoints but when the operate in a single system they can be extremely complementary and synergistic.

    “Maintaining the proper balance between the ITSM and DevOps helps the organizations to scale and grow and become more organized,” said Sachin Agarwal, DevOps engineer and solution architect for Mantra Labs. Operating at DevOps speeds with ITSM frameworks in place keeps finger-pointing at a minimum and reduces the risks of failed deployments. “Both in harmony will increase the efficiency and reduce the blame-game within the organization.”

    As organizations try to figure out how to achieve that harmony, here are some pitfalls they should seek to avoid:

    Letting Each Tribe’s Preconceived Notions Hold Back Overall IT Progress

    It can be easy to get wrapped up into dogmatic fervor that your particular philosophy is the One True Path for IT success, but that really does nobody any favor. Organizations that want to pick up the best of both ITSM and DevOps need to encourage stakeholders to approach the table with an open mind.

    “You need to compromise. A lot of the new, young practitioners in DevOps and Agile will pretty much automatically discount opinions coming from people who have got ITIL certifications,” said Andi Mann, chief technology advocate for Splunk. “And vice versa—a lot of traditional operations and development teams will look at the people in DevOps and Agile and think they’re all cowboys. But they all have something to learn from each other.”

    For DevOps practitioners, it’s important to keep in mind that ITSM controls such as service requests, change management or structured management of incidents and problems don’t have to stop a team from delivering quality software at speed, said Troy Vetter, COO of Coda Global.

    “In fact, speed and quality are actually helped by being closely monitored. In IT, just like in science, if it isn’t seen, measured, and “reproducible, it doesn’t really count,” he noted. “So, ITSM is actually the perfect complement to DevOps because it aids in the visibility, measurement and reproducibility of key DevOps processes.”

    Failing to Address the Elephant in the Room: Change Management

    Analyst Roy Atkinson agrees that ITSM has a lot to bring to the table for DevOps workflows. ITSM’s rich history in thinking about and tracking operational best practices offers tons of value in improving the customer experience with the software being delivered. But on the flip side, ITSM teams can’t continue to operate as they did in the past. They need to compromise and learn new ways of accommodating DevOps approaches—particularly in the area of change management.

    “Change management has often been where DevOps and ITSM collide,” said Atkinson, who analyzes the service management market for HDI. “The stereotypical Change Advisory Board must radically revise its approach to be in sync with the enormous amount of change that DevOps can produce in a day.”

    The good news is that thought leaders in the ITSM world have been thinking hard about this issue and have been evolving practices so they can embed change management controls into a DevOps framework without holding back continuous delivery.

    “There is some excellent recent thinking in the ITSM world about how change management can serve its intended purpose without being a roadblock by acting as an enabler of change and being involved all along the way, rather than a bolt-on quality check at the end of the line before the production environment,” Atkinson said.

    Maintaining Separate Toolchains

    Often times one of the big reasons ITSM will remain a roadblock for DevOps isn’t because the ITSM folks aren’t willing to collaborate, but simply because each team relies on completely separate toolchains.

    “You might have a tool that developers use for tracking bugs or product backlog and then there’s a completely separate IT service management tool,” said Donna Knapp, curriculum development manager for ITSM Academy and author of the DevOps Institute’s course on ITSM for DevOps. “So what you end up doing is forcing developers to use some process that’s outside of their norm in order to address issues.”

    As Vetter explained, a lot of the knee-jerk DevOps reaction against ITSM has to do with an aversion to the historical view of ITSM requiring manual inputs in monolithic ticketing systems. But the technology of ITSM controls has advanced—it just needs to be appropriately implemented.

    “Self-service requests, automation flows and IT orchestration are now our front door to actions, and IT service management can and should be happening in the background, automagically created, maintained and measured by our DevOps tools,” he said.

    — Ericka Chickowski