devopsy.kinsta.cloud

Tag: workflow optimization

  • Workflow Automation Poised to Accelerate DevOps

    Workflow Automation Poised to Accelerate DevOps

    Automation is taking the market by storm. Research firm Gartner predicts that by 2020, more than 40% of data science tasks will be automated, and that is just the tip of the iceberg. Businesses are turning to automation to reduce costs, increase productivity and gain a competitive advantage. Add DevOps to the mix and it becomes readily apparent how DevOps teams can benefit by leveraging a little workflow automation. After all, the DevOps process is framed by repeatable workflows, wherein the agile manifesto dictates progress or other frameworks define how change moves through the development and deployment cycles.

    “Most development projects are hindered by numerous manual steps and the lack of defined collaborative communications methodologies, leaving far too much room for error,” said Raj Mehta, CEO of Raj Technologies. “Defining a workflow and bringing automation into the picture could go a long way toward reducing errors.”

    The DevOps community has come to rely on many manual processes and numerous tools to keep the pace of rapid application delivery and change management. Those solutions often have points of failure, where important tasks can slip through the cracks. Even so, many developers are resistant to change and embrace the processes they have already established. Yet, intertwining manual processes with automation and management tools has been demonstrated as rather inefficient, slowing the perception that workflow automation can actually achieve the productivity promises claimed.

    Perhaps the challenge comes from the fact that many DevOps teams require assistance to achieve any type of automation goals. That indicates there may be a lack of qualified professionals to bring forth workflow automation. “You have to crawl before you can run, and that means you must become an expert in designing workflows, before deploying one, or at the very least, turn to an expert,” Mehta noted.

    A workflow is a fundamental process that dictates the steps to achieve a goal. Workflows have been used for centuries by businesses and have become digitized over the last few decades, with numerous workflow tools coming from leading vendors.

    Many technologists may have first encountered the workflow ideology back in the days when Microsoft’s Sharepoint first came on the scene, which offered rudimentary tools for routing content through various steps. Of course, the workflows in use today have become much more complex, requiring designers to not only know how to script actions but also understand the overall goal of those actions.

    That’s a simple truth understood well by Workfront, a Lehi, Utah, based vendor of workflow solutions which is looking to better service various organizations, as well as DevOps professionals, to create workflow automation opportunities. The company’s primary product goes by the name of Workfront as well and uses a platform-based approach for creating workflows that can be integrated across multiple technologies, bringing centralized workflow orchestration to many use cases.

    Tools like Workfront fit well into the automation equation by creating the foundation of workflows. However, most DevOps professionals need more than workflow design to achieve improved automation. Other elements must be considered, such as demand management, resource management, project templates, policies and scripted actions. Efficient workflow tools should also incorporate task lists, collaboration capabilities and reporting.

    Of course, Workfront is only one of many workflow automation companies on the market. Many other vendors are established in the market, with Market News Reports mentioning leaders such as Pegasytems, Tibco Software and Sourcecode Technology Holdings dominating the market.

    HNY Research projects workflow automation market size will grow to $16.58 billion by 2023 from $4.68 billion in 2017, at an estimated CAGR of 23.48%. Key factors such as ease in process with the installation of workflow automation tools, convergence of workflow automation with traditional business processes, focus on streamlining business processes and cost efficiency through workflow automation are driving the workflow automation market.

    “Business leaders must carefully analyze different processes and identify which workflows can automated. Therefore, business leaders should avoid rushing into workflow automation adoption and analyze the various potential use cases and choosing which vendors best fit in their organization,” said Naveen Joshi, founder and CEO of Allerin.

    Many businesses are leveraging workflow solutions on their path to digital transformation and bringing once-manual processes into a world where automation can be applied. While some may argue over the intricacies of digital workflows and how those workflows fit into business process management (BPM), one truth remains evident: The goal is to digitally transform and introduce automation, which in turn indicates robotic process automation (RPA), becomes the next logical step.

    According to Grand View Research, the global RPA market is expected to register a CAGR of 31.1%, from $597.5 million in 2018, driven largely by demand for business process automation via artificial intelligence (AI) and software robots.

    RPA paired with workflow design may prove to bring full digital transformation to DevOps, and potentially grow beyond just a niche in those challenging environments. Workflows, BPM, BPA and RPA are becoming increasingly important to DevOps organizations seeking to lower error rates, boost productivity and improve communications.

    — Frank Ohlhorst

  • In AI Initiatives and Hybrid Clouds We Trust

    In AI Initiatives and Hybrid Clouds We Trust

    The 2019 State of the Union Address, delivered Feb. 5, was light on anything tech-related. But just a week later, Feb. 11, technology was the focus of an executive order that launched The American Artificial Intelligence (AI) Initiative. Why do we need a national AI initiative, and what does it have to do with DevOps? (Hint: You may need hardware and a hybrid cloud strategy for your AI initiative.)

    Investments in AI technology are exploding worldwide. Most analysts believe that while the total number of startups and equity deals to date still favors the United States, China is spawning startups rapidly, thanks to their recent AI initiative. According to just-released data from CB Insights:

    “While China accounted for only 10% of global AI deals in 2017, Chinese AI startups took 48% of all AI funding dollars that year, surpassing the US in AI funding for the first time.”

    The long tail doesn’t lie. A big global competition here is a good thing, though, and the scoreboard has spurred movement. The American AI Initiative lays out five principles, paraphrased in brief:

    1. We need to drive technological breakthroughs in AI.
    2. We need technical standards for development and testing.
    3. We need workforce training in skills and applications.
    4. We need to be able to trust AI solutions for privacy.
    5. We need both open markets and protection for “critical” technologies.

    Go, Team America! What, you expected more than motherhood and apple pie out of the White House? I’m not going there. I don’t think any administration could prepare the level of technical insight needed for something of this magnitude. It can, however, lay a course and task agencies for technology plans and funding proposals—and about two-thirds of this executive order is devoted to timelines for those.

    Let’s assume this AI initiative comes together with real dollars, and everyone including your competition is at work on AI. You may be thinking, “This is easy. I’m looking at a prominent cloud vendor’s machine learning webpage right now and it says, ‘No machine learning skills required.’ Its solution is based on its libraries of algorithms plus cloud-based workload acceleration hardware.”

    If that approach solves your specific problem, that’s great. Many use cases will fit those offerings. However, I’m thinking that a lot of you out there won’t turn over your data and your privacy control to a single vendor. Based on what I’ve seen, while the public cloud may be fast enough for some use cases, it’s woefully non-deterministic for others. You may be able to afford all kinds of time and elastic cloud resources for training, where time may be measured in weeks or months on bigger machine learning data sets.

    Real-time inference is a different game. There. Will. Be. Slowness. One missed result can be very costly—ask any autonomous vehicle developer, then look at all the hardware sitting in the trunk. Public cloud solutions are woefully non-deterministic, especially as everyone and their dog starts using them. The CEO of another big cloud vendor just said in an event keynote that most AI solutions will split 40/60 private/public cloud. She went on to say if you’re regulated, it’ll be more like 60/40.

    Most current startup investment is headed for AI chips. Machine learning is the ultimate workload optimization problem. By adding compute nodes with clusters of GPUs, FPGAs or dedicated system-on-chip (SoC) designs, inference speed-up is achievable. Whatever approach you chose, you’re probably going to want workload optimization safely in your private cloud instance within a hybrid cloud AI solution.

    You’re also going to want control over differentiation of your solution. Open source AI software solutions are great because they handle a fundamental base of value. However, AI-enabled applications are very likely to run into problems at scale. Open source will get you to prototyping quickly. Your innovation will be tested as real-world, real-time issues develop. If you can get a truly differentiated hybrid cloud AI solution that proves to be fast, reliable and secure, you’ll have a huge competitive advantage.

    Again, this looks like a perfect job for a DevOps team. Here’s where to start on your AI initiative, in increasing order of difficulty:

    • Have someone get familiar with machine learning and AI public cloud solution offerings—those vendors are easily found.
    • Also get familiar with open source frameworks such as TensorFlow, Caffe, Cognitive Toolkit, PyTorch, MXnet, DeepLearning4J and others.
    • Look at your preferred distributed computing framework for recent innovations in AI workload optimization; for instance, a lot of work is happening around Hadoop and Spark.
    • Inventory the problems you think AI might solve in your organization and see how they line up with those potential solutions.
    • Experiment and discover where the bottleneck in your preferred algorithms might be; those will be candidates for a hybrid cloud workflow optimized solution.
    • If there’s a profound bottleneck, you’ll need to evaluate the best hardware solution, meaning someone on your team (or a consultant) should understand GPU code or FPGA design.
    • For the most severe AI challenges, more companies will be considering either off-the-shelf AI chips or custom SoC designs—even companies that have never designed chips.

    My point here wasn’t to provide a primer on AI technology; that’s a huge topic. The American AI Initiative is a call that teams need to move on AI now or risk being left behind in the competitive prop wash. You also shouldn’t wait for someone to generate the exact solution you need out in the marketplace. With the feds driving a sea change in AI, investments and technology will be shifting even more rapidly, and it’s best to get in front of it.

    For the curious, here’s the entire executive order on the American AI Initiative.

    If you’ve started already, how’s your AI initiative going? Are there other areas in AI or machine learning you’re curious about that would be a good topic for future posts? Reach out if you’d like to discuss this.

    — Don Dingee