Intel this week said a gold image of its oneAPI for unifying infrastructure as code will be available starting next month.
Jeff McVeigh, vice president of data center XPU products and solutions Intel, said the goal is to provide a standard application programming interface (API) for accessing the compute capabilities of any processor regardless of what company may have manufactured it.
The availability of oneAPI coincides with the launch of a series of dedicated graphical processor units (GPUs) from Intel that are intended to supplant the dominance of NVIDIA.
McVeigh said Intel is encouraging cloud service providers and builders of systems deployed on-premises to embrace oneAPI as a way to standardize the way any CPU, GPU, field-programmable gate arrays (FPGAs) or any other class of processor is incorporated within an environment rather than continuing to relying on programming models such as CUDA from NVIDIA that are designed for only one class of processor.
The average developer may never interact directly with a low-level oneAPI, but McVeigh said a standard set of APIs at the processor level should ultimately serve to reduce the total cost of IT as developers invoke multiple classes of processors with their applications.
Intel, of course, also sees oneAPI as an opportunity to usurp NVIDIA as the dominant providers of GPUs that are now widely employed in the cloud to train machine learning models. While Intel CPUs are still employed most to run the inference engine that enables those models to be deployed in a production environment, NVIDIA is now seeking to leverage its success in the cloud to provide either GPUs that can be used to run inference engines or processors from Arm. NVIDIA is in the process of acquiring Arm, a provider of processors that are widely employed in embedded system and edge computing environments.
Microsoft and the open source contributors that oversees the development of the TensorFlow framework for building artificial intelligence (AI) applications have thus far endorsed oneAPI. In addition, the University of Illinois Beckman Institute for Advanced Science and Technology this week announced it is creating a oneAPI center of excellence (CoE) that will extend the oneAPI programming model to nanoscale molecular dynamics (NAMD) environments that employ code to simulate large biomolecular systems.
Previously, the University of Stockholm (SeRC) has committed to adding support for oneAPI to GROMACS, open source software for building simulations of biomolecules, while Heidelberg University is working on adding support for oneAPI to GPUs from other vendors. Intel is also making available software based on oneAPI that is compatible with CUDA, added McVeigh.
Finally, Intel also announced this week that the Intel Implicit SPMD Program Compiler (ISPC) will run on top of oneAPI Level Zero, which provides low-level, direct-to-metal interface. Written in X, ISPC is used to accelerate Intel OSPRay ray tracing engine on Intel CPUs. Support for ISPC on Intel GPUs is forthcoming.
It may a little while before DevOps teams see the benefits of oneAPI; however, as it becomes easier to mix and match processors as they best see fit, chances are it’s because the overhead on the back end has been substantially reduced using a common API.
IT organizations today are faced with the challenge of delivering new capabilities at higher velocity to enable digital transformation. While enterprise resource planning (ERP) systems provide a stable integrated foundation and emerging capabilities to meet business needs, their integration and deployment processes were built for a waterfall methodology.
As business agility is paramount to market success, Intel IT works to move to an agile and DevOps methodology to deliver new capabilities at high velocity while maintaining stability at optimal cost. This program built additional capabilities on top of SAP Solution Manager to enable self-service, continuous integration (CI) and continuous delivery (CD). The velocity of self-service has improved by 341 percent while decreasing deployment costs by 45 percent, with no change induced incidents. The quality of integration and deployments was improved, while the complexity of the integrated environment increased. The maturity level for DevOps adoption has been improved and 97 percent of our agile teams have L3 agile maturity with plans to reach L4 within a year.
The features designed to support DevOps included:
Enabled a complex SAP landscape
Enabled self-service by bundling needed dependent objects
Provides adherence to the orchestration of change and release management steps
Full audit traceability
Auto rollback capability
Architected with full DevOps CI/CD
The Agile and DevOps Program Transformation team did an industry scan and conducted detailed vendor evaluations to meet these needs. Most of those tools available introduce redundant change management functions currently provided by SAP Solution Manager. Due to the lack of feature-rich tools to meet the demands of the DevOps culture, Intel created the Enterprise Deployment Automation Tool (EDAT). EDAT solves the problem by providing self-service continuous delivery capabilities to development teams and enabling high velocity while leveraging SAP Solution Manager.
EDAT is a custom-built ABAP Web Dynpro program that integrates Solution Manager with self-service capabilities with workflow automation. The design was reviewed and approved by our internal audit, security, SOX, change management, enterprise and platform review teams. To meet these requirements, strong authentication and tight controls were needed to perform directed steps while restricting access. EDAT automation performs validations of code, approvals and other checks at increased speeds and improved quality.
One feature that makes EDAT unique is the ability to support step releases. Step releases allow transports across multiple changes and systems to deploy simultaneously while providing the ability to pause and resume later. It enables the ability to resume by showing the list of changes in the release and displays the progress completed, which easily allows for co-shared deployment across multiple regions and resources.
Workflow Automation
The picture above depicts the workflow automation implemented in EDAT. Once a user story is created and work is assigned to an iteration, development begins. Unit testing begins in development. Once tasks are released, automated testing is completed in each environment. Before the change can proceed to the next environment, the testing must result in a pass. If there is a failure, the developer is notified so a fix can be created which results in a repeat test until it passes. An automated backup of objects is completed prior to landing to the next environment, so they could be brought back to the previous state should user testing and validation show issues. An automated workflow is orchestrated within each action and code movement, validating and advancing Solution Manager, the service management ticketing system, agile planning tools and automated testing tools. Automated synchronization keeps the environments in sync.
Validation series include both standard and custom validations prior to allowing three methods of deployment
Three methods of deployment are included; One-By-One, Set-Order, and Import-All. One-by-one best suits releases that require many manual steps between transports. Set-order allows for proper sequencing and was designed to allow imports to sequence in serial, while pausing on any landing error. This is often used in releases benefiting from a faster release with on-error stops. Import-all allows transports to land all at once, which best suits releases where the transports do not share objects.
Both standard and custom validations are enabled within EDAT. A structure was set up that allowed for standard validations, which include the various SOX and audit requirements common across all release types. Another structure was set up that allowed for custom validations, which include architecture-specific setup, rule logic and configuration requirements. This dual structure introduced the ability to deploy changes across SAP pipes from within a single screen and deployment rule logic.
EDAT enforces these validations:
Rule logic based on system and objects.
SOD/SOX compliance checks.
Prevention of backdoor addition of unapproved transports.
Change Manager SAP pipe ownership.
Release date and the approval date to ensure change management process is adhered.
Target release systems availability check.
Lockout, for upgrades and support packs.
Release sequencing is a critical requirement for releases to maintain high quality and ensure a successful release, and is one of the top five reasons why additional resources were needed prior to EDAT. Allowing development teams to order transports and enable the system to import in this order alleviated resource constraints and helped achieve the self-service release capability necessary to increase velocity and improve quality at the same time.
Self-service ability to re-land transports is a key feature that is necessary. Some releases require re-landing a transport to ensure all objects are activated. The only caveat to this feature is to maintain a date check for production landing, to comply with change management.
Landing return codes are shown to the user automatically and full logs are also accessible to the development teams.
Plugin integration enables the ability to allow and support integration across various tools and provide these capabilities:
Agile planning.
Auto rollback.
Pipeline architecture.
Test automation.
Live undo.
These capabilities allow EDAT to integrate with other technologies and other platforms through an open API.
EDAT automatically detects objects and platforms which require specific rule logic. Once detected, a summary is displayed to the user so there is an awareness of what will be completed prior to execution. In an upcoming iteration, predictive detection will validate inactive objects in target environments. This will detect issues in advance and initiate fixes automatically.
A single interface with integration across all platforms allows for increased quality, tool reduction and reduced confusion, and provides clear results. This benefit can be seen across multiple service groups and teams. With standard ERP release functionality, users had to use several tools and instances to complete releases. EDAT enabled synchronized and bundled releases from a single tool, where all platforms would import, whether SAP, ABAP or another platform. The ability to implement releases across all platforms is enabling our vision of integrated release across all platforms.
In an upcoming iteration, full test automation will be enabled, where tests will run and are automated. When these automated tests return with a pass, then EDAT will allow them to proceed to the next environment. When the automated tests return with a fail, then EDAT will not allow the code to proceed to the next environment. The development teams will be notified of the failure so a new set of tests can be run after needed fixes. These test results are updated in the SAP Solution Manager change screen for easy retrieval.
One key highlight, there were no change-related incidents and the cadence of releases significantly increased with continuous deployment compared to time-based sweeps. Self-service migrations from Q1 to Q3 2017 increased 341 percent over the same period in 2016.
EDAT made it possible to decrease migration management resources by nearly 45 percent. Migration and release managers are now able to move to higher-value activities. User feedback has been overwhelmingly positive. SAP has since performed a review of the tool and process and acknowledged the change control process has become very agile. The excitement and value realized are positioning the program to continue with improvements and advancing its DevOps’ capabilities.
Intel IT’s Enterprise landscape is highly complex and diverse. The Enterprise Change Management team has the responsibility of managing the changes in this landscape, which is of the order of hundreds of changes every week. On one hand, we had to make sure we keep the change management processes stringent so that defects/major incidents did not happen in the production environment, leading to productivity loss, shipment delays or cost impacts. On the other, there is a need to assist the capability teams to land changes into production faster than ever to meet their dynamic business demands.
A challenge that many global IT organizations face is, How do we adhere to ITIL processes while teams transform into agile/DevOps way of working? In other words, how do we manage the two seemingly conflicting goals from two different IT organizations, one focusing on the agility of the development and measuring the success of the projects by their throughput time and the other measuring the success by the quality?
In the IT enterprise, the quality is measured by the number of change induced incidents in production—incidents that are caused due to change(s) landing in production. Our team had a challenging goal of keeping it to zero! The year 2017 started with this million-dollar (or is it worth more than that?) question: How do we meet this goal while not impacting the speed of the agile teams? We came up with an agile solution. A DevOps solution for the DevOps problem sounds logical, right?
We formed a scrum team to address this problem with the theme, “Simplify and automate, keeping the focus on quality, velocity and productivity.” The scrum team had change management process owners and the representatives from all the development technologies. The chart below summarizes the steps followed:
Simplify and Automate, Keeping the Focus on Quality, Velocity and Productivity
The first sprint was all about getting the alignment. As much as the number of teams that follow agile, as many definitions of agile exist! The focus was to demystify the myths about agile as well as ITIL change management processes. At a glance, processes look incompatible with Agile, but the goals are the same. Both aim to add business value incrementally in a continuous manner without disrupting the operations. They are not contradictory but complimentary.
Subsequent to this alignment, the second sprint went to the details of the enterprise change management process that’s in place. The process was broken down into the lowest level of details based on the 5W and 2H’s (who, why, what, when, where, how, how much). For example, there are process requirements from legal standpoint that are non-negotiable. There are certain processes required to ensure architectural compliance. We also acknowledged and aligned on some of the challenges and the time taken for process adherence. Multiple discussions happened in this space and it helped all of the teams to come to a common understanding on why these process requirements exist. Process swim-lane categorizing all of these aspects was internalized as part of this sprint.
Change Management Process Swim Lane
The third sprint got things into action mode. Each of those process steps was identified for potential simplification and automation. They were further classified as short-term and long-term based on the automation efforts on continuous integration/continuous delivery (CI/CD) that were underway. The development teams were shown the flexibility and empowered to meet the process need by way of automation rather than completing a manual checklist for every change. This way of policy and rule-based automation helps in a longer way in integrating ITIL Change Management and DevOps.
All along these three sprints, process improvements were rolled out. Several automation efforts were launched. Objective risk assessment enabled us to identify changes that required advance communication to all segments. Multiple gate approvals were reduced into only two—one when the development starts and later before it goes to production. Test cases and results from the CI/CD tools were used effectively to replace documentation.
Process refresher and engagement sessions were held across the organization to share the improvements, gather feedback and incorporate them into further sprints. This helped to gain the confidence of various teams and made the journey further better.
Now the focus is on continuous improvement, which is to implement all those action items identified and also keep them updated as other factors change in the internal and external environments. Engagement between theAgile/DevOps teams and the change management teams are getting refined based on the scrum development models. The success of ITIL Change Management and DevOps integration relies on calling out the agile process followed for the project and having the right engagements at the right points in time. The below picture depicts the engagement in a simplistic manner:
Scrum and Change Management Intersection
The efforts yielded fruits and the organization met the goal. The number of change induced major incidents was ZERO for the Enterprise Segment for the whole of 2017. We need to remember that these process improvements were done without impacting anything underway; it was akin to servicing a flying aircraft in the middle of the sky. The feather in the cap was the positive feedback from the ERP vendor: “The change control process at Intel has become very agile in recent years, with transports to production now occurring nearly every day of the year. … In general, we do not consider such frequent changes to Production to be desirable due to the risks involved; however, Intel appears to be managing the process well as the quality of transports is closely monitored.”
References:
ITIL® Service Management v3
The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations by Gene Kim, Jez Humble, Patrick Debois & John Willis
McVeigh said Intel is encouraging cloud service providers and builders of systems deployed on-premises to embrace oneAPI as a way to standardize the way any CPU, GPU, field-programmable gate arrays (FPGAs) or any other class of processor is incorporated within an environment rather than continuing to relying on programming models such as CUDA from NVIDIA that are designed for only one class of processor.
