devopsy.kinsta.cloud

Tag: agile

  • Continuous Testing: Complementary to Agile and DevOps

    Continuous Testing: Complementary to Agile and DevOps

    Digital transformation brought about by quality software applications and robust hardware platforms helps to create a competitive advantage for enterprises across verticals and geographies. Now it is no longer about choosing between speed or quality of software but to incorporate both when delivering to customers. With Agile and DevOps being embraced by enterprises to stay up to speed on the competitive curve, continuous integration (CI), continuous delivery (CD) and continuous testing (CT) have become the catalysts driving quality digital transformation.

    It goes without saying that continuous testing has become critical and is challenging. CI and CD are primarily tools and team-driven activities, while CT is comprehensive in its scope, including teams, tools, testers and services. The importance of driving and integrating code changes cannot be ignored in the software delivery life cycle. At the same time, unless the automated delivery system can figure out the impact of such code changes on customer experience, CI and CD could become meaningless.

    Rising competition and changing market dynamics have put pressure on enterprises to innovate and change. Agile and DevOps in consonance with continuous testing offer the framework to address the requirement. At the heart of digital transformation lies the need to deliver software quickly, safely and reliably, which can be achieved through continuous testing in DevOps.

    Why Continuous Testing?

    Customers today use software applications across a plethora of devices, operating platforms, browsers and networks. They want the applications to be high-performing, navigable, scalable, secure, intuitive and fast-loading. These attributes can only be fostered if the software development process supports a robust QA mechanism in the Agile or DevOps mold. Unlike in the traditional or manual way of testing, where QA comes after development as a separate discipline, the Agile and DevOps models have QA as part of an integrated team with development.

    In Agile testing, the software is tested for performance, security, usability, scalability and more throughout the various stages of development. Thus, glitches are identified and fixed early in the SDLC, leading to the delivery of quality software. A continuous testing strategy enables enterprises to put in place the processes, systems and automation to accelerate the time to market, install improved feedback loops and deliver desirable business outcomes including high-quality products and services, operational efficiency, responsiveness, competitive differentiation and enhanced customer experience.

    Continuous testing services create a quick and risk-free feedback loop for the QA specialists in the Agile testing pipeline to understand (and act upon) the issues faced by the end customers. These services help to manage business risks borne out of the complexity and pace of application delivery.

    Why Agile and DevOps?

    To thrive in a competitive business environment, enterprises must address issues such as time constraints and resource mobilization to build-test-deliver applications that are higher quality, secure and stable. This engenders a renewed focus on continuous test automation and collaboration.

    • The DevOps approach is ideal for enterprises that struggle to manage demand and implement rapid changes in the SDLC.
    • DevOps allows enterprises to address their acceleration and integration needs by establishing a toolchain that cuts across the realms of development, QA, integration, management and delivery.
    • The silo-driven development and operations teams should be integrated to drive better collaboration. Further, the pipeline should have a built-in mechanism to receive continuous feedback and to generate better quality features and functionalities.

    DevOps: The Extension of Agile

    In the Agile model of software development, developers create user stories comprising smaller chunks of the build to generate feedback loops and to align the product to the market need. It started with small teams comprising developers and QA testers working collaboratively and efficiently. Thereafter came the notion of scale, wherein the software code was developed in short iterations. However, down the pipeline, when it came to deployment and receiving feedback, the silo-driven processes stifled what was expected of Agile.

    Consequently, the process was further refined by automating and streamlining the SDLC and incorporating continuous integration. In CI, code is tested repeatedly and integrated to create a stream of small but frequent releases. However, these challenged the Ops team, which found it difficult to keep pace with development. So, while Agile addressed the issues developers faced with glitches being passed on to the next phase, the DevOps approach turned out to be more effective, as it treated the entire SDLC as a monolith of collaborative processes that generate better outcomes.

    Continuous Delivery and Testing

    To support the strategic business needs, many enterprises have graduated from performing CI to implementing CD. In doing so, they have fine-tuned both processes and technical frameworks. From creating CI builds and executing testing to achieving deployment, the continuous delivery and continuous testing methodologies have covered all dimensions of the organization’s operations. These ensure the quick and frequent deployment of quality software applications to end users. The questions to be asked while implementing a continuous testing framework are:

    • Is the software application deployable during its life cycle?
    • Can the software application be deployed while the team is working on its new features?
    • Can the process receive quick and automated feedback each time a change is implemented?
    • Can a push-button deployment be performed for a software version in any environment?

    How Agile, DevOps and Continuous Testing Benefit Enterprises

    These interconnected methodologies help enterprises in several ways:

    • Accelerated time to market.
    • Better productivity and efficiency.
    • Frequent releases of reliable and stable builds.
    • Capability to build products aligned to the market.
    • Better feedback mechanism.
    • Superior customer experiences.

    Conclusion

    The need for enterprises to be the disrupter in the market by producing or delivering top-quality software products or services has put focus on methodologies such as Agile, DevOps and continuous testing. The complementary relationship among the three methodologies has led to the overhauling of processes and working culture in favor of a new paradigm in which quality deployment is a continuous process.

  • Review, Review, Review

    Review, Review, Review

    Ossification is an issue in IT, and always has been. It is borne largely out of staff and management knowing what to expect from existing software/infrastructure, and the deep product knowledge that allows teams to make existing systems do what is needed. But this becomes a series of rocks that the ship of progress smacks into.

    Contrary to popular belief, DevOps and other new methodologies are not immune to this problem. Quite the contrary, we’ve spoken with people that say “We’re a Jenkins shop,” which sounds exactly like “We’re an Oracle shop” of a decade ago. The wave just ushered in new tools, it didn’t change tendencies. OK, it did more than usher in new tools, but it didn’t change tendencies.

    So, it’s time for a friendly reminder: Review what you are doing, review cloud usage, review tools in the toolchain, review methodologies, review security posture (which has suffered in this environment for far too many shops) and review customer satisfaction.

    Many organizations used the push of Agile and DevOps to update stagnant parts of the infrastructure and development toolsets. Don’t let them become stagnant again. Just as you don’t want change for the sake of change, you don’t want to stay with one tool or methodology just because it is comfortable.

    This isn’t some easy task that I ⁠— or anyone else, no matter how much they try to tell you differently ⁠— can dictate to an organization. There isn’t a handy list, because no two organizations are exactly the same. Not in toolsets, not in culture and not in appetite for risk. The only thing we can offer is guidance to look closely at where you are, what’s available in a given area ⁠— Application Release Orchestration, for example ⁠— and evaluate the available tools just like you were making a fresh choice and did not have a tool in place. Then do the same for processes. Is this the best process for what we’re trying to achieve? Are there tweaks we can make to it? Should we completely change the process to take in changes in environment that have happened since we adopted this process?

    Finally, if you don’t have someone responsible for monitoring cloud costs, appoint someone. Get them the tools, help them understand where the money is going and what cloud-based server usage is. Could these servers be spun down outside of business hours? Could we follow the sun better by staggering cloud data centers in which cloud servers are active, rather than hosting them always in one location or leaving them on 24/7 in multiple locations? Is one team using a cloud feature no one else is? If so, how does it apply, and are the other teams not using it because they don’t need it, or is there some other solution they’ve adopted?

    Basically, question everything. We’re launching a new project that will be a simple dynamic app. Make a few selections from a limited list, get a summary of what you should do and why. It is all really quite simple. Should we host in a web page or make it a standalone mobile app? If mobile app, what tools for development and what targets? Integration of Git — should it be local or on one of the big providers? Do we want any team member to trigger builds on commit or do we want someone managing branch merging before-hand?

    These are all things we already do on other projects, but we’re digging in and seeing what could be better. What could we improve, and what would the rewards for that improvement be? It takes a bit of time and some discussion. We’re confident we’ll burn through it in the next day or so and come up with a plan for development and architecture that best suits the app, but is informed by where we’re at now. It’s been a fun exercise, to be honest. If nothing else, it gave us a strong reminder that any given cloud provider’s “simple” pricing is not that simple, and we need to pay more attention to app characteristics versus fees.

    You’ve rocked it this far, even kept the lights on during the great shift to working from home. Don’t stop improving the environment though, find innovative ways to make it better and keep clipping along.

  • Survey Sees Increased Agile Adoption During COVID-19

    Survey Sees Increased Agile Adoption During COVID-19

    A report published by Digital.ai, the parent company of CollabNet VersionOne, finds 43% of organizations have increased their reliance on agile application development methodologies in the wake of the COVID-19 pandemic, with 15% saying it has increased significantly in the last 90 days.

    The report also notes 33% of respondents say they have increased or expanded Agile adoption in the last 90 days to help manage distributed teams, while 55% said their company plans to increase the use of agile in the next 12 to 14 months.

    Based on a pair of surveys conducted prior to and after the pandemic started, the 14th annual “State of Agile” report also finds 60% of respondents credit agile methodologies with having helped increase speed to market, while 58% say it has improved team productivity. Another 41% say they are also better able to manage distributed teams.

    Mike O’Rourke, chief research and development officer for Digital.ai, said there is a clear correlation between the ability to manage distributed application development teams and successfully navigating the pandemic now that most application development and deployment teams are working from home.

    Many of those teams are also seeing application development deadlines being moved up as organizations prioritize digital business transformation initiatives that now are core to any business continuity strategy.

    That shift is also accelerating a shift toward focusing more on business metrics for measuring success as defined by, for example, the Scaled Application Framework (SAFe), said O’Rourke, noting as a result, there’s also a lot more focus on value stream management across the enterprise as business and IT teams work closely together. The survey finds just over half of the respondents report their organization has either already implemented value stream management (VSM) or are planning to do so.

    The survey also finds more focus on risk and compliance, with the number of respondents identifying and measuring technical risk prior to deployment increasing 54%, while the importance of automated audit compliance and governance across control points increased 10% year over year.

    Perhaps most surprising, the survey finds reducing project cost is cited as an important reason for adopting agile by only 26% of the respondents, a 41% drop from the previous year. The number of organizations focused on release planning decreased by 11%; however, organizations focusing on product road mapping increased 9% over the previous year. The survey also finds slightly more respondents are using automated acceptance tools (3%). Another 12% said they plan to adopt agile project management tools in the future.

    It’s not clear whether increased focus on application development might result in bottlenecks when it’s time to deploy those applications. Organizations that have already embraced best DevOps practices alongside agile development methodologies will naturally be in a better position to deploy applications. At the same time, increased focus on digital business transformation is likely to accelerate the adoption of both agile development methodologies and best DevOps practices across the enterprise.

    In fact, it’s arguable that organizations that are not able to build and deploy applications adroitly may soon find they are unable to compete at a time when software development has become critical to their survival.

  • What Is DevSecOps and How to Enable It on Your SDLC?

    What Is DevSecOps and How to Enable It on Your SDLC?

    For the past three to four years, all the companies around the IT world have adopted agile and different application development methodologies that leverage the work for different departments or areas and helps them to develop new products and release new features to improve their processes and infrastructure.

    In this new Agile and DevOps world where everybody on a team is involved in the rapid-changing and evolution of their application, we are promoting accountability for everybody in terms of security—this is when DevSecOps joins the party.

    What is DevSecOps?

    DevSecOps is a new model that provides accountability for the security implementation in the application; from the planning, design, development, QA/testing, to release and when operating on a production environment.

    When implementing DevSecOps on the Software Development Lifecycle (SDLC), an organization will experience the continuous integration and will notice that the costs for compliance are reduced, code is constantly being analyzed, tested, delivered and released properly.

    DevSecOps enables the process of implementing security to everybody and makes them accountable.

    Why Is It important?

    As I stated previously on this blog, on this rapid-changing era, everything is evolving at a very accelerated pace. We continue to discover vulnerabilities and breaches across platforms and operating systems, patches are released constantly but we—as part of the operating team of a company—cannot afford the risk of having a vulnerability on any side of our IT system/application.

    Main Benefits

    • Reduces vulnerabilities present on your code.
    • Reduces vulnerabilities present on your IaC technologies.
    • Reduces the number of ways to exploit your application
    • Reduces downtime.
    • Improves your application stability, availability and security.

    How Can I Enable DevSecOps on My Current DevOps Pipeline or SDLC?

    DevSecOps is a must-have methodology that needs to be integrated into your DevOps process/pipeline to help you improve your security on your SDLC.

    There are five important phases that need to be followed in order to enable DevSecOps on a current DevOps pipeline or in the SDLC. Here are the crucial phases to enable it:

    Phase 1: Secure Local Development. Start by implementing secure working-environments. When you are developing an application, in most cases you will use open source technologies. Docker is a great helper at this phase since it automates the infrastructure and services deployments on local machines. So when you are using this ready-to-go docker environment, make sure that you are using the most recent/updated versions of the Docker Images and scan them for vulnerabilities. Even the images from official providers have vulnerabilities that need to be patched.

    Phase 2: Version Control and Security Analysis. Enable Vulnerability while uploading your source code.  Having multiple hands or people working at a piece of code can lead to vulnerabilities, especially when they are remote. Git systems have been a great improvement for collaboration between team members and code. When a team member uploads a piece of code, I strongly suggest that you enable automated testing for security on your code dependencies and core. Some good alternatives to do it are Snyk or Sonatype’s Nexus.

    Phase 3: Continuous Integration and Build. When creating the development image/package, you’ll need to make sure that your build tool or system has the proper security in place. It uses https:// protocol, it’s properly hardened and secure, it’s available and protected for attack mitigation or even not accessible via the internet. The tools that you can use here are Jenkins, Circle CI, AWS CodeBuild, Google Cloud Functions, Azure DevOps.

    Phase 4: Promotion and Deployment. When deploying to an environment, insert the environment variables through your CI/CD tool and try to manage them as secrets. Proper encryption and management of these are recommended in order to enhance your security protocols.

    Phase 5: Infrastructure Security. When your app is deployed, make sure that you have an IDS (Intrusion Detection System). Tools such as OSSEC or Wazuh will help on this matter to protect all your hosts.

    Once your code gets to the production, it doesn’t mean that it will be 100% secure. New vulnerabilities are disclosed every day but this cycle will help you and your team to test your code against all the repository of known vulnerabilities, at the time of monitor, configure, reconfigure, adapt and deploy solutions to the environment.

    Tools and Processes

    These are the tools you will need to enable in your DevSecOps process:

    • CI/CD Tool.
    • Secrets Manager.
    • Version Control System.
    • Docker Orchestration tools.
    • Security Analysis tools.

    And these are the processes:

    • Automated regression testing.
    • Dynamic testing.
    • Automated Vulnerability Assessment.
    • Automated Integration.
    • Promotion.
    • Automated Deployment.

    What Is the Ideal Workflow for DevSecOps?

    1. A developer creates a new code and integrates it into the VCS.
    2. Members from the QA team retrieves the code to perform the static code analysis to identify security flaws or functional tasks.
    3. A test environment is created automatically using IaC such as Terraform, CloudFormation, Chef or Puppet and the security configurations are added to the system.
    4. The test automation suite is performed to the application with a tool usually Selenium or any other tool that performs backend, UI, integration, security and API tests.
    5. After the test suite is performed and successful the new changes are sent to the production environment.
    6. The new version of the code is now going to be monitored in the production environment using an APM or Cloud-native monitoring tool.

    Following these points you are ensuring that your application is following TDD practices improving the code quality, compliance, increasing the number of releases of code to production and reducing the time to market which is essential for any organization.

    What Are the Challenges When Enabling DevSecOps?

    • Enabling too Many Tools: Enabling too many tools can become a problem on your SDLC, especially when your team is not used to work and relate with DevOps/security tasks. The main recommendation here is to start slow. Start by enabling only the necessary tools to get your team familiar with the process and add more when you feel your team is prepared for it.
    • Getting Used to the Methodology: It will take some time for all the team to get used to the DevSecOps methodology/culture and also to keep following it in order to be compliant with the normative that your business demands. Try to always stay up-to-date and coaching your team for the newest technologies.
    • Chasing Perfection on the Process: Keep in mind that all the DevSecOps process won’t be perfect, but it will get mature over time. Teams always try to chase perfection and this only leads to more problems with even more integrations or dependencies.

    Conclusion

    In the end, I think every organization must make the effort to shift to a DevSecOps methodology or process and come up with a multidisciplinary team with a focus on security. That’s how an organization will transition from doing DevOps to DevSecOps. Allowing all their collaborators to have accountability on the part they are actively developing.

    — Guillermo Vélez

  • Is Agile Methodology Right for Your Project?

    Is Agile Methodology Right for Your Project?

    I’ve read a few articles of late that have declared the end of Agile development. They are interesting reads for the most part, and they make some interesting arguments, but I see none of them as compelling. I do, however hope that this is the beginning of applying “Right Tool for the Job” to how projects are handled in development/DevOps.

    Agile officially kicked off in 2001, but comes from 90s concepts such as Extreme Programming. That makes it old. But, age isn’t in and of itself a problem. Adaptability beats age every time, and there have been movements that update parts of Agile to keep up with the times/changing environment. My issue is that “We’re an Agile shop” becomes a mantra that implies “Everything is an Agile project” which is absolutely untrue. You can do any development you can consider in Agile. You can also ride your bike from Main to California. Doesn’t make either one the best option.

    As with any methodology, we’ve also had a couple decades for a burgeoning oversight function that interferes with the goals of Agile. The point was to rapidly iterate to get features out the door, low-hanging fruit first. But the weakness of any high-speed development effort is quality, and as fingers have gotten burned from too much change causing problems, oversight has been built in to temper the volume of change. That oversight inevitably becomes rigid and slowing.

    As others have repeatedly pointed out, Agile works best for small teams with shared focus and geographic proximity. That hasn’t changed much. The opposite problem—a project with large, geographically dispersed teams having divergent foci—are a terrible use case for Agile, no matter the flavor of Agile you are implementing. Incremental development is also not the best answer to all problems.

    Indeed, contrary to popular belief, large initial implementations do best if using traditional development methodologies with an Agile approach to feature inclusion. That is because what undermines large projects is growing the size of the project by throwing every requirement and the kitchen sink into Minimum Viable Product, or whatever you call the actual must-have features for your application. Using Agile mindset as a filtering tool, and development methodologies that get a base level of functionality up and running (rather than incrementally implemented) allows more time for testing complete features and limits the number of features dumped into the must-have bucket.

    I’m happy to see this conversation starting to happen more seriously. Agile is a powerful tool that is the silver bullet for some development efforts. But it is only a tool, not a lifestyle. Using the Agile tool does not preclude choosing other methodologies (even waterfall, though it is currently passe’ to suggest its use) when the project is best suited to it. Most large organizations do some amount of this already, but it should be standard practice to choose methodology based upon project and suitability to task. Hopefully this is the first step toward doing so.

    — Don Macvittie

  • SRE Is the Most Innovative Approach to ITSM Since ITIL

    SRE Is the Most Innovative Approach to ITSM Since ITIL

    For over a decade, ITIL has been the leading ITSM framework adopted by enterprises across the globe. So, what is driving a rapidly increasing interest in Site Reliability Engineering (SRE) as a service management alternative?

    In its own words, Google refers to SRE as its approach to service management: “The SRE team is responsible for the availability, latency, performance, efficiency, change management, monitoring, emergency response and capacity planning.”

    In traditional ITSM terms, the role of the SRE is responsible for service level, change, availability, event, incident, problem, capacity, performance, infrastructure and platform management. While the operational practice areas may be similar, there are significant differences in how the practices are approached.

    ITIL4 Framework Compared to SRE

    Released in 2019, the newest update to ITIL4 remains a complex governance model with four dimensions, seven guiding principles, a Service Value System and 34 processes (now renamed as practices). While ITIL4 pays homage to Agile and DevOps, there is little depth in the publications released to date—to be fair, several other publications in the series are in queue  for gradual release. The framework spans just about every aspect of software delivery and operations and seemingly is trying to be the single source of truth in both principle and practice for IT management.

    SRE Is More Closely Aligned to Agile and DevOps

    By comparison, SRE focuses specifically on the reliability and resilience of complex production system operations. As an engineering discipline, SRE more closely aligns with the Agile and DevOps patterns that are being adopted by product development teams including continuous integration, testing, delivery and deployment. SREs bring the wisdom of production into the teams thereby breaking down many of the silo walls that have impeded IT for so long.

    As a service management alternative, SRE also updates traditional ITSM activities with innovative and self-organizing concepts such as management to service level objectives, error budgets, toil reduction, release engineering, monitoring/observability and embracing risk as neutral approaches to service management. The core SRE book provides practical and actionable guidance for Site Reliability Engineers on managing incidents, learning from failure, testing for reliability, load balancing, handling different types of emergencies, software engineering and capacity planning. Related publications such as the Site Reliability Engineering Workbook and Seeking SRE provide additional insight.

    Most importantly, a Site Reliability Engineer is an actual hireable job with a defined role, set of responsibilities and skills. SREs and SRE teams are encouraged to be creative, accountable and must spend 50% of their time reducing toil by engineering automation in order to make tomorrow better than today. Like Agile and DevOps, SRE supports self-regulation with policies and consequences. In fact, some consider SRE to be the third piece of the Develop (Agile), Deploy (DevOps), Operate (SRE) feedback loop.

    What This Means for the “Humans of DevOps” in Operations and ITSM

    SRE is breathing new life and opening new career paths for operations and ITSM professionals who a few years ago were battling against the mantra of “NoOps.” According to Linkedin’s 2020 Emerging Jobs Report, SRE is the fifth fastest-growing job role with 34% growth.

    Services will always need to be managed. However, competing in a digital age requires new ways of working and thinking with speed and quality as key metrics. Agility must be instilled across the value stream spectrum in order to increase flow and deliver an exemplary customer experience. Your organization may not be as complex as Google’s, but the principles and practices of SRE are applicable to all environments.

    You can read the Site Reliability Engineering and Site Reliability Engineering Workbook publications for free from Google. For those wanting to learn more about the practices and patterns associated with SRE, DevOps Institute recently released its Site Reliability Foundation certification with accredited training being offered globally by its Global Registered Education Partner network. Either way, if you are an operations or ITSM professional, I would highly recommend learning more about SRE. It is the future of ITSM as we cross the digital divide.

    — Jayne Groll

  • Cloud Center of Excellence: Pivot for Enterprise Digital Transformation

    Cloud Center of Excellence: Pivot for Enterprise Digital Transformation

    Across industry verticals, enterprise cloud adoption has increased significantly. Enterprises are leveraging cloud models to transform applications, infrastructure and operations. According to a survey by LogicMonitor, 83% of enterprise workloads will be in the cloud by 2020. This, however, is also giving rise to concerns across areas such as security, cost management, compliance, governance, lack of cloud expertise, vendor lock-in and partner management.

    To address these issues, an enterprise requires common functions under one umbrella that can define architecture standards, security guidelines and governance policies, thereby establishing an industrialized transformation approach for applications and IT infrastructure across various cloud models. This central team, which defines cloud strategy, execution and operational approach, is called Cloud Center of Excellence.

    The Cloud Center of Excellence plays the role of a centralized team and acts as the guiding and governing unit for planning, executing and operating an organization’s cloud transformation process. The Cloud Center of Excellence (CCoE) owns an enterprise’s most important cloud initiatives and works on multiple dimensions to make the client transformation journey successful.

    Advisory and Evaluation

    Enterprises running legacy applications on traditional infrastructure require a cultural shift to enable their digital transformation journey. The advisory and evaluation function of CCoE works toward evangelizing cloud across enterprises, creating cloud awareness and providing solutions advice. It also assists in identifying applications for cloud fitment by discovering application data and assessing applications besides evaluating features of cloud service providers to ensure alignment with application reference architectures.

    Design and Build

    Cloud is being adopted at a very fast pace across enterprises. This necessitates defining common design standards, configurations guidelines and automating application environment build and operational support processes. This will not only save costs but also shorten time to market and improve service levels. This function of the CCoE defines architecture, network, security and integration design guidelines besides creating application blueprint templates, configuration parameters and deployment process.

    Develop and Deploy

    The ever-changing customer expectations lead to frequent changes in business requirements. This requires fast-paced development, error-free functionalities and releases in a shorter time. This function of the CCoE defines and designs CI/CD processes, identifies and configures CI/ CD tools and implements agile development processes.

    Governance

    The CCoE needs to monitor enterprise cloud spend, ensure efficiency and establish governance policies as a part of its objectives. It also needs to define and implement a process for cloud financial management, define security compliance controls, communication processes and change management. Technical (business continuity, architectural and performance, etc.) and operation governance are some of the other focus areas of a CCoE.  

    Operation

    Operating in the cloud efficiently requires proper planning combined with an operational strategy and discipline. The CCoE defines future mode of operation for applications running on cloud. It ensures ITIL compliant processes are followed for managing cloud environment.  It enables automation of operational processes such as monitoring, backup, patch and self-healing apart from others, and defines SLAs based on business criticality for application availability and business continuity requirements. Additionally, it also defines change management and incident service requests processes.

    Partner/Product Connect

    There are multiple cloud service providers (CSPs), a plethora of tools and SaaS products that exist in the market. This CCoE function focuses on coordinating with product and tool vendors for implementation and service realization. It works with cloud service providers’ technical teams to evaluate new services besides assisting the enterprise applications team. The CCoE liaises between applications teams and CSPs/external product teams to resolve technical challenges.

    Innovation/Ideation and Knowledge Management

    CSPs continually evolve to keep pace with innovation. This function of the CCoE evaluates new services by developing proof of concepts and publishes guidelines and best practices based on PoC learnings. It also creates a knowledge base leveraging past learnings. 

    Conclusion

    CCoE plays a critical role in implementing the cloud strategy of an enterprise. It defines a framework for governance, designs guidelines and manages the cloud-specific knowledge base. It is, however, of paramount importance that it should start with a small and right kind of team. As cloud adoption increases in an enterprise, responsibilities and scope of the CCoE will expand accordingly. It is important for enterprises to empower the CCoE team allowing them to teach and uphold best practices.

    — Akhilesh Laddha

  • Pendulums and DevOps

    Pendulums and DevOps

    I have long noted the trend of pendulums in IT, particularly in organizations with longer histories. Centralized IT will be achieved, its weaknesses noticed and a movement will begin to decentralize. Decentralized IT is achieved, and people remember its weaknesses, causing centralization.

    The same is true with speed of delivery versus quality. Faster will be more important for a while, then the reality that faster causes quality concerns will surface, and reliable becomes the keyword, causing slower rollouts of better product.

    This has been going on since I started in IT, and while it is more evident in enterprises, any company that lasts long enough will see these cycles run through.

    Agile and DevOps, as the Great Disruptors, stood a chance of redefining the tensions that cause these pendulum swings. Centralized versus decentralized is less important if stakeholders are all truly involved and informed—something that the culture side of the DevOps crowd aimed to achieve. Similarly, fast and quality were promised, as the ability to recover from inevitable mistakes would be rapid, while integrated testing would uncover most issues.

    We have enough years under our belts to know at this point that these issues are, at best, resolved with a resounding asterisk that is clarified sometimes.

    One nice thing about DevOps (more than agile) is that it has not stopped evolving yet, which means there is still hope that rapid delivery and high quality can both be answered while involvement of business owners and all the various IT groups will reduce the centralized/de-centralized cycle. But we’re not there yet.

    Split.io—purveyor of feature flagging/tagging tools—has a pretty good blog about what they discovered when they went out and talked to hundreds of shops. It shows that failures are happening at an alarming rate, and at some number of shops, for your average bug, they will still be fixing this released issue when the next release goes out. That is asking for trouble, in my experience.

    Better integrated testing is certainly part of the answer. TDD is good, but not enough, since it is more granular than some tests require. Split believes that feature flags are part of the answer, and I believe they’re right. But this approach is a band-aid to the problem.

    By the time you flip a feature flag to turn off a bad feature, you’ve already released a bad feature. And you will be spending time researching and fixing it. I believe feature flags should be used to catch it when this (inevitably) happens in a complex coding environment, but at the same time the source of issues should be looked into. An ounce of prevention is indeed worth a pound of cure.

    Figuring out what is allowing bugs to slip through into production rather than counting on remediating them (using precious time to do re-work) should be a priority moving forward. Is there a process/policy that is at the source? Is it simply a training issue? Training issues are repetitive—teaching today does nothing for new algorithms/tools put to use after today, and turnover can create a training gap if you’re not staying on top of it. For some shops, is speed of delivery faster than some piece (normally testing, but some shops suffer for development on certain platforms also) of the delivery chain can adequately respond to? Do you need to re-evaluate toolchain and delivery timelines?

    We are delivering more code, faster than ever, and that is exciting. But quality matters, and anyone who relies on end-users to test is playing a very dangerous game. Find ways to ensure quality without slowing down the process more than necessary. At this point, I honestly would implement runtime-flippable feature flags as a starting point, simply to be more responsive to issues. From there, work to resolve the underlying reasons that bugs are cropping up.

    Until then, keep cranking it. Keep working with the business to get quality systems in front of users, and enjoy doing some great work.

    — Don Macvittie

  • Microservices: The Advantages of SOA Without Its Drawbacks

    Microservices: The Advantages of SOA Without Its Drawbacks

    Service Oriented Architecture (SOA) was the great hope of organizations decades ago when they sought to advance legacy system integration, reduce and bypass layers, and rapidly access the system of record. At the time, the existing solution was point-to-point integration, creating a brittle “spaghetti” middle layer that was hard to manage. This was replaced by SOA, which was later augmented with ESB. This fixed the spaghetti mess by creating an intermediate set of layers, which added complexity to the integration process.

    Unfortunately, most IT departments with legacy systems and SOA still struggle to be as agile as needed in this ever-increasing global and digitized world. 

    SOA Solved Many Problems, But Not All

    Today, upward of 90% of the world’s enterprise applications are monolithic. When they were created, monolithic design was the best approach available. However, as business needs evolved and the demand for greater agility intensified, developers and IT teams are considering breaking free of these monolithic architectures.

    SOAs have become a hindrance, not the solution they once were.

    The SOA Integration Challenge

    The promise of SOA initiatives was extending the reach of core business functions while reducing the internal expenses and complexity that grew alongside monoliths. SOAs achieved these goals by breaking the core functions of a monolith into web services using protocols such as simple object access protocol (SOAP) and eXtensible markup language (XML).

    SOAP was built for universal application communications. Because it’s based on XML, SOAs designed with SOAP could, in theory, be used to create an agnostic integration layer. Rather than struggling to piece together various proprietary systems, these protocols would open monoliths on different operating systems so they could work together.

    This agnostic integration layer would let system administrators connect pieces of a monolith to an enterprise service bus (ESB) to achieve an agile, plug-and-play SOA. In some cases, this approach succeeded.

    A great deal of “legacy SOA” still provides value. In these cases, the value the SOA provides is the behind-the-scenes server-to-server communication that mostly aids developers. When it comes to serving internal and external customers–and their rapidly changing demands–SOA isn’t always up to the task.

    To achieve service integration, the ESB must oversee the messages from start to destination. This communication isn’t as simple as the SOA vendors may have promised. Consider an SOA integration within a core banking application. The message must go from the core application on the mainframe to a branch office server. If the business logic states a message is only relevant for one business day, administrators must decide whether to move it to a queue, log or disregard when the day passes. This is a common scenario for core banking applications in an SOA, but how well does it work?

    In this example–and others like it–the ESB must know whether a business day has passed to make the right decision about where to send the message. Therefore, the integration requires an algorithm–meaning that ESB cannot simply rely on a universal rule for sending messages between the monolith and external integration.

    When administrators introduce new business logic into the monolith integration, it turns what was supposed to be an agnostic service layer into a new application layer, increasing complexity. Instead of simply integrating the monolith into a new digital service, the enterprise ends up with a larger monolith–one that includes the mainframe and all the new integration stacks.

    Despite the promises of SOA, the integration problems result in increased maintenance time, greater complexity of code and software and the continued growth (not elimination) of monolithic applications.

    The first rule of effective integration is “smart end-points and dumb pipes.” Building logic and layers into the service layer breaks that rule, increases the overall complexity and adds another legacy application to your portfolio.

    Bottom line: Trying to optimize the SOA approach will only result in larger monoliths.

    Improving SOAs with Microservices

    Using microservice architectures helps realize the original promise of SOAs, delivering the benefits of SOAs while removing many disadvantages.

    For decades, CIOs have tried to transition away from monoliths by taking traditional approaches to integration, only to find they’ve doubled down on legacy investments. All these integration stacks end up coupled to legacy systems and only result in more work for the IT team–and a less agile organization.

    The integration step of SOA was never intended to include business logic. Trying to force business logic into this approach leads to workarounds and extra effort just to achieve a less-than-ideal result.

    The key factor is how to incorporate microservices without adverse effects. Luckily, microservices architecture can be forged from SOAs by introducing these new principles:

    •   Context Mapping: When replacing an existing SOA, teams must consider the size and scope of the new microservice and apply proper contextual boundaries. For example, SOAs that typically sought broad integration with digital services should be broken down into smaller domains to simplify operation.
    •   Shared-Nothing Architecture: Too many SOA integrations create sprawling dependencies that create complexity in the tech stack. Microservices avoid these cross-service dependencies. For those looking to move to microservices from an existing SOA, it’s important to look at the list of dependencies and work toward standalone functionality.

    Ultimately, the goal should be to refactor monoliths in a way that shifts the IT stack toward microservices. By taking the right approach, you can make the most of both microservices and APIs within your SOA. You enhance your digital journey, leading you to IT efficiency, faster cycles, greater scalability and competitive differentiation.

    — Zeev Avidan

  • These Three Approaches Will Help Your Enterprise Be More Agile

    These Three Approaches Will Help Your Enterprise Be More Agile

    Nearly every organization that tries to change its culture to be more agile and better adapt to changing environments discovers that it’s harder than it looks. Agile transformation can’t be superficial or only embraced by the IT department–it needs to be completely pervasive throughout the business.

    There is no one-size-fits-all approach or playbook that can be used to fit any organization’s unique situation. Yet, there are some common takeaways and best practices that can be gleaned from organizations that have successfully undergone agile transformation. Here are three common themes I’ve come away with from talking with organizations at various stages of their agile journey.

    Creating an Agile Enterprise Relies on Successfully Orchestrating Change in Culture, Organizational Structure and Technical Capabilities at the Same Time

    Changing enterprise organizational structure without changing culture and technology will lead to factors that drag down any potential transformation. It’s equally true when addressing any one of the elements in a vacuum. Instead, enterprises must address all three at once, which makes for a tricky prioritization exercise. 

    In many cases, new solutions will lead to new problems that organizations must address as they arise. Agile teams enabled to self-organize and pursue customer problems with greater freedom may find themselves hamstrung by inflexible legacy technology. Changing technology to enable agility may uncover additional organizational difficulties, and so on. Agile transformation is a long journey, and enterprises must adjust on the fly to their own unique circumstances.

    When Making Changes to Drive Agility, Think About What Scarf Everyone Wears

    Missy Lawrence-Johnston, a principal consultant from ISG’s Organizational Change Management practice, suggests that leaders should remember a warm and fuzzy acronym when working toward change: status, certainty, autonomy, relatedness and fairness (SCARF). The SCARF model was developed by neuroscientist David Rock, and explains how individuals react to change based on perceived threats and rewards. 

    If a change negatively impacts someone on one of the dimensions they consider important, they’re more likely to push back on it. For example, a manager who cares a great deal about their status will probably have difficulty embracing an organizational shift that works to cut down on organizational hierarchy. But that same change could be highly valued by a rank-and-file employee who prefers greater autonomy. How people react is individual, which is why the SCARF everyone wears is unique to them, as Lawrence-Johnston points out.

    As enterprises make the changes necessary to embrace agility, keeping this concept in mind will make it easier to navigate the threat responses employees inevitably have.

    Measure Culture by the Behaviors People Exhibit at Work Every Day

    ISG research shows that culture and team enablement are the most important factors when transforming into an agile enterprise. While culture can seem like something nebulous and difficult to pin down, it’s best evaluated by noticing what people do every day. Odds are, any employee of a firm has at least one story that illustrates what they see as the culture of the organization where they work. An enterprise’s culture is the sum of these stories.

    Agile transformation requires some fairly radical changes for businesses that are used to working in a waterfall mindset. This often looks like a shift from a hierarchical, rule-driven culture that focuses on command and control to a collaborative, self-organizing culture with servant leadership. Implementing this degree of change takes significant work. 

    When changing culture, it’s not enough to put up a fresh set of posters with new values. Living those values and building organizational structures to support them is the key to success.

    — Blair Hanley Frank