devopsy.kinsta.cloud

Category: DevOps and Open Technologies

Editorial Calendar for February 2016

  • Permiso Makes Open Source Tool to Identify Cloud Threats Available

    Permiso Makes Open Source Tool to Identify Cloud Threats Available

    Permiso today launched an open source tool dubbed CloudGrappler that surfaces indicators of compromise in cloud computing environments.

    Based on a cloudgrep project originally created by Cado Security, CloudGrappler detect patterns that Permiso, a provider of an identity threat detection and response platform, associated with tactics and techniques commonly used by various cybercriminal syndicates.

    The tool enables DevSecOps teams to define the data sources they want to scope in a scan of an Amazon Web Services (AWS) or Microsoft Azure cloud. Via a JSON file, DevSecOps teams can access a list of predefined tactics, techniques and procedures (TTPs) that are commonly used to exploit cloud computing environments. DevSecOps teams can also dynamically add queries or a file with multiple queries to scan a specific data set. After scanning, CloudGrappler delivers a JSON report that provides a breakdown of the scan results.

    Ian Ahl, senior vice president for P0 Labs, the research arm of Permiso, said that capability makes it possible for DevSecOps teams to identify issues in cloud computing environments that warrant further investigation before there is a major incident. The primary goal is to make it simpler for DevSecOps teams to determine if they need additional assistance to thwart a potential threat, he added.

    Cybercriminals are, like almost everyone else, creatures of habit and often tend to leverage the same TTPs multiple times simply because they know they’re effective. However, those TTPs also provide clues to their presence in an IT environment. Once discovered, a DevSecOps team can decide if they can remediate the issue themselves or need to tap external cloud security expertise, noted Ahl.

    In general, cloud computing environments may be more secure than most on-premises IT environments. However, the processes used to provision cloud computing environments are often flawed because developers typically don’t have much cybersecurity expertise. Misconfigurations are often rife across cloud computing environments.

    Cybercriminals are also especially adept at stealing credentials to gain initial access to cloud computing environments, which they leverage to escalate privileges in a way that provides them unfettered access to the entire cloud computing environment.

    Adoption of DevSecOps best practices has improved cloud security, but there are still plenty of instances where, for example, a port inadvertently left open has enabled cybercriminals to infest a cloud computing environment with malware that could lie dormant within cloud applications for months before finally being activated.

    There are simply not enough cybersecurity experts available to secure cloud computing environments, so it’s going to be largely up to DevOps teams to secure them. The issue is that in the race to build and deploy applications as quickly as possible, cloud security best practices are either forgotten or simply ignored. Of course, it’s that very carelessness that cybercriminals are counting on to allow them to wreak havoc in an era where the percentage of mission-critical applications running in cloud computing environments will only continue to increase exponentially.

  • 7 API Observability Anti-Patterns to Avoid

    7 API Observability Anti-Patterns to Avoid

    Observability has become a buzzword, and the API world is slowly catching up. But be careful when looking for content about API observability- there are many outdated best practices and irrelevant pieces of content.

    I was recently invited as a guest to Stephen Townshend’s SlightReliability podcast, and his brilliant blog post on “Bad Observability” inspired me to start ranting about the topic.

    Here is a follow-up to our discussion: The observability anti-pattern to avoid when working with APIs. Thanks, Stephen, for the inspiration!

    Anti-Pattern: You Forgot Your Users

    It’s easy to lose sight of the fact that APIs are built for people to use. APIs are not just a technical interface; they are a bridge that connects digital experiences and enables people to accomplish tasks efficiently.

    There are two different kinds of users for your API:

    • The developers who are integrating your API into their system.
    • Your customers’ users, who will ultimately interact with the applications or services that rely on your API.

    If you can’t understand how they interact with your APIs, you’re missing half the picture.

    When developers integrate with your APIs, how long does it take them to make their first successful API call? What are typical errors slowing them down? Observability plays an essential role in surfacing insights to improve developer experience.

    Once your customers’ applications or services relying on your APIs are in production, can you track how your releases impacted their requests? Which version of your APIs are they using? What are the most popular endpoints? Are all customers getting the same performance? Same error rates?

    Anti-Pattern: Relying Only on API Monitoring

    API monitoring typically involves sending automated test requests to API endpoints and verifying the responses. It helps you keep track of the health of your APIs even when there is no customer activity at all, and can be used to report uptime.

    But API monitoring is nothing more than black box testing; you don’t see what is happening inside. You just know that your requests reach the API and receive responses, but you can only create and maintain tests for some possible use cases. You will miss something.

    Anti-Pattern: Using API Access Logs for Troubleshooting

    API access logs are records of requests and responses made to an API, capturing details such as the requestor’s IP address, HTTP method, requested endpoint, status codes and other relevant information for monitoring, debugging and security purposes.

    Using access logs to troubleshoot API issues is like searching for a needle in your microservices haystack. One of our users had a performance issue with their APIs; they spent days searching for the root cause using access logs. Is it the gateway? Does it need more CPU? Is it a configuration issue? They enabled distributed tracing and, within seconds, they could see what was happening: All the time was spent in the authentication server issuing JWT tokens.

    To really understand what is happening in your system, you need to be able to track a single user request from the API gateway to the upstream services and their dependencies for effective troubleshooting.

    As Martin Thwaites puts it: “Logs are a poor man’s observability tool.” You need distributed tracing.

    Anti-Pattern: Different Teams, Different Tools, Different Data

    Different teams have different needs. The tool that works for your DevOps team might not fit your product team best. By forcing everyone to use the same tool, you’re not promoting unity but inefficiency.

    However, while promoting tool diversity might be the right move for your organization, there should be some coordination and integration between different tools and data sources. Open standards like OpenTelemetry help ensure that information can be shared and leveraged across teams and tools.

    The DevOps and the product teams should rely on the same error rate when assessing the performance and reliability of the newest API product launched.

    Anti-Pattern: One Size Fits All

    Different API architecture styles have different needs when it comes to observability.

    REST APIs might be today’s default, but newer styles like GraphQL and gRPC are gaining popularity. Each of these styles has distinct characteristics and requirements regarding observability. Just think about GraphQL’s way of handling errors, which differs significantly from REST APIs. Web Socket and event-driven APIs have their own unique observability challenges, as well.

    Different API architectures have different needs, and your observability strategy needs to reflect that to improve the overall reliability of your application.

    Anti-Pattern: Observability for Production Only

    Why aren’t you using observability in pre-production?

    By using distributed tracing during the API development life cycles, developers can trace the flow of requests and responses across different services, providing a comprehensive view of how the API behaves under various scenarios.

    With observability, they can identify performance bottlenecks and even detect architectural problems like N+1 problems in GraphQL queries.

    For teams leveraging APIOps practices, observability data can be used as an additional validation before promoting an API to the production stage.

    Anti-Pattern: Starting the Trace at the API Gateway is Overrated

    Tutorials about observability usually start the instrumentation process at a microservice level. It is great to have detailed insights into your microservices. But a lot will happen at the API gateway level when dealing with APIs.

    You want to capture all your user transactions, including those that never reach your microservices because of rate-limiting rules, an authentication problem or a caching mechanism.

    Starting the trace at the API gateway gives you a clear entrance point and a complete picture of the journey of all your users. That’s why it’s important to use a modern API gateway with built-in support for OpenTelemetry.

    So there you have it: Seven API observability pitfalls to avoid. Keep your APIs observable and your users happy!

    To hear more about cloud-native topics, join the Cloud Native Computing Foundation, Techstrong Group and the entire cloud-native community in Paris, France at KubeCon+CloudNativeCon EU 2024 – March 19-22, 2024.

  • Driving the DevOps Evolution: ArgoCD, Tekton and Seamless Migrations

    Driving the DevOps Evolution: ArgoCD, Tekton and Seamless Migrations

    The landscape of CI/CD frameworks has evolved over the last couple of years. From the early days of Flux and Jenkins to the current focus on ArgoCD, this journey reflects the industry’s momentum toward efficient and reliable deployments.

    In recent times, we have seen many modern software delivery platforms adopt ArgoCD or Flux for automating deployments within their ecosystems. However, as multi-cloud deployments and scalability still remain a challenge, we are seeing the industry shift toward more powerful Kubernetes-native frameworks like Tekton.

    Evolution of CI/CD Frameworks

    Flux: Paving the Way for GitOps

    Flux was one of the pioneers in promoting the GitOps methodology. It allowed developers to manage infrastructure and deployments declaratively through version-controlled manifests.

    GitOps, as introduced by Flux, emphasized the use of Git repositories as the single source of truth for configuration, making deployments more reliable and traceable.

    Jenkins: The CI/CD Workhorse

    With its extensibility and vast plugin ecosystem, Jenkins quickly became the go-to CI/CD solution for many organizations. While Jenkins was powerful, its complexity and challenges in maintaining pipelines led to the exploration of alternative solutions.

    Spinnaker: Scaling Deployments

    Spinnaker emerged as a powerful tool for continuous delivery with a focus on scalability and multi-cloud deployments.

    Its support for canary releases, blue-green deployments, and pipeline orchestration attracted organizations aiming for more sophisticated deployment strategies.

    ArgoCD: The Rise of GitOps

    ArgoCD brought a renewed focus on GitOps, building upon the foundation laid by Flux. It provided a declarative way to manage Kubernetes applications, ensuring that the desired state in the Git repository is continuously in sync with the cluster state.

    The adoption of ArgoCD has been widespread due to its simplicity, scalability and ability to synchronize with multiple clusters.

    ‘Modern’ DevOps Platforms: Riding the ArgoCD Wave

    Many ‘modern’ software delivery platforms have either incorporated or integrated separately with ArgoCD to attempt a complete software delivery approach. This integration enables users to adopt GitOps principles seamlessly within their existing workflows.

    Here’s a glimpse into what it looks like to work with ArgoCD on these platforms:

    1. First off, you will have to orchestrate CI separately, which leads to context switching and a non-unified CI/CD workflow. You will need to leverage plugins on most of these platforms and hop back and forth between exploring steps and adding them to your pipeline for a few others.
    2. For CD, it is essential for you to connect your ArgoCD instance by keying in the server URL. This requires a working understanding of ArgoCD and also a working setup of the same before even signing up to these DevOps platforms.
    3. ArgoCD has to be defined as an artifact source and its specific steps must be added as deployment steps while configuring a pipeline. Other platforms require you to include steps or commands within the YAML files to interact with ArgoCD.

    While GitOps automation can be achieved by integrating ArgoCD with DevOps platforms, it has to be noted that these are not native workflows. There will be challenges that crop up, like:

    1. Improper synchronisation between repository and the deployment state on these platforms could result in unintended deployments or configuration drift.
    2. GitOps practices, when coupled with certain platforms, may have limitations in handling multi-cloud or hybrid cloud deployments.
    3. The complexity increases with the size of the infrastructure. The larger the number of clusters and environments the greater the number of pipelines and messier the orchestration.
    4. The fact that ArgoCD needs to be set up and running before even subscribing to DevOps platforms defeats the purpose of having low-code/no-code DevOps.

    The Rise of Tekton

    As the industry looks for solutions to address the challenges posed by existing CI/CD frameworks, Tekton has emerged as a promising alternative. Prominent tech enthusiasts in the DevOps space, like CloudBees co-founder Sacha Labourey, and market momentum make it clear that there’s a significant number of organizations working on integrating Tekton within their platforms, just like we saw with ArgoCD a couple of years back.

    What’s amazing is we already had platforms like Ozone, which started working on making Tekton enterprise-ready almost three years back and have further evolved over the years.

    Tekton: Container-Native CI/CD

    Tekton is a Kubernetes-native framework for building CI/CD systems. It provides custom resources to define CI/CD pipelines as code, allowing for greater flexibility and extensibility.

    This is done by introducing the concept of tasks that run within a pipeline. These tasks are reusable and pre-defined, all of which can be viewed in the Tekton hub.

    What ArgoCD achieves for GitOps is done with just one of the hundreds of task templates that Tekton comes with.

    Too good to be true? Well, every rainbow has its rain! Migrating from one framework to another is a huge task, and that is precisely why many enterprises shy away from migration.

    Can LLMs Help Automate the Migration of Existing Pipelines to Tekton?

    If you look at the diagram below, the answer is a resounding yes. The model is trained based on existing pipelines and what their corresponding Tekton pipelines look like.

    The overarching objective of this workflow is to facilitate a seamless conversion process using large language models (LLM) for generative AI.

    Central to this approach is using the chat context and the strategic application of roles for LLM models to establish context. Additionally, sample pipelines are leveraged to train and fine-tune the model to define how the corresponding YAML files of all these platforms look for any given pipeline.

    Here’s a more detailed read presented by Ozone as an industry-first last year at KubeCon North America.

    Conclusion

    The evolution of CI/CD frameworks reflects the DevOps industry’s dynamic nature and the demand for efficient software delivery pipelines. From Flux and Jenkins to ArgoCD and Tekton, each framework addresses deployment challenges.

    ArgoCD gained traction based on its GitOps principles and Kubernetes integration, yet faces synchronization, multi-cloud support and scalability hurdles. Tekton, a Kubernetes-native CI/CD framework, offers scalability, event-driven architecture and cloud-native practices to make up for these shortcomings.

    Whether embracing ArgoCD or exploring Tekton, the goal is resilient, scalable pipelines driving innovation and time-to-market acceleration.

    To hear more about cloud-native topics, join the Cloud Native Computing Foundation, Techstrong Group and the entire cloud-native community in Paris, France at KubeCon+CloudNativeCon EU 2024 – March 19-22, 2024.

  • Navigating the Rise of AI-Generated Code in Software Development

    Navigating the Rise of AI-Generated Code in Software Development

    The year 2023 saw the rise of many technologies and innovations, but none quite as prominent as generative AI. Trained on vast amounts of data, these models are designed to “generate” the next sequence of values based on inputs and create novel content for various purposes. The emergence of generative AI has forced industries to rethink how they operate and where adjustments should be made. A recent study from ISG indicates that financial services are the leading industry for generative AI adoption, making up 24% of total use cases. This is followed by manufacturing at 14%, healthcare and pharma at 12% and business services at 11%.

    More recently, organizations have started contemplating generative AI’s impact on software development and its ability to effectively generate code. AI-enabled tools will have a major impact on software development, increasing the volume and velocity of code delivery and innovation. However, there are ethical and practical guardrails that need to be considered.

    Key Capabilities of Generative AI in the SDLC

    The advantage of AI tools lies in their ability to enhance the software development workflow by automating repetitive tasks. Mundane and time-consuming boilerplate code generation or routine code reviews can now be handled by the same machines that we are writing the programs for. This not only accelerates the software development lifecycle (SDLC) but liberates developers from monotonous tasks, allowing them to focus on more creative projects that require human insight.

    For developers, the strength of AI-enabled tools lies in their ability to expedite the development process. By harnessing machine learning (ML) algorithms, AI can analyze vast datasets, find patterns and generate simple code quickly, often requiring numerous iterations to get it right — hopefully! But at its core, the benefit of AI-enabled tools stems from the premise that it’s easier to refine a set of initial ideas than it is to generate them. In other words, generative AI models create the “sparks” that a good developer can turn into a full creative process.

    Challenges, Concerns and Ethical Considerations

    As the adage goes, there are two sides to every coin. While AI tools hold the power to provide immense benefits to developers, they also introduce an array of risks that cannot be overlooked:

    ● Challenges in understanding context and user requirements: Despite its sophistication, AI struggles to grasp the intricacies of specific project contexts and the diverse needs of end-users. The contextual understanding that human developers possess, gleaned from experience and intuition, is still a facet where current AI encounters hurdles.
    ● Potential security risks: The automated nature of AI-generated code introduces potential security risks that demand careful consideration. While AI algorithms strive for efficiency, they may inadvertently generate code with vulnerabilities. Addressing security concerns requires ongoing vigilance, thorough testing, and the incorporation of robust cybersecurity measures to ensure the integrity of the developed software.
    ● Adapting to evolving technological environments: The dynamic nature of technology presents another challenge for AI-generated code, as rapid changes in programming languages, frameworks and methodologies require an adaptive approach. AI models may struggle to seamlessly transition to new technological advancements, potentially causing compatibility issues and requiring continuous updates to maintain relevance.

    It’s important to note that developers will not be the only ones affected by AI’s integration into the SDLC. Security teams are barely keeping pace now, and in many organizations, they’ve become the bottleneck. Increased developer velocity will put security teams under even more pressure—the volume of scans, the range of vulnerabilities and the patches that need to be applied are all guaranteed to skyrocket.

    Automation Alone is Not Enough; We Still Need Human Developers

    While the introduction of AI marks a significant milestone in the evolution of software development, it’s essential to recognize the nuanced interplay between automation and human ability. Rather than replacing human expertise altogether, generative AI is poised to augment and complement the skills of developers. Human creativity, critical thinking and the ability to navigate complex problems will continue to be invaluable assets in the development process — true potential only appears when collaborating seamlessly with human developers, blending the efficiency of automation with the creativity and problem-solving skills inherent to human intelligence.

    Looking towards the future of software development, the trajectory of generative AI presents intriguing possibilities. It has become increasingly clear that the rise of AI-generated code is not a story of replacement but one of enhancement. While AI brings undeniable efficiency and automation to certain aspects of coding, the role of human developers is still irreplaceable. The understanding of context, the intuition required for creative problem-solving, and the adaptability to evolving technological environments are hallmarks of human expertise. These are facets that, as of now, AI struggles to replicate.

    As we embrace the future of AI-generated code, let us not forget the essence of our craft. The evolution of technology is inevitable, but it is the human touch, ingenuity and ethical foundation that will define the software landscape of tomorrow.

    Image courtesy of Josh Calabrese via Unsplash

  • 3 Quick Fixes for Event-Driven Architecture (EDA) Complexity

    3 Quick Fixes for Event-Driven Architecture (EDA) Complexity

    Event-driven architecture (EDA) and the adoption of event streaming throughout enterprises are increasingly becoming essential architectural requirements for businesses across almost every industry. However, the more companies use it, the greater the complexity becomes. Event portals can play a pivotal role in not only gaining comprehensive insights into the intricate and expanding event-streaming ecosystem but also facilitating the auditing, management and governance of events.

    According to a recent IDC Infobrief, 93% of companies employing EDA across various applications reported that EDA either met or exceeded their expectations. Moreover, 82% of IT leaders intend to apply EDA to two to three new use cases within the next 24 months.

    Nonetheless, the number of event streaming use cases results in a surge of data volumes. This, in turn, entails the adoption of more brokers, projects and products for event streaming. For instance, an organization might use open source Kafka for one specific use case while turning to brokers like Confluent for another and Amazon MSK for yet another distinct purpose.

    While the real-time processing of data is certainly a positive development, numerous experienced EDA adopters have yet to maximize its potential. Frequently, the situation involves a one-to-one exchange, where a stream is consumed only once, or organizations, having advanced through various stages of maturity with multiple streaming use cases, find themselves entangled in a complex network of brokers, clusters, topics and overlapping schemas.

    Let’s explore three typical challenges that organizations encounter as they progress toward event-driven maturity.

    Success Needs Visibility – Make Sure Your Event Portal Can Provide It

    Application decoupling is great for runtime, but this lack of visibility on both the publish and subscribe side can cause issues when making changes to existing applications. Application producers don’t know who the consumers are and consumers don’t know who the publishers are.

    An event portal offers a single window into an event-steaming ecosystem. It provides a native discovery agent to scan, for instance, a Kafka cluster and its schema registry to produce a visual representation of every topic and its schema(s) across multiple versions–and, importantly, who its consumers are.

    If an organization were to decompose its monolithic applications into event-driven microservices that communicate with each other via, say, Apache Kafka, one of the biggest challenges is to understand and manage the infrastructure and information flows.

    Specifically, how to see the microservices affected by a given change before deploying a new feature or function to ensure it won’t bring that system down – even for as few as five seconds.

    By using an event portal, organizations can automatically scan their system and visualize a complete map of endpoints and event streams instead of manually diagraming event streams between microservices.

    Using Data at Its Best – in Real-Time

    Let’s extrapolate the poor visibility issue from one application across the entire enterprise.

    For instance, if you’ve been using Kafka for several years, especially with many departmental or specific application-type use cases, your Kafka cluster now is a treasure trove of real-time data. But data is most valuable at the moment it’s produced – as per Forrester: “Data is, without a doubt, valuable. But when stored in vaults and locked down, it is not.” Real-time data is the most valuable data that exists. But siloed event streaming data means other departments, decision-makers, customers and partners don’t know about it – therefore, it’s not getting shared or reused to its full potential.

    Typically, developers don’t have anywhere to go to find this data treasure trove. Some have resorted to building Confluence or Wiki pages that try to document that this data exists, using SharePoint or Word documents. Noble intentions, but without real-time data mining and updates, this information quickly gets stale and out of date.

    Again, this is where an event portal makes a difference, providing a perpetually up-to-date catalog of data detailing all topics, event streams, schemas and pub/sub interfaces for each application, along with owners and points of contact, as well as changes for each of the managed EDA entities. This helps expedite development by letting developers easily share, discover and re-use any existing Kafka or event streaming asset, both inside and outside the organization.

    For a real-world example of this in action, look at the Federal Aviation Administration and its SWIM (system-wide information management) infrastructure that distributes real-time information to FAA systems across the United States.

    Through secure gateways, external partners of the FAA can tap into the flow of events. Airlines and other industry partners that need the SWIM data get it in real-time, without perpetually requesting updates. Whether it’s the availability of gates at an airport, the position data of planes in the sky, or the weather for a region – external partners have the latest information without needing to ask.

    Don’t Forget to Govern Event Streams!

    The decentralized and dynamic nature of event-driven systems introduces unique security challenges. One of the common trade-offs with event streaming is that it does include access control rules, but developers can err on the side of being too permissive to ensure agility. If streams are not visible, properly cataloged, and data updated regularly, this presents problems with data security, governance and compliance.

    Using an event portal designed to give organizations visibility and control over their event streaming can turn security from reactive – to proactive. Users can organize systems into application domains, create and import payload schema definitions in a variety of formats, including AsyncAPI, define event interactions between applications and microservices, and create events and associated topic addresses using topic structure-proven practices. With the right visibility, administrators can ensure security, governance and compliance with internal policies and government regulations.

    EDA to the Rescue!

    A single, multi-broker event gateway technology offers a solution for organizations to uncover, regulate and oversee the life cycle of their real-time event streams throughout the enterprise. Its significance will continue to grow as more organizations adopt EDA as a fundamental platform.

  • 6 Transformative Mainframe Predictions for 2024

    6 Transformative Mainframe Predictions for 2024

    The success of organizations in today’s digital economy demands speed — the ability to quickly respond to market trends with new applications and services, instant access to critical data, and resolution of issues even before they affect service. As we move into the new year, the adoption and adaptation of emerging technologies on the mainframe will enable enterprises to embrace changing market forces, turning them into a competitive advantage.

    Below are six predictions of how mainframe organizations will leverage new technologies to advance the platform in 2024.

    Prediction: AI/ML Adoption Grows

    The roles of artificial intelligence (AI) and machine learning (ML) and their adoption across the mainframe, from application development to monitoring to data management, will continue to grow.

    The generative AI revolution is real and here to stay. In the coming year, organizations will continue to explore best-use cases for its integration into mainframe management. Development teams will leverage key AI-recommended insights into automated workflows and code generation, improving developer productivity. Database administrators (DBAs) will look to automate data management processes to efficiently handle the demand for the diverse datasets required by generative AI tools, and mainframe operations teams will increase their reliance on AI-driven predictive analytics and root cause analysis.

    As adoption increases and experience is gained, organizations will work past their AI trust issues and begin to harness the power of automation to become a fully autonomous digital enterprise. At the same time, there will be a growing need to govern AIOps and MLOps to ensure the validity and accuracy of ongoing model development.

    Prediction: DevEx Comes to the Forefront of Companies’ Innovation Strategies

    With the long-awaited shift of mainframe workforces from those with decades of experience to a new generation of stewards now in full effect, organizations will find a new imperative to attract and retain top talent by providing developers with familiar tools and development environments.

    Professionals who have recently graduated from college or are moving from other platforms bring with them an expectation of integrated and automated DevOps practices. The move towards enterprise-wide DevOps and continuous integration and continuous deployment (CI/CD) pipelines will continue, with many companies also looking to employ cross-platform tooling. By giving developers the freedom to use their integrated development environment (IDE) of choice, organizations can decrease onboarding time and get new mainframe developers up to speed faster, enabling them to understand code and its function quickly and easily, and creating a seamless, frictionless way to make changes, debug, and deploy.

    Prediction: Database DevOps Bridges the Gap Between DBAs and Developers

    Changing customer demand and the need for agility will accelerate the drive for developers to seamlessly access and manage data sources. The demand for an agile development environment that extends from applications to databases will increase the inclusion of database DevOps in CI/CD pipelines, including version-controlled database schemas and data, automated database provisioning, and database testing as part of the pipeline.

    By embracing agile practices, such as incremental database changes and rolling deployments, organizations will bridge the gap between DBAs and developers and ensure that mainframe databases evolve alongside application code. This cohesive approach will enhance collaboration, reduce conflicts, and expedite application development, positioning mainframes as agile and adaptable components within the rapidly changing IT landscape.

    Prediction: Democratizing Mainframe Data

    The move ― and the need ― to democratize mainframe data, making it available to AI/ML analytics engines, will gain momentum.

    Organizations have matured their data lakes and gained valuable insights from data housed on distributed and cloud platforms, but the efforts needed to extract and transform mainframe data stored on tape and virtual tape libraries (VTL) has left volumes of business-critical data behind. By operationalizing mainframe integration with the cloud, organizations can make it easier and faster to access mainframe data and transform it into open formats, making it accessible to analytics engines and AI/ML applications.

    The ability to realize the hidden potential of data they already store will make cloud-based storage of mainframe data a prerequisite for enterprises looking to digitally transform their businesses and gain competitive advantage.

    Prediction: Object Storage Becomes Critical for Security

    Mainframe-dependent organizations will seek to integrate hybrid cloud object storage to fortify mainframe data protection against escalating cyberthreats and ransomware attacks.

    In 2024, cyberthreats will almost certainly increase in number and sophistication. The European Union’s Digital Operational Resilience Act (DORA) seeks to ensure that organizations remain vigilant and protect their systems and their customers’ data against increasing threats.

    With DORA set to come into effect in January 2025, and further regulations and guidelines on the horizon, managing mainframe data across diverse locations while prioritizing immutable data copies will be imperative. Organizations will pivot towards proactive strategies, aligning their mainframe infrastructure with hybrid cloud solutions to meet evolving compliance standards and shield against the growing sophistication of cyberthreats.

    Prediction: Innovation Takes the Lead

    Organizations will adopt ruthless prioritization, shifting focus from maintaining services and applications to building new ones for the future.

    Having seen firsthand the success of ongoing mainframe modernization efforts, organizations will place a new focus on building skills and specialties that align with future goals and product roadmaps, putting them in a position to capitalize on coming trends and giving them the ability to quickly respond to changing market forces with innovative new products and services.

    As companies mature their approaches to AI on the mainframe, many will turn to building “applied AI” teams tasked with developing proactive solutions to better analyze data and forecast trends. In short, organizations will think logically about how to embrace AI rather than being embraced by it.

    In 2024, mainframe organizations will accelerate the adoption of emerging technologies like AI and the cloud to increase the adaptability, resilience, and efficiency of the platform while situating it for even greater success in the future. The mainframe will continue to be a platform of innovation, integrating with new technologies and practices borrowed from other platforms to further the evolution of a hybrid IT environment that meets and exceeds the demands of the modern digital economy.

    The role of AI, the impact of data governance regulations, the mainframe skills gap, affordability, and interoperability requirements are all brewing a perfect storm around the mainframe that will mark 2024 as a year of significant transformation that will define the leaders from the laggards.

  • Software Development in the Age of AI: How to Balance Quality and Speed

    Software Development in the Age of AI: How to Balance Quality and Speed

    There’s no mistaking the reality of AI’s influence. Across industries, businesses are calling on tech workers to make the most of this transformative technology as demand skyrockets and companies race to overcome the competition. A flagging economy only pushes expectations for AI use—in software especially. To contend with arduous workloads, 92% of developers are already using AI code generators, according to GitHub. That’s an overwhelming majority, not to mention that this usage is poised to continue generating steam.

    In light of this, we must ask ourselves a critical question: How can businesses ensure developers retain the quality of their AI-generated code? AI is a useful complement to human developers to maximize their time and allow them to focus on projects best suited to their skill sets. That said, companies have to put safeguards in place to harness the technology, not just for boosting developer productivity but also for quality outputs.

    AI Can’t Replace Human Developers

    AI code generators are a useful tool for a developer’s toolbox. Efficient, easy to use, and helpful (in a fairly rudimentary way in these early AI days), they give developers the ability to outsource some code-writing, which helps alleviate mounting demands as companies expect them to produce massive amounts of code — and fast. As most developers continue to buy into AI for productivity benefits, though, comes the responsibility to gut-check what these AI tools produce.

    There are multiple limitations and risk considerations when it comes to AI-generating code. Lack of source knowledge and context is one of them. AI pulls from what it’s taught (i.e., past data, natural language processing, machine learning), meaning it combines too many sources to trace specific pieces of coding back to one place; it’s impossible to trace the code back to a single developer or project. Security, too, becomes an issue; you can’t guarantee that what is produced is safe or clean as is. And, finally, AI doesn’t check for quality, which increases the likelihood that the code will be difficult and costly to maintain. All this ladders up to the mounting technical debt companies are already contending with.

    Developers can’t assume that any AI-generated code is of quality to be put directly into production. Human workers must review code thoroughly to ensure it meets clean code attributes–secure, reliable and maintainable. Ignoring these tenets puts businesses at risk of falling victim to the trillion-dollar cost of bad software. That said, there are still undeniable benefits to using AI in a coding environment that make it an invaluable tool for any company that wants to stay at the top of their software game.

    Working Responsibly With AI in Software Development

    Despite its potential shortcomings, when used safely and effectively alongside a clean code solution, AI brings a tremendous opportunity to the software space that has already made it a key tool for developers as their workloads and demands mount. For one, AI has the incredible ability to churn out code at a speed that far outpaces its human counterparts. This tech can complete tedious work that may take valuable time out of a developer’s day in a matter of seconds. AI can not only move a project forward at a faster rate, but its quick coding capabilities can also give developers a jump start on their projects and help them figure out where to begin.

    While the source of AI’s knowledge base can present a challenge, the vastness of that knowledge is a benefit. As long as a developer can input their preferred language and their specific needs, AI can share the best approach to solving a problem. Its algorithmic assistance can help lead a developer through parts of their projects step by step. This also alleviates the burden of research from developers, who can spend less time checking for accuracy than they would trying to understand different programming concepts or code snippets outside their area of expertise. AI can also handle the slog of documentation, which can not only bring clarity to what didn’t work during the development process but reduce time to delivery overall.

    Developers who don’t harness AI do so at the risk of falling behind the competition as this tech becomes such a vital piece of everyday work in software. But there’s no mistaking the fact that to best incorporate AI, developers still need to ensure its accuracy before any code generated by AI makes it to production.

    GenAI Requires Automated Tools as Safeguards

    As I previously said, developers can’t assume that any AI-generated code is clean and fit for production. That means human skills are still required to check the output. However, this kind of work can often fall to developers or DevOps, and having human workers check code on their own creates an incredible workload that can even negate the use of AI in the first place.

    The best way for companies to mitigate this and mitigate technical debt is to implement automation in the part of the process that scans and monitors AI-generated code. But that can’t be done without developer buy-in. Those creating the software are the people who will best understand the development process as they work through it day in and day out, so their input is required to make the most out of automation.

    There are developer tools that can handle this kind of work while still including developers in the process. For example, introducing code-scanning into the CI/CD process can not only help continuously monitor AI-generated code for bugs and issues, allowing developers to focus on other more challenging aspects of their projects, but it also gives them insight into errors and allows for speedier resolutions. These tools aren’t necessarily new to development, but their ability to identify bugs can help developers prioritize other aspects of the process, particularly as the amount of code reaches exponential levels compared to decades ago with the introduction of GenAI.

    Static application security testing (SAST) also allows for the identification of security vulnerabilities that can potentially arise from AI-generated code. This is critical in the software development lifecycle (SDLC) so developers can spot vulnerabilities before the code goes into production. AI-generated code is only as strong or secure as that source, and since it’s not entirely possible to verify, developers need to take every precaution available to avoid security issues that rack up costs in the long term.

    The vital part of using these tools is ensuring they’re integrated into the process in a way that makes sense for developers rather than creating additional work and headaches with a cumbersome process that doesn’t fit the way teams think and work. But, the need to check AI-generated code shouldn’t be underestimated. Developers can’t simply trust what they’re given. By including a level of automation within the development process—and doing so early and often—developers are in a position to produce better-quality, more secure software to push a business forward.

    Effective Adoption of AI Will Encourage Business Growth

    While it’s difficult to predict exactly where AI will take us in the years to come, it’s equally undeniable that it will play a critical part in the future of software development. Yes, AI is already helping to streamline processes and letting developers prioritize the tasks that best take advantage of their skills. But, as noted above, that doesn’t mean the code output is entirely safe, secure or accurate. Developers still need a level of automation to deliver code that best takes advantage of AI while keeping their workload manageable and their focus on other parts of the development process.

    With the right guardrails in place, developers can guide AI to create the kind of clean, secure, and reliable code that aids businesses rather than create costly problems for them, both now and down the line. Human skills, with a boost from AI, plus code scanning safeguards, give organizations the winning combination to put themselves in the best possible place to succeed in a rapid, fierce competitive landscape of digital business.