Category: Continuous Delivery

Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.

  • Low-Code: Bridging the Gap Between DevOps and Business

    Low-Code: Bridging the Gap Between DevOps and Business

    The need to innovate faster, manage multiple environments, optimize an application’s life cycle and achieve continuous delivery are aspirational goals that a successful DevOps team strives for. But there are loads of challenges, as meeting these goals increases the pressure on DevOps teams to adopt improved application architecture, new tools and developer workflows.

    The overarching principle of DevOps is continuous effort, including continuous business planning, collaborative development, continuous testing, continuous release and deployment, continuous monitoring and collaborative customer feedback. This is where low-code development platforms are emerging as an integral way to empower DevOps teams.
    If you look closely, ​​low-code and DevOps share common benefits: The ability to offer faster, scalable and complex applications consistently to meet business needs.

    However, many DevOps teams still rely on conventional manual coding and automation. Low-code can also supplement the talent shortage by enabling folks without extensive programming experience to develop and deploy applications faster.

    How Low-Code Platforms Impact DevOps

    Automated Application Deployment: Low-code platforms increase the speed of DevOps while also reducing the overall complexity by providing a unified platform to enable the cultural shift. From managing version control to validating builds, low-code platforms do all the heavy lifting for your developers. With low-code, you can deploy almost anything in a single click—user interface, workflows, database and/or integration components.

    Clutter-Free DevOps: Low-code platforms allow you to integrate with different programming tools using APIs, SOAP and REST web services.

    Hassle-Free Migration: Organizations can keep their legacy systems alive longer by making their functionality available to many other systems. Consider developing an Android and iOS mobile application on top of your existing CRM system. While your core system maintains, a low-code platform allows you to develop add-ons or new apps on top of it. Furthermore, because low-code platforms provide easy-to-use tools, citizen developers may participate in the migration process.

    Maintenance at Scale: With low-code, you don’t have to worry about updating apps over time because the platform designer always updates and refines the stored interfaces—and, as a result, your application—without even bothering you.

    Enterprise-Grade Security: Security is a top consideration when building scalable apps. Low-code platforms enable your IT and DevOps teams to manage data and user access. Each user can be provided with access to only the information and functionality that they require based on their roles and responsibilities.

    Compliance and Security: Legitimate low-code platforms adhere to the Global Compliance Assurance Standards, as well as ISAE and ISO certifications. Security should be a top priority for your platform, which will serve clients from a variety of businesses, including government entities. Besides integrating robust security and governance features, make sure that your platform has also implemented an ISO/IEC 27001-compliant information security management system (ISMS).

    Democratizing IT: Bringing Business and DevOps Together

    A major challenge for DevOps is to constantly meet the changing needs of the business. The invariably dynamic nature of processes demands continuous optimization, which turns out to be a bottleneck for business, especially if IT can’t deliver applications on time. To make this whole approach scalable, low-code platforms provide multiple environments for your business and DevOps team to test, build, deploy and modify applications collaboratively.

    A comprehensive low-code platform offers the means of both visual programming and coding capability. For usability reasons, these tools are clearly separated and presented to the development team in a way that allows citizen developers and business users—who are most commonly the domain experts—to use the visual components while developers can work on adding custom code to extend the capabilities of the application.

    While a complex app can only be developed by an experienced professional developer, modifying the look and feel of an app or building/modifying basic functionality can be carried out by business users/citizen developers.

    Moving IT Up the Value Chain From Executor to Enabler

    IT teams are now viewed as critical to enabling their businesses to achieve greater productivity and profitability. By providing a self-serve model using a low-code platform, you can enable citizen developers to build custom apps, employees to use portals at their convenience and create dashboards to improve everyone’s visibility, among other value-adds.

    The key is to provide your teams with the right set of tools with the right level of access while still enabling IT to manage and monitor these low-code applications’ reach. For DevOps and IT ­leaders, it provides an opportunity to empower businesses with strategic insights and recommend solutions that best suit business goals, mitigating risks and offering budgeting strategies, rather than just being a facilitator of technology.

  • Overcoming Challenges to Automating DevSecOps

    Overcoming Challenges to Automating DevSecOps

    In the last few years, DevSecOps has been widely adopted among organizations looking to get proactive with their security. Traditionally, development teams would continuously implement and deploy new applications into the enterprise and security was an additional bolt-on at the end. 

    However, as the threat posed by cybercrime has increased, organizations have been ‘shifting left’ with their security and weaving it into the beginning of the development life cycle. 

    This offers significantly increased security benefits and means software vulnerabilities can be identified and remediated before they are exposed to the enterprise. The one drawback is around the additional time this takes. As a result, many organizations are also looking to automate their DevSecOps processes. 

    Automation is a critical feature of DevSecOps as it improves efficiency, cuts costs and can streamline operations. However, to get the most out of DevSecOps automation so it delivers real value to the organization, there are several challenges that must be considered first. But what are these challenges and how can they be overcome?

    How to Approach DevOps Security Automation

    When you first move to security automation, you’ll find some security tasks that can easily be automated and others that are harder to fit in. DevSecOps teams will need to use many tools to cover all their bases, but no one wants teams checking lots of different tool outputs. Making it easier to see what’s going on and where risk lies by consolidating security tools and results in a central platform is the way to go. This makes life easier by giving a single pane of glass. Everyone generally agrees that DevOps teams must adopt cybersecurity best practices, but testing takes time. Development teams don’t have time to learn, use and check lots of security tools on top of their existing workload. Automated solutions get to work without any real need for maintenance or management. These solutions trigger tools to work at the right time depending on the outcomes found without any manual input needed. This gives you and your team valuable time back to focus on other important matters.

    Fitting DevSecOps Into Secure CI/CD Pipelines

    The goal of CI/CD pipelines is fast delivery of build and release steps, typically through automation. A key goal of DevSecOps is to alert someone to a new problem as early in that automated process as possible. You need to develop solutions that don’t overwhelm CI/CD pipelines, yet allow flexibility for differing tech stacks, security tools and environments. What you need to ask is, have the latest changes caused any new and important security issues that should be reported before going live?

    Doing DevSecOps Without Constant CI/CD Changes

    Trying to manually run security tools on your entire application source code each day can be time-consuming and hinder your ability to keep up with daily changes. For security checks to keep pace with code delivery in a CI/CD environment, security automation is a must. Security products need to integrate into the development pipeline and enable both the development and security team to work together instead of just throwing things over the fence to each other. 

    Focusing on Issues Raised for Faster Remediation

    All the artificial intelligence and machine learning (AI/ML) in the world isn’t going to remove non-issues better than a manual triage, so here we recommend being resigned to getting stuck. Treat the first run of a set of security tools as a normal security test. Expect hundreds of issues and be prepared to go through them and mark the false positives, duplicates and issues that just aren’t important right now. But do this with the expectation that you’re creating a baseline against which your future pipeline scans can be measured so you’re only alerted to new important issues. By automatically managing and recording the lists of real issues existing in every DevSecOps pipeline run, we can then concentrate on the differences. These new issues need to be flagged and then made easily visible and actionable for all teams.

    Running Multiple Security Tools in CI/CD

    DevSecOps includes setting up many automated security tools to cover the many types of security issues that need to be checked for. Many organizations run multiple security tools. However, as the number of security tools grows and the DevSecOps processes become more complex, many companies then realize the hardest challenge is to make sense of all of the test results within a short period of time. For some companies, this task of bringing together the results from all of these tools is a full-time job, and one ripe for automation. 

    Triaging Security Issues

    False positives and nonsense issues are consistent problems for using security tools. Removing these gives better results and increases the chances that development teams will have a better relationship with DevSecOps processes. Organizations should build a list of issues that don’t need to be actioned and also use automated security tools to flag new issues and differentiate them from the previous backlog of issues. Automating the ‘best fixes’ across all issues also speeds up fix times, while reducing the workload on security teams. 

    Mapping Security to Development Processes

    Dev and DevOps teams are under pressure to get pipelines running smoothly and don’t want security slowing things down or complicating it. By automating security testing in such a way that it can easily handle changes in repos and containers, organizations can overcome this challenge without compromising security. 

    Risk Prioritization

    DevSecOps will increase the number of issues found as well as the speed with which they should be handled. In reality, only a small number of issues will pose a massive risk to the business. Being able to pinpoint the really big issues in real-time helps security know the big risks are being covered and allows developers to focus on the real issues, not lots of small bugs. You can also intelligently determine the changes in risk for quick and automated decisions in the CI/CD pipeline without slowing down the development team.

  • Red Hat Brings Ansible Automation to Microsoft Azure Cloud

    Red Hat Brings Ansible Automation to Microsoft Azure Cloud

    Red Hat this week announced that a private preview of the Red Hat Ansible Automation Platform is now available on the Microsoft Azure Cloud.

    Tom Anderson, vice president of Red Hat Ansible automation, said making the Red Hat instance of the open source automation framework available on Azure is the first step toward making it possible for IT teams to use the IT automation framework across multiple clouds.

    The goal is to make it simpler for IT teams to bridge multiple IT environments using a common framework to automatically configure everything from operating systems to security platforms across a hybrid cloud computing environment.

    Ansible is emerging as a de facto standard for automating IT environments largely because it provides an IT team with a declarative approach to automating IT processes. An IT team can write a playbook that describes their desired state for a platform without having to know anything about its current state.

    Interest in IT automation has soared as the number of workloads being deployed in the cloud and on-premises IT environments have steadily increased. The simple truth is that many organizations can’t keep pace with the rate at which applications are built and deployed because they are unable to hire and retain enough IT staff. The challenge is that writing a playbook to automate any IT process requires a fair amount of expertise that many organizations lack. There are, however, now large numbers of existing templates that organizations can reuse.

    Anderson also noted that security concerns will also force the IT automation issue. Many of the application workloads running in cloud computing environments are misconfigured because they were provisioned by developers that have little to no security expertise. A playbook greatly reduces the chances an application workload will be misconfigured in the cloud without having to turn every developer into a security expert, Anderson explained.

    Ultimately, IT automation frameworks reduce the level of friction that often exists between developers and the rest of the IT operations team in a way that actually serves to increase developer productivity, added Anderson.

    Regardless of how IT organizations automate IT processes, it’s now a question of the rate at which that goal will be achieved rather than if. DevOps teams, of course, have always been at the forefront of automation, but now it’s clear that as IT environments become more complex, IT organizations are embracing automation more aggressively. Most of those initial efforts are focused on repetitive, tedious and manual IT tasks, but the potential to automate IT processes at levels of unprecedented scale has never been greater.

    In fact, many organizations may soon discover it’s difficult to retain IT professionals without investing in automation. After all, the most talented IT professionals will not want to work for an organization that requires them to always manually configure IT resources. Many IT teams might also discover that automation frameworks like Ansible are the gateway to embracing additional DevOps best practices. The challenge, of course, is finding the time to get started automating processes when there are so many immediate issues that IT teams can’t ignore.

  • Vercel Acquires Turborepo to Gain Build System

    Vercel Acquires Turborepo to Gain Build System

    Vercel today announced it has acquired Turborepo, a provider of a build system for JavaScript and TypeScript applications that provides developers with access to an easy-to-use monorepository for their code.

    In the wake of that acquisition, the command line interface (CLI) for accessing Turborepo will now be available under an open source license.

    Jared Palmer, creator of Turborepo and other popular open source projects including Formik and TSDX, will continue working on accelerating Turborepo’s capabilities as the new leader of a build performance team at Vercel.

    The goal is to move existing users of the Turborepo cloud service to a Vercel platform for building applications using the Next.js framework, which makes it possible to build web applications based on the React library for creating user interfaces that enable both static site generation (SSG) and server-side rendering (SSR).

    Palmer said monrepositories such as Turborepo not only make it easier for developers to manage builds but also reduce the amount of time required to process those builds using a continuous integration/continuous delivery (CI/CD) platform.

    Rival build systems are also much more complex to configure and manage, which Palmer said is a primary reason many developers have not yet set up their own mononrepositories.

    As the number of applications that organizations want to build and deploy in the digital business transformation era increases, the focus on developer productivity is intensifying. Rather than having to maintain a complex build system, Turborepo is designed to be simple to configure. As a result, more developers can reduce build times by as much as 50%, noted Palmer.

    In effect, front-end developers are able to more easily become full-stack developers by combining that build system with the Next.js framework, he added.

    DevOps teams will need to adjust to applications capable of providing richer application experiences without relying as much on backend infrastructure. That shift has implications for everything from the amount of network bandwidth consumed to the performance of web applications. The Vercel platform employs caching, routing and a React framework to optimize application performance in a way that reduces the dependency on backend infrastructure to optimize application performance.

    As web application development continues to evolve, the overall rate at which they web apps are built should increase significantly. Naturally, DevOps teams will need to consider the implications of having more applications move through DevOps pipelines and the need to continuously update those after they are deployed.

    It’s not clear whether or how much DevOps teams will nudge developers toward frameworks such as Next.js to develop web applications that might not tax backend infrastructure as much as previous generations of applications. In effect, developers can take advantage of frameworks that enable them to treat the underlying IT infrastructure as if it were serverless whenever possible with tools that have a familiar JavaScript construct. Vercel claimed there are already more than 30,000 sites running Next.js in production at organizations like Airbnb, Hulu, Nike, Ticketmaster and Uber.

    One way or another, it’s clear that the way web applications are being built is fundamentally changing. The only issue now is determining to what degree the rest of the IT organization is prepared to absorb that level of change.

  • Securing the Software Supply Chain with Behavioral Analysis

    Securing the Software Supply Chain with Behavioral Analysis

    Lately, software supply chains find themselves in a very interesting and uncomfortable position—the industry spotlight—and not in a good way. While significant and costly breaches such as  SolarWinds or Kaseya make front-page news, supply chain attack tactics (e.g. typosquatting or dependency confusion) that target package managers such as npm, PyPI or WinGet can poison downstream applications or compromise developer credentials.  

    The software engineering community has been steadily increasing its use of third-party and open source solutions to help create new offerings. In fact, over the last five years, there has been a 259% increase in the use of open source.

    One reason for this is the simple fact that creating new applications takes time and can strain budgets that are already tight. With open source code, software engineers gain fast access to pre-existing code that’s already demonstrated a proven ability to address the very unique challenge they are looking to solve. By removing the need to reinvent the wheel, the development timeframe is accelerated, the costs are lowered and teams are freed to focus on the creation of new and unique features that the customers demand.

    At this point, turning to third-party code may seem like a no-brainer. However, it can also open the door to potential risks. This is especially true with engineers who, rather than checking software to ensure it is secure and compliant, take the providers’ assurances as gospel. 

    They are not alone. Other groups will try to conduct their initial due diligence at the onset if that’s possible. Soon after they then ease into a “set it and forget it” approach, where they fail to update the code over time. As a result, while the code may appear secure initially, vulnerabilities can emerge over time, and if the software is not continually updated, bad actors can leverage an unprotected window of opportunity. In fact, according to Enterprise Security Group’s “Modern Application Development Security” eBook, 48% of organizations regularly push vulnerable code. The same research also found that 60 percent of organizations have had production application vulnerabilities exploited.

    Organizations conduct static application security testing (SAST) and dynamic application security testing (DAST) to assess pre-production code vulnerabilities. They will also scan with AV before releasing software, examine signing certificates, and code quality. Yet, despite these efforts, they remain challenged by gaps in security processes and the ability to sufficiently inspect large, complex and multi-layered software packages. What’s missing is the ability to conduct in-depth analysis to pinpoint behavioral differences between compiled software versions.  

    Malicious tampering can often involve changing how the software interacts with the system on which it is deployed.  Differential behavior analysis highlighted below makes it easier to detect those unusual changes. For example, having an application suddenly add a pre-install script that downloads files via certutil is highly unusual. If the addition was planned, reasons for the change would be well documented and it would be associated with a significant new feature. For this type of change to happen in any other scenario is suspicious and deserves further investigation before the software is released. In this case, such action would have caught ua-parser-js developer’s account compromise.

    supply chain

    With the software supply chain at risk and with shared code the new normal, what’s needed is software assurance. 

    Software assurance allows teams to inspect every application layer within an installation package for the presence of software tampering, malware and other indicators of software supply chain attacks.   

    Begin the software assurance process with the deep inspection step through recursive package decomposition. The resulting software bill of materials (SBOM) becomes so much more than a build manifest (a list of components that should be in the final build). It becomes the means to determine exactly which components have unacceptable build quality or those that contain unwanted software behaviors. This enables developers to verify the code they are working with and take appropriate action to protect their customers and partners from the majority of software risks. These assessments include:

    • Software grading: The software package and every included component can be graded based on the types and severity of security issues found. Using a grading system makes it easy for software buyers to gauge the level of risk posed by a software package and determine whether additional deployment measures can be taken to mitigate those risks. For developers, component grading makes it easier to prioritize their remediation efforts and aids in the selection of higher-quality third-party components.  
    • Application hardening: Activating specific build or compiler options can mitigate vulnerabilities and protect applications against attack—but only if they are implemented correctly. Assessing whether these mitigations are correctly implemented ensures that these protection mechanisms will work as expected. 
    • Binary behavior analysis: This allows teams to fully understand how each software component interacts with the system on which it will run. The ability to understand third-party component behaviors, even when developers have no access to the source code, delivers a more comprehensive understanding of supply chain risks. Checking for unusual software behaviors or unexpected behavioral changes between one version and the next is often the only way to identify software tampering by a malicious actor. The ability to have this final check before distribution enables software developers to validate that the software being released is trustworthy.

    For companies that are using software to create business value or producing software that is mission-critical for their customers, turning a blind eye or making software assurance assumptions can no longer be tolerated. In today’s climate where increasingly sophisticated threats loom, inspecting software and monitoring updates for signs of tampering, malicious or unwanted additions is essential and must become a part of standard software development practice. 

  • Iterative Adds Experiment Versioning to MLOps Platform

    Iterative Adds Experiment Versioning to MLOps Platform

    Iterative today added an experiment versioning capability to an open source platform for managing machine learning operations (MLOps) using GitOps workflows.

    Dmitry Petrov, Iterative CEO, said the latest version of the Data Versioning Control (DVC) platform makes it simpler to save, compare and reproduce machine learning (ML) experiments at scale without requiring organizations to set up an additional repository to track them. Instead, organizations can store those experiments alongside other software artifacts in a Git repository, he said.

    As a result, IT teams will no longer need to try to keep track of experiments using spreadsheets or notebook tools such as Jupyter, he noted. Organizations that are building AI models for applications constructed using DevOps workflows will also find it easier to comply with a wide range of existing and forthcoming compliance mandates, added Petrov.

    The DVC platform makes use of a Git-like interface to enable organizations to track multiple versions of data sets, models and pipelines across an MLOps workflow. That capability is being extended to make it easier to manage the experiments data science teams create during the construction of an AI model.

    Iterative is at the forefront of an emerging debate over how best to infuse AI models into applications. Providers of platforms used by data science teams are making a case for separate repositories for tracking the artifacts used to construct an AI model. Iterative, on the other hand, is arguing for the use of an AI model build framework that uses the same Git repositories where any other type of software artifact is stored and shared. Iterative’s argument is that, in effect, AI models are just another type of software artifact, noted Petrov.

    In addition to reducing the total cost of building AI models by reducing the number of platforms required, a Git-based approach also makes it simpler for DevOps and data science teams to collaborate because the DevOps team will have more visibility into what AI models will eventually need to be incorporated into an application.

    It may be a while before the development of AI models and applications fully converges. Data science teams today typically have defined their own workflow processes using a wide range of graphical tools. However, as it becomes obvious that almost every application is going to be infused with machine learning and deep learning algorithms to some degree, the need to bridge the current divide between DevOps and data science teams will become more acute.

    In the meantime, DevOps teams should assume that many more AI models are not only on the way but that those models will need to be continuously updated. Each AI model is constructed based on a set of assumptions; however, as more data becomes available, AI models are subject to drift that results in less accuracy over time. Organizations may even determine that an entire AI model needs to be replaced because the business conditions on which assumptions were made are no longer valid. One way or another, the updating and tuning of AI models is likely to soon become just another continuous process being managed via a DevOps workflow.

  • CI/CD Is Still All About Open Source

    CI/CD Is Still All About Open Source

    One of the lessons I’ve learned in my years at the helm of DevOps.com is that the heart of DevOps is CI/CD. And at the heart of CI/CD is open source. While the names have been changed to protect the innocent (and the obsolete), these lessons are as true today as they ever were. CI/CD is still all about open source.

    When we first started DevOps.com, folks said that DevOps was just a symptom; CI/CD was the cure. For many of you, that cure was delivered via open source—probably using Jenkins. Jenkins, which itself was a fork of the open source Hudson project, was created by Kohsuke Kawaguchi.

    Jenkins was a CI tool at heart and later morphed into a CI/CD tool. Many people think that this fork in the road may have hurt the continued evolution of continuous delivery in the long term. But that is an argument for another DevOps.com article (or maybe even a panel discussion at an upcoming DevOps live event).

    Regardless of where you stand on that issue, as an open source project, it is hard to argue with the success of Jenkins. Driving a lot of that success is the Jenkins plug-in architecture. There are literally thousands of plugins that allow Jenkins to work with just about anything. That is the engine that powered Jenkins, yes; but its secret superpower was and is open source.

    That said, Jenkins has grown a bit long in the tooth over the years. It’s not that it doesn’t do what it always did, it’s that what we do and how we do it has changed. Microservices, Kubernetes and even cloud have changed the very fabric of the tapestry in front of which Jenkins sits. The open source community that supports Jenkins should receive enormous credit here: It has tried mightily to keep up with the many changes. 

    Jenkins X sought to bring Jenkins to Kubernetes, for instance. But ultimately, nothing lasts forever and, as it says in the Bible, “For everything there is a season, and a time for every purpose under heaven: A time to be born, and a time to die.” Now, I am not saying Jenkins is dying, but I think its time as the leader of the pack is behind it.

    The good news is there are a host of new CI/CD tools that have risen to fill the vacuum left by Jenkins. Some of them are entirely open source; some are based on open source. Solutions such as Buildkite, Harness, Codefresh, GitLab, GitHub, CircleCI, JFrog and Tekton to name but a few. 

    Another major development has been the rise of GitOps. Weaveworks has led the way in adoption, not to mention coining of the term GitOps itself, but clearly, they are not alone. What these solutions all have in common, though, is open source.

    I used to think it was sort of a chicken-or-the-egg question when it came to CI/CD and open source. But I realize now that it is not. I firmly believe that, if not for open source, CI/CD would not be the force it is today. Make no mistake, it is truly a force. CI/CD has become the dominant method for software development.

    Of course, you could say “Why stop at CI/CD? Open source has become the dominant software development foundation in observability, OS, cloud and so much more!” Yes, I agree—we are living in the open source age. This rising tide may in fact lift all boats, including CI/CD.

    This wave is forcing all vendors to either go entirely open source or at least support open source. What’s more, for the foreseeable future, I don’t see that changing. Open source has also wrought other changes in the market. How software tools, including CI/CD, are distributed and consumed. Who is hiring enterprise software salespeople anymore? Doing large POCs? No, open source has resulted in getting software into the hands of developers and DevOps teams directly and delivering delight. If your product works, they buy it. If it doesn’t work, they won’t, either—and they will let you know, too!

    This meritocracy is another benefit of open source and is helping to speed the development of better, next-gen CI/CD tools in the market. And at the end of the day, we are all better off for that.

  • DevSecOps for Databases: Data Masking, Cloud Backup, WAF and More

    DevSecOps for Databases: Data Masking, Cloud Backup, WAF and More

    DevSecOps is everywhere. More and more organizations realize that the traditional separation between DevOps and security is wasteful and dangerous. As a result, they are rushing to integrate security into all their software development processes. 

    Adding security to containerized applications, cloud resources and other parts of modern DevOps infrastructure is already well understood. But what about databases? Databases were always—and remain—a critical system component and a prime target for attackers. Here, I’ll review the basics of modern database security and show you how to create a continuous security process for databases in a DevSecOps organization.

    What is DevSecOps?

    DevSecOps incorporates security practices into the DevOps methodology. DevSecOps promotes a ‘security-as-code’ approach via the partnership between release engineers and security teams. Similar to DevOps, the DevSecOps approach focuses on creating innovative solutions for complex software development processes. Teams complete this within the framework of an agile approach.  

    DevSecOps aims to close the gap between security and IT teams while ensuring the speedy and safe delivery of code. Organizations replace the siloed development method with a new approach that promotes the shared responsibility of security tasks during the delivery process. 

    In DevSecOps, two supposedly opposite aims—delivery speed and secure code—are united into one effective process. 

    Database Security Basics

    The term “database security” is an umbrella term used to describe all of the tools created to develop and preserve database availability, integrity and confidentiality. A key responsibility for any DBA is to ensure that the information stored in databases is always secure.

    The goal of database security is to protect:

    • Database management systems
    • Applications that access database information
    • Datastores within databases
    • Hardware servers
    • Virtual database servers, such as cloud-based servers
    • Network access points

    It is critical to strike a balance between security and usability. While you must prioritize data security, information needs to be accessible to offer value to the organization. The more access you provide to the information retained in a database, the more vulnerable the information is. However, an overly secure database is neither practical nor functional.   

    A good way to achieve a balance between accessibility and security is to create meaningful security protocols, create tiered levels of information access and invest in a robust database security and activity monitoring tooling. 

    Common Threats and Challenges

    Here are some of the most common types of database attacks:

    • Insider threats—These involve privileged users misusing their access credentials. An insider could be a negligent employee who unwittingly exposes the database to attack, a malicious insider who abuses the database on purpose or an outside infiltrator who steals credentials to breach the database. The risk of a breach increases as more employees hold privileged access credentials.
    • Injection attacks—These involve inserting arbitrary attack strings into the database queries that HTTP headers or web applications serve. These attacks can be SQL or NoSQL. Secure coding practices and regular vulnerability tests should help protect you against injection attacks.
    • Database software vulnerability exploits—Database software providers regularly release security patches for vulnerabilities in their software. However, if you don’t apply the patches quickly, you prolong your exposure and run the risk that hackers will exploit vulnerabilities in your database. 
    • Backup attacks—If you don’t protect your backup data with sufficient controls, you expose your database to an attack on your backups. 

    The following challenges can exacerbate these threats:

    • Large volumes of data—Organizations must adequately manage increasingly large volumes of data using scalable tools and practices.
    • Sprawling infrastructure sprawl—Organizations are increasingly shifting to hybrid and multi-cloud environments, making their networks more complex and security harder to manage. 
    • Evolving regulations—With the global regulatory landscape changing and growing more complex, it is increasingly challenging to maintain compliance.
    • Skill shortage—The demand for cybersecurity personnel far outstrips supply, and this shortage is only likely to increase in the future. 

    4 Techniques for Continuous Database Security With DevSecOps

    A DevSecOps approach calls for continuous database security during all stages of database development and usage. Let’s explore a few techniques for achieving continuous security.

    Data Masking

    Data masking technology lets you create a structurally similar, anonymized version of an organization’s data. You can use this data for user training or software testing. The aim is to safeguard the real data while using a functional substitute for use cases in which the actual data is not needed.  

    In data masking, you keep the format of the data the same. You only change the values of the data. You can alter the data in various ways including character shuffling, encryption and word or character substitution. Whichever method you choose, you have to modify the value in some way that makes reverse engineering or detection impossible. 

    Use data masking when presenting data to users (whether inside or outside the organization) who don’t need to access the data as part of their role. You should apply data masking to all sensitive information before displaying it in public forums. 

    Use Web Application and Database Firewalls

    Protect your database server from security threats using a firewall that stops access to traffic as a default. The only traffic permitted should come from particular web servers or applications that require access to the information. The firewall must also prevent your database from establishing outbound connections unless they are specifically needed. 

    Protecting the database using a firewall is not enough. You should also deploy a web application firewall (WAF). That’s because SQL injection attacks, which target web applications, can be employed to delete or exfiltrate data from the database. A database firewall may not stop this from occurring if attacks are initiated from an application that is a permitted source of traffic. However, a web application firewall likely will stop attacks of this nature because it can analyze traffic at the application layer and identify malicious query patterns.

    Cloud Backup

    Database backup solutions help organizations safeguard their data with backup database copies in the event of user error, physical hardware failure, data corruption or ransomware attacks. Organizations can ensure their information is always accessible by using database backup tools, even if their central database fails or is compromised. 

    Cloud backup services allow you to send a database copy to an off-site, secondary location for safekeeping if there is a disaster or failure. A third-party service provider typically hosts the secondary data storage systems and servers. This provider bills the backup customer based on capacity used or storage space, number of users, data transmission bandwidth, number of times the information is accessed or the number of servers. 

    Using cloud data backup can help you improve data protection without overwhelming IT staff. This labor-saving advantage could be significant and may even offset some of the added costs connected with cloud backup, including data transmission costs. 

    Secure Database User Access

    You must strive to limit the number of individuals with access to the database to the extreme minimum. Administrators should only have minimal privileges—those they require to carry out their job and only when they require access. 

    For small organizations, this might be difficult to achieve because employees have multiple roles, and there may be only one or two database administrators. However, you should at least manage permissions using roles or groups rather than granting access directly.  

    If you have a large organization, you must think about automating access management via access management software. Access management software can give authorized users a provisional password with the privileges they require every time they want access to the database. This software also logs the activities conducted during that time and stops administrators from sharing their passwords. 

    It is critical to lock down any form of password sharing—this is a common practice by administrators and database users, but it makes accountability and effective database security nearly impossible. 

    In addition, you should follow standard account security procedures: 

    • Enforce the use of strong passwords
    • Store, encrypt and salt password hashes
    • Lock accounts after three or four login retries
    • Deactivate accounts when employees leave or switch to different roles

    DevSecOps and Database Security

    In this article, I explained the basics of DevSecOps and database security, covered the primary threats facing database systems in a modern IT environment and provided four ways you can implement continuous security for databases:

    • Data masking—All modern RDBMS systems offer data masking mechanisms that can reduce the impact of compromised accounts.
    • Using WAFs and database firewalls—Firewalls provide excellent runtime protection and visibility for application-layer attacks.
    • Cloud backup—Backing up databases in the cloud is a continuous process that can enhance availability and reduce the impact of ransomware attacks.
    • Secure database user access—Using least-privilege principles can dramatically reduce the threat surface of a database.

    I hope this will be useful as you extend your DevSecOps practices to critical database systems in your environment.

  • API Sprawl a Looming Threat to Digital Economy

    API Sprawl a Looming Threat to Digital Economy

    New estimates say the total number of public and private APIs in use is approaching a whopping 200 million. APIs are becoming increasingly crucial to the global digital economy. They are the backbone of many digital platforms and drive the composable enterprise model. But this ubiquity presents sprawl issues.

    F5 recently released a study that examines the conditions of the API economy at large. Authored by Rajesh Narayanan and Mike Wiley of F5, the paper, Continual API Sprawl: Challenges and Opportunities in an API Driven Economy, articulates the state of API sprawl and the conditions behind its arrival.

    According to Narayanan and Wiley, “If data is the new oil, then APIs will become the new plastic.” This looming reality will require a continuous approach to API management to avoid further polluting the digital ecosystem.

    Below, I’ll review the report’s main takeaways to see what has given rise to API sprawl and consider how IT leaders should respond.

    State of API Growth

    APIs have evolved as a standard mechanism for businesses and services to connect and share value. And, as more companies begin to rely on them, the API economy is becoming big business—83% of organizations today consider API integration a critical part of their business strategy, according to the 2020 Cloud Elements State of API Integration Report.

    There is a proliferation of API styles. Thousands of public productized APIs exist, but more private and partner APIs are in use. Technically speaking, APIs come in many different forms—they may be web-based, browser-based or embedded into devices. The report also identifies single-purpose APIs and those that aggregate multiple data providers.

    API use cases are pervasive, from hotel bookings to weather, stock tickers, transportation, IoT, DevOps workflows and many other areas. “API-powered apps have permeated every aspect of our lives,” said the report. Yet, typical market estimates are usually quite conservative when sizing up this market, said F5. According to F5’s aggressive calculations, we will be approaching 1.7 billion active APIs by 2030.

    APIs present a high value from startup use cases to enterprise applications. However, a downside is that this growth is leading to sprawl, said Narayanan and Wiley.

    API sprawl is the term used to describe both the exponentially large number of APIs being created and the physical spread of the distributed infrastructure locations where the APIs are deployed.

    Factors Driving API Sprawl

    Sheer growth. Sources predict the number of developers will grow to 45 million by 2030. SlashData also estimates that 30% of developers already use APIs. As the number of APIs on the market moves into the millions, managing growth poses a significant challenge, especially without proper governance and best practices.

    Lack of standards. “The lack of a common shared model contributes to API sprawl,” the report said. API design standards do exist, yet guidelines often leave room for nuances between services. Standards have also emerged around specific industries, like financial data exchange (FDX). While helpful for a single sector, this doesn’t advance multiple industries simultaneously. A lack of standardization leads to differing versioning approaches and integration challenges.

    New development approaches. Integration requirements are forcing many new APIs to unite disparate business apps. The microservices architecture trend is also adding to sprawl, as “APIs are both northbound to interfaces via microservices and horizontally between microservices.”

    Continuous software development. Continuous development, paired with the need to connect business units with bespoke requirements, could produce multiple versions of the same API. This quickly leads to maintenance difficulties, out-of-date documentation and broken clients. The report also cites rising data creation and “everything-as-a-service” trends as harbingers of an API sprawl.

    Various computing evolutions. New computing trends are also prompting a sprawl, says the report. Business units may be operating on different on-premises or cloud environments. When this hybrid situation occurs, APIs could get dispersed over many locations and become difficult to track. Specific connections may be created to cater to new environments like edge computing and IoT devices, further increasing sprawl.

    Problems With API Sprawl

    A digital economy reliant on APIs also relies on the underlying stability and availability of these services. Yet, “APIs have a shelf life and become unsupported if ignored by the developers,” the paper acknowledged. It could be hard to maintain service reliability for APIs sprawled across a distributed cloud. In addition to reliability issues, there are many other possible repercussions of an API sprawl.

    For one, management and operation at scale become difficult. Not all APIs are based on a specification, like OpenAPI. This means that API documentation may not be as streamlined and accessible, thus hurting discoverability and onboarding. “A simple means to connect these APIs may not be possible with conventional or legacy networking approaches,” the report added.

    As developers are top consumers of APIs, a sprawl could negatively impact their integration experience. Maintaining a swelling library of API dependencies can be cumbersome. Plus, APIs are prone to evolve and version over time and when endpoints are altered without advance notice, it leads to broken clients. Even small changes in API functionality can have a huge impact since all observable API behaviors will become relied upon, whether documented or not. (This is known as Hyrum’s Law).

    But the security ramifications of API sprawl are perhaps the most troubling. A whopping 91% of enterprises experienced an API security incident in 2020. Malicious API traffic also rose by a staggering 300% in mid-2021, Salt Labs found. Increased API use will undoubtedly cause more frequent attacks.

    “Unmanaged API sprawl is a security breach waiting to happen,” said the report. APIs typically adopt API keys for authentication and authorization, but this method is prone to misuse. Credentials are often exposed or misconfigured. Bad actors can use APIs to steal loads of data and computing power. As a result, sprawl is undermining trust in API connections and responses.

    Mitigating The Sprawl

    With the growth of the API economy slated to reach the billion mark in the next ten years, figuring out secure, stable inter-cluster communication will become more critical. According to F5, existing solutions (gateways, ingress controllers, service mesh and forward and reverse proxies) are not an adequate response.

    “We believe the solution should hence involve an intermediary (proxy) device focused on solving inter-cluster connectivity, security and integration challenges.” The writers call this solution API Gateway 2.0, which they describe as a bi-directional application-level proxy. The technology exists to implement such middleware and perhaps it could help companies address sprawl issues.

    Outside of Narayanan’s and Wiley’s recommendations, there are plenty of actions API owners can take to avoid adding to the sprawl. Here are a few:

    • Treat the API as a product
    • Improve developer experience
    • Use spec-driven development
    • Ensure up-to-date documentation and code libraries
    • Use consistent endpoint naming
    • Set clear guidelines for versioning and deprecation
    • Go beyond API keys with OAuth and OpenID Connect

    For more context, check out Continual API Sprawl: Challenges and Opportunities in an API Driven Economy for free without an email gate here.

  • Is TypeScript the New JavaScript?

    Is TypeScript the New JavaScript?

    While JavaScript is frequently the language of choice for all sizes of frontend and backend applications, it’s not the only option; nor is it necessarily the most efficient or cost-effective. Increasingly, TypeScript is becoming the go-to language for app development—particularly for larger apps. The time- and cost-saving benefits are significant enough that some organizations are even taking projects initially started in JavaScript and migrating them to TypeScript. 

    What is TypeScript?

    While the number of programs written in JavaScript has grown exponentially, the programming language’s ability to express the relationships between different units of code and mitigate coding errors early on hasn’t kept pace. Along with JavaScript’s inconsistent semantics, this makes JavaScript-driven app development difficult to manage at scale.

    Released in 2012, TypeScript was created to address JavaScript’s deficiencies in developing large-scale apps. It’s an open source, strongly typed programming language that builds on JavaScript by adding optional static typing. Types enable structuring and validating code before it’s executed, which is beneficial when developing large apps. They also provide additional information about code, which serves as better documentation for other developers and facilitates collaboration.

    TypeScript is a superset of JavaScript, which means any JS code is also valid TS code—provided the TS configuration is set to be compatible with it. It outputs code in pure JavaScript and allows developers to liberally use JS libraries, tools and frameworks. It runs on Node.js or any browser that supports ECMAScript 3 or higher. It also supports object-oriented programming features. 

    JavaScript Drawbacks

    JavaScript is one of the world’s most popular programming languages and works extremely well for small projects. However, its disadvantages—many of which affect project costs and code delivery time—make a strong case for considering alternatives such as TypeScript, particularly for larger projects.

    Most of these drawbacks have to do with the lack of types and compile-time error checks, making JavaScript less than desirable for server-side code in enterprises and large codebases. The following are just a few of its disadvantages:

    • JavaScript employs dynamic typing, which means scripts can compile even if they contain errors that can prevent them from properly running.
    • Because JavaScript is dynamically typed, it allows one variable to have multiple properties assigned to it. It doesn’t instantly let developers know what a variable can contain. That makes it easy to assign the wrong properties.
    • Since JavaScript is an interpreted language, errors can only be found during runtime. Code needs to run before being tested and validated, so it can take considerable time to find bugs and errors in JavaScript code.
    • To eliminate inaccuracies, it’s necessary to manually verify types and the syntactic correctness of the code. This lengthens development time and extends the delivery cycle to production, increasing development costs.
    • JavaScript code is executed on the client side so it’s viewable to the user. As a result, bugs and oversights have the potential to be exploited for malicious purposes. 
    • Lengthy onboarding of new developers can result because developers must figure out the properties of the structures they’re working with, as well as the data types. While JSDoc can be used for documenting code and annotating types, there’s still the need to synchronize the actual code and documentation. The lack of synchronization, in turn, can mislead developers and complicate the introduction of new, business-requested features.
    • JavaScript is prototype-based, not class-based. It’s not considered a pure object-oriented programming language, although it can follow some object-oriented programming principles.

    The TypeScript Advantage

    TypeScript is a superset of JavaScript. If a developer knows JavaScript, there’s not much of a learning curve to take advantage of the features TypeScript offers that make up for many of JavaScript’s deficiencies—and it can offer additional benefits.

    For example, with JavaScript, variables can start as one property, and then change into an object or a string. These inconsistencies can generate problems that are difficult to resolve in large apps. TypeScript, on the other hand, analyzes the code and tries to determine proper variable types prior to runtime. Once a variable type is assigned, it stays unchanged. TypeScript’s compiler also helps shorten the QA and testing process in later stages of development.

    TypeScript also helps developers quickly figure out the purpose of a variable within the code. It can also suggest available properties in functions, classes or components. Being able to quickly look up a variable is important because it reduces the likelihood of calling the wrong function or accidentally skipping a variable declaration. Any reduction in bugs and errors reduces the time required for fixing those issues and, therefore, reduces overall development time. That gives developers more time to work on app logic and fix errors that can be detrimental to in-app performance and usability. According to a postmortem analysis by Airbnb, 38% of bugs they encountered were preventable with TypeScript after the company adopted it throughout the organization. 

    As a static language, TypeScript performs type checks upon compilation, flagging type errors and helping developers spot mistakes early on in development. Reducing errors when working with large codebases can save hours of development time.

    Clear and readable code is easy to maintain, even for newly onboarded developers. Because TypeScript calls for assigning types, the code instantly becomes easier to work with and understand. In essence, TypeScript code is self-documenting, allowing distributed teams to work much more efficiently. Teams (and individual developers) don’t have to spend inordinate amounts of time familiarizing themselves with a project.

    TypeScript’s integration with editors also makes it much easier to validate the code thanks to context-aware suggestions. TypeScript can determine what methods and properties can be assigned to specific objects, and these suggestions tend to increase developer productivity.

    TypeScript is widely used to automate the deployment of infrastructure and CI/CD pipelines for backend and web applications. Moreover, the client part (for example, when using Angular) and the backend can be written in the same language—TypeScript. This flexibility allows an engineer who knows one programming language to cover all parts of the system.

    Because TypeScript essentially transpiles to JavaScript, migrating existing code to TypeScript is easy and fast. It can typically be accomplished simply by running the compiler and adding typing where it’s not recognized by the language. There’s no need to make changes to the code.

    The benefits of TypeScript have been noticed by developers. TypeScript was used by 78% of the 2020 State of JS respondents, with 93% saying they would use it again. TypeScript continues to grow in popularity. It was voted the second most-loved programming language in the Stack Overflow 2020 Developer survey. 

    TypeScript & AWS: Better Together 

    There’s another compelling reason to consider TypeScript for large-scale app development projects. It’s fully supported by AWS, the leading cloud platform for modern app design and development as noted in Gartner’s Magic Quadrant for Cloud Infrastructure and Platform Services.

    The following services and tools have seamless integration out-of-the-box with TypeScript, enabling you to use this language for a vast number of tasks:

    • AWS CDK provides infrastructure-as-code (IaC) to automatically deploy the entire infrastructure in one click. You can also automate the creation of CI/CD pipelines for the future launch of specific jobs on demand.
    • AWS Lambda allows you to run computation tasks in serverless mode using automatic scaling and an efficient pricing scheme.
    • Amazon EC2, ECS and EKS can cover any of your tasks and provide a wide range of solutions for running apps written in TypeScript—from bare metal servers to complex clusters of hundreds of Docker containers.

    Optimize Your Application Development Projects

    All decisions made during an application development project can impact overall costs and time to market. That includes going with the right programming language and using the most appropriate cloud platform and resources. 

    When approaching an app dev project, a one-size-fits-all approach rarely works. Take the time to define your needs and priorities and select the best resources to deliver the best results.