devopsy.kinsta.cloud

Category: Blogs

  • Securing the Software Supply Chain with Behavioral Analysis

    Securing the Software Supply Chain with Behavioral Analysis

    Lately, software supply chains find themselves in a very interesting and uncomfortable position—the industry spotlight—and not in a good way. While significant and costly breaches such as  SolarWinds or Kaseya make front-page news, supply chain attack tactics (e.g. typosquatting or dependency confusion) that target package managers such as npm, PyPI or WinGet can poison downstream applications or compromise developer credentials.  

    The software engineering community has been steadily increasing its use of third-party and open source solutions to help create new offerings. In fact, over the last five years, there has been a 259% increase in the use of open source.

    One reason for this is the simple fact that creating new applications takes time and can strain budgets that are already tight. With open source code, software engineers gain fast access to pre-existing code that’s already demonstrated a proven ability to address the very unique challenge they are looking to solve. By removing the need to reinvent the wheel, the development timeframe is accelerated, the costs are lowered and teams are freed to focus on the creation of new and unique features that the customers demand.

    At this point, turning to third-party code may seem like a no-brainer. However, it can also open the door to potential risks. This is especially true with engineers who, rather than checking software to ensure it is secure and compliant, take the providers’ assurances as gospel. 

    They are not alone. Other groups will try to conduct their initial due diligence at the onset if that’s possible. Soon after they then ease into a “set it and forget it” approach, where they fail to update the code over time. As a result, while the code may appear secure initially, vulnerabilities can emerge over time, and if the software is not continually updated, bad actors can leverage an unprotected window of opportunity. In fact, according to Enterprise Security Group’s “Modern Application Development Security” eBook, 48% of organizations regularly push vulnerable code. The same research also found that 60 percent of organizations have had production application vulnerabilities exploited.

    Organizations conduct static application security testing (SAST) and dynamic application security testing (DAST) to assess pre-production code vulnerabilities. They will also scan with AV before releasing software, examine signing certificates, and code quality. Yet, despite these efforts, they remain challenged by gaps in security processes and the ability to sufficiently inspect large, complex and multi-layered software packages. What’s missing is the ability to conduct in-depth analysis to pinpoint behavioral differences between compiled software versions.  

    Malicious tampering can often involve changing how the software interacts with the system on which it is deployed.  Differential behavior analysis highlighted below makes it easier to detect those unusual changes. For example, having an application suddenly add a pre-install script that downloads files via certutil is highly unusual. If the addition was planned, reasons for the change would be well documented and it would be associated with a significant new feature. For this type of change to happen in any other scenario is suspicious and deserves further investigation before the software is released. In this case, such action would have caught ua-parser-js developer’s account compromise.

    supply chain

    With the software supply chain at risk and with shared code the new normal, what’s needed is software assurance. 

    Software assurance allows teams to inspect every application layer within an installation package for the presence of software tampering, malware and other indicators of software supply chain attacks.   

    Begin the software assurance process with the deep inspection step through recursive package decomposition. The resulting software bill of materials (SBOM) becomes so much more than a build manifest (a list of components that should be in the final build). It becomes the means to determine exactly which components have unacceptable build quality or those that contain unwanted software behaviors. This enables developers to verify the code they are working with and take appropriate action to protect their customers and partners from the majority of software risks. These assessments include:

    • Software grading: The software package and every included component can be graded based on the types and severity of security issues found. Using a grading system makes it easy for software buyers to gauge the level of risk posed by a software package and determine whether additional deployment measures can be taken to mitigate those risks. For developers, component grading makes it easier to prioritize their remediation efforts and aids in the selection of higher-quality third-party components.  
    • Application hardening: Activating specific build or compiler options can mitigate vulnerabilities and protect applications against attack—but only if they are implemented correctly. Assessing whether these mitigations are correctly implemented ensures that these protection mechanisms will work as expected. 
    • Binary behavior analysis: This allows teams to fully understand how each software component interacts with the system on which it will run. The ability to understand third-party component behaviors, even when developers have no access to the source code, delivers a more comprehensive understanding of supply chain risks. Checking for unusual software behaviors or unexpected behavioral changes between one version and the next is often the only way to identify software tampering by a malicious actor. The ability to have this final check before distribution enables software developers to validate that the software being released is trustworthy.

    For companies that are using software to create business value or producing software that is mission-critical for their customers, turning a blind eye or making software assurance assumptions can no longer be tolerated. In today’s climate where increasingly sophisticated threats loom, inspecting software and monitoring updates for signs of tampering, malicious or unwanted additions is essential and must become a part of standard software development practice. 

  • Iterative Adds Experiment Versioning to MLOps Platform

    Iterative Adds Experiment Versioning to MLOps Platform

    Iterative today added an experiment versioning capability to an open source platform for managing machine learning operations (MLOps) using GitOps workflows.

    Dmitry Petrov, Iterative CEO, said the latest version of the Data Versioning Control (DVC) platform makes it simpler to save, compare and reproduce machine learning (ML) experiments at scale without requiring organizations to set up an additional repository to track them. Instead, organizations can store those experiments alongside other software artifacts in a Git repository, he said.

    As a result, IT teams will no longer need to try to keep track of experiments using spreadsheets or notebook tools such as Jupyter, he noted. Organizations that are building AI models for applications constructed using DevOps workflows will also find it easier to comply with a wide range of existing and forthcoming compliance mandates, added Petrov.

    The DVC platform makes use of a Git-like interface to enable organizations to track multiple versions of data sets, models and pipelines across an MLOps workflow. That capability is being extended to make it easier to manage the experiments data science teams create during the construction of an AI model.

    Iterative is at the forefront of an emerging debate over how best to infuse AI models into applications. Providers of platforms used by data science teams are making a case for separate repositories for tracking the artifacts used to construct an AI model. Iterative, on the other hand, is arguing for the use of an AI model build framework that uses the same Git repositories where any other type of software artifact is stored and shared. Iterative’s argument is that, in effect, AI models are just another type of software artifact, noted Petrov.

    In addition to reducing the total cost of building AI models by reducing the number of platforms required, a Git-based approach also makes it simpler for DevOps and data science teams to collaborate because the DevOps team will have more visibility into what AI models will eventually need to be incorporated into an application.

    It may be a while before the development of AI models and applications fully converges. Data science teams today typically have defined their own workflow processes using a wide range of graphical tools. However, as it becomes obvious that almost every application is going to be infused with machine learning and deep learning algorithms to some degree, the need to bridge the current divide between DevOps and data science teams will become more acute.

    In the meantime, DevOps teams should assume that many more AI models are not only on the way but that those models will need to be continuously updated. Each AI model is constructed based on a set of assumptions; however, as more data becomes available, AI models are subject to drift that results in less accuracy over time. Organizations may even determine that an entire AI model needs to be replaced because the business conditions on which assumptions were made are no longer valid. One way or another, the updating and tuning of AI models is likely to soon become just another continuous process being managed via a DevOps workflow.

  • CI/CD Is Still All About Open Source

    CI/CD Is Still All About Open Source

    One of the lessons I’ve learned in my years at the helm of DevOps.com is that the heart of DevOps is CI/CD. And at the heart of CI/CD is open source. While the names have been changed to protect the innocent (and the obsolete), these lessons are as true today as they ever were. CI/CD is still all about open source.

    When we first started DevOps.com, folks said that DevOps was just a symptom; CI/CD was the cure. For many of you, that cure was delivered via open source—probably using Jenkins. Jenkins, which itself was a fork of the open source Hudson project, was created by Kohsuke Kawaguchi.

    Jenkins was a CI tool at heart and later morphed into a CI/CD tool. Many people think that this fork in the road may have hurt the continued evolution of continuous delivery in the long term. But that is an argument for another DevOps.com article (or maybe even a panel discussion at an upcoming DevOps live event).

    Regardless of where you stand on that issue, as an open source project, it is hard to argue with the success of Jenkins. Driving a lot of that success is the Jenkins plug-in architecture. There are literally thousands of plugins that allow Jenkins to work with just about anything. That is the engine that powered Jenkins, yes; but its secret superpower was and is open source.

    That said, Jenkins has grown a bit long in the tooth over the years. It’s not that it doesn’t do what it always did, it’s that what we do and how we do it has changed. Microservices, Kubernetes and even cloud have changed the very fabric of the tapestry in front of which Jenkins sits. The open source community that supports Jenkins should receive enormous credit here: It has tried mightily to keep up with the many changes. 

    Jenkins X sought to bring Jenkins to Kubernetes, for instance. But ultimately, nothing lasts forever and, as it says in the Bible, “For everything there is a season, and a time for every purpose under heaven: A time to be born, and a time to die.” Now, I am not saying Jenkins is dying, but I think its time as the leader of the pack is behind it.

    The good news is there are a host of new CI/CD tools that have risen to fill the vacuum left by Jenkins. Some of them are entirely open source; some are based on open source. Solutions such as Buildkite, Harness, Codefresh, GitLab, GitHub, CircleCI, JFrog and Tekton to name but a few. 

    Another major development has been the rise of GitOps. Weaveworks has led the way in adoption, not to mention coining of the term GitOps itself, but clearly, they are not alone. What these solutions all have in common, though, is open source.

    I used to think it was sort of a chicken-or-the-egg question when it came to CI/CD and open source. But I realize now that it is not. I firmly believe that, if not for open source, CI/CD would not be the force it is today. Make no mistake, it is truly a force. CI/CD has become the dominant method for software development.

    Of course, you could say “Why stop at CI/CD? Open source has become the dominant software development foundation in observability, OS, cloud and so much more!” Yes, I agree—we are living in the open source age. This rising tide may in fact lift all boats, including CI/CD.

    This wave is forcing all vendors to either go entirely open source or at least support open source. What’s more, for the foreseeable future, I don’t see that changing. Open source has also wrought other changes in the market. How software tools, including CI/CD, are distributed and consumed. Who is hiring enterprise software salespeople anymore? Doing large POCs? No, open source has resulted in getting software into the hands of developers and DevOps teams directly and delivering delight. If your product works, they buy it. If it doesn’t work, they won’t, either—and they will let you know, too!

    This meritocracy is another benefit of open source and is helping to speed the development of better, next-gen CI/CD tools in the market. And at the end of the day, we are all better off for that.

  • DevSecOps for Databases: Data Masking, Cloud Backup, WAF and More

    DevSecOps for Databases: Data Masking, Cloud Backup, WAF and More

    DevSecOps is everywhere. More and more organizations realize that the traditional separation between DevOps and security is wasteful and dangerous. As a result, they are rushing to integrate security into all their software development processes. 

    Adding security to containerized applications, cloud resources and other parts of modern DevOps infrastructure is already well understood. But what about databases? Databases were always—and remain—a critical system component and a prime target for attackers. Here, I’ll review the basics of modern database security and show you how to create a continuous security process for databases in a DevSecOps organization.

    What is DevSecOps?

    DevSecOps incorporates security practices into the DevOps methodology. DevSecOps promotes a ‘security-as-code’ approach via the partnership between release engineers and security teams. Similar to DevOps, the DevSecOps approach focuses on creating innovative solutions for complex software development processes. Teams complete this within the framework of an agile approach.  

    DevSecOps aims to close the gap between security and IT teams while ensuring the speedy and safe delivery of code. Organizations replace the siloed development method with a new approach that promotes the shared responsibility of security tasks during the delivery process. 

    In DevSecOps, two supposedly opposite aims—delivery speed and secure code—are united into one effective process. 

    Database Security Basics

    The term “database security” is an umbrella term used to describe all of the tools created to develop and preserve database availability, integrity and confidentiality. A key responsibility for any DBA is to ensure that the information stored in databases is always secure.

    The goal of database security is to protect:

    • Database management systems
    • Applications that access database information
    • Datastores within databases
    • Hardware servers
    • Virtual database servers, such as cloud-based servers
    • Network access points

    It is critical to strike a balance between security and usability. While you must prioritize data security, information needs to be accessible to offer value to the organization. The more access you provide to the information retained in a database, the more vulnerable the information is. However, an overly secure database is neither practical nor functional.   

    A good way to achieve a balance between accessibility and security is to create meaningful security protocols, create tiered levels of information access and invest in a robust database security and activity monitoring tooling. 

    Common Threats and Challenges

    Here are some of the most common types of database attacks:

    • Insider threats—These involve privileged users misusing their access credentials. An insider could be a negligent employee who unwittingly exposes the database to attack, a malicious insider who abuses the database on purpose or an outside infiltrator who steals credentials to breach the database. The risk of a breach increases as more employees hold privileged access credentials.
    • Injection attacks—These involve inserting arbitrary attack strings into the database queries that HTTP headers or web applications serve. These attacks can be SQL or NoSQL. Secure coding practices and regular vulnerability tests should help protect you against injection attacks.
    • Database software vulnerability exploits—Database software providers regularly release security patches for vulnerabilities in their software. However, if you don’t apply the patches quickly, you prolong your exposure and run the risk that hackers will exploit vulnerabilities in your database. 
    • Backup attacks—If you don’t protect your backup data with sufficient controls, you expose your database to an attack on your backups. 

    The following challenges can exacerbate these threats:

    • Large volumes of data—Organizations must adequately manage increasingly large volumes of data using scalable tools and practices.
    • Sprawling infrastructure sprawl—Organizations are increasingly shifting to hybrid and multi-cloud environments, making their networks more complex and security harder to manage. 
    • Evolving regulations—With the global regulatory landscape changing and growing more complex, it is increasingly challenging to maintain compliance.
    • Skill shortage—The demand for cybersecurity personnel far outstrips supply, and this shortage is only likely to increase in the future. 

    4 Techniques for Continuous Database Security With DevSecOps

    A DevSecOps approach calls for continuous database security during all stages of database development and usage. Let’s explore a few techniques for achieving continuous security.

    Data Masking

    Data masking technology lets you create a structurally similar, anonymized version of an organization’s data. You can use this data for user training or software testing. The aim is to safeguard the real data while using a functional substitute for use cases in which the actual data is not needed.  

    In data masking, you keep the format of the data the same. You only change the values of the data. You can alter the data in various ways including character shuffling, encryption and word or character substitution. Whichever method you choose, you have to modify the value in some way that makes reverse engineering or detection impossible. 

    Use data masking when presenting data to users (whether inside or outside the organization) who don’t need to access the data as part of their role. You should apply data masking to all sensitive information before displaying it in public forums. 

    Use Web Application and Database Firewalls

    Protect your database server from security threats using a firewall that stops access to traffic as a default. The only traffic permitted should come from particular web servers or applications that require access to the information. The firewall must also prevent your database from establishing outbound connections unless they are specifically needed. 

    Protecting the database using a firewall is not enough. You should also deploy a web application firewall (WAF). That’s because SQL injection attacks, which target web applications, can be employed to delete or exfiltrate data from the database. A database firewall may not stop this from occurring if attacks are initiated from an application that is a permitted source of traffic. However, a web application firewall likely will stop attacks of this nature because it can analyze traffic at the application layer and identify malicious query patterns.

    Cloud Backup

    Database backup solutions help organizations safeguard their data with backup database copies in the event of user error, physical hardware failure, data corruption or ransomware attacks. Organizations can ensure their information is always accessible by using database backup tools, even if their central database fails or is compromised. 

    Cloud backup services allow you to send a database copy to an off-site, secondary location for safekeeping if there is a disaster or failure. A third-party service provider typically hosts the secondary data storage systems and servers. This provider bills the backup customer based on capacity used or storage space, number of users, data transmission bandwidth, number of times the information is accessed or the number of servers. 

    Using cloud data backup can help you improve data protection without overwhelming IT staff. This labor-saving advantage could be significant and may even offset some of the added costs connected with cloud backup, including data transmission costs. 

    Secure Database User Access

    You must strive to limit the number of individuals with access to the database to the extreme minimum. Administrators should only have minimal privileges—those they require to carry out their job and only when they require access. 

    For small organizations, this might be difficult to achieve because employees have multiple roles, and there may be only one or two database administrators. However, you should at least manage permissions using roles or groups rather than granting access directly.  

    If you have a large organization, you must think about automating access management via access management software. Access management software can give authorized users a provisional password with the privileges they require every time they want access to the database. This software also logs the activities conducted during that time and stops administrators from sharing their passwords. 

    It is critical to lock down any form of password sharing—this is a common practice by administrators and database users, but it makes accountability and effective database security nearly impossible. 

    In addition, you should follow standard account security procedures: 

    • Enforce the use of strong passwords
    • Store, encrypt and salt password hashes
    • Lock accounts after three or four login retries
    • Deactivate accounts when employees leave or switch to different roles

    DevSecOps and Database Security

    In this article, I explained the basics of DevSecOps and database security, covered the primary threats facing database systems in a modern IT environment and provided four ways you can implement continuous security for databases:

    • Data masking—All modern RDBMS systems offer data masking mechanisms that can reduce the impact of compromised accounts.
    • Using WAFs and database firewalls—Firewalls provide excellent runtime protection and visibility for application-layer attacks.
    • Cloud backup—Backing up databases in the cloud is a continuous process that can enhance availability and reduce the impact of ransomware attacks.
    • Secure database user access—Using least-privilege principles can dramatically reduce the threat surface of a database.

    I hope this will be useful as you extend your DevSecOps practices to critical database systems in your environment.

  • API Sprawl a Looming Threat to Digital Economy

    API Sprawl a Looming Threat to Digital Economy

    New estimates say the total number of public and private APIs in use is approaching a whopping 200 million. APIs are becoming increasingly crucial to the global digital economy. They are the backbone of many digital platforms and drive the composable enterprise model. But this ubiquity presents sprawl issues.

    F5 recently released a study that examines the conditions of the API economy at large. Authored by Rajesh Narayanan and Mike Wiley of F5, the paper, Continual API Sprawl: Challenges and Opportunities in an API Driven Economy, articulates the state of API sprawl and the conditions behind its arrival.

    According to Narayanan and Wiley, “If data is the new oil, then APIs will become the new plastic.” This looming reality will require a continuous approach to API management to avoid further polluting the digital ecosystem.

    Below, I’ll review the report’s main takeaways to see what has given rise to API sprawl and consider how IT leaders should respond.

    State of API Growth

    APIs have evolved as a standard mechanism for businesses and services to connect and share value. And, as more companies begin to rely on them, the API economy is becoming big business—83% of organizations today consider API integration a critical part of their business strategy, according to the 2020 Cloud Elements State of API Integration Report.

    There is a proliferation of API styles. Thousands of public productized APIs exist, but more private and partner APIs are in use. Technically speaking, APIs come in many different forms—they may be web-based, browser-based or embedded into devices. The report also identifies single-purpose APIs and those that aggregate multiple data providers.

    API use cases are pervasive, from hotel bookings to weather, stock tickers, transportation, IoT, DevOps workflows and many other areas. “API-powered apps have permeated every aspect of our lives,” said the report. Yet, typical market estimates are usually quite conservative when sizing up this market, said F5. According to F5’s aggressive calculations, we will be approaching 1.7 billion active APIs by 2030.

    APIs present a high value from startup use cases to enterprise applications. However, a downside is that this growth is leading to sprawl, said Narayanan and Wiley.

    API sprawl is the term used to describe both the exponentially large number of APIs being created and the physical spread of the distributed infrastructure locations where the APIs are deployed.

    Factors Driving API Sprawl

    Sheer growth. Sources predict the number of developers will grow to 45 million by 2030. SlashData also estimates that 30% of developers already use APIs. As the number of APIs on the market moves into the millions, managing growth poses a significant challenge, especially without proper governance and best practices.

    Lack of standards. “The lack of a common shared model contributes to API sprawl,” the report said. API design standards do exist, yet guidelines often leave room for nuances between services. Standards have also emerged around specific industries, like financial data exchange (FDX). While helpful for a single sector, this doesn’t advance multiple industries simultaneously. A lack of standardization leads to differing versioning approaches and integration challenges.

    New development approaches. Integration requirements are forcing many new APIs to unite disparate business apps. The microservices architecture trend is also adding to sprawl, as “APIs are both northbound to interfaces via microservices and horizontally between microservices.”

    Continuous software development. Continuous development, paired with the need to connect business units with bespoke requirements, could produce multiple versions of the same API. This quickly leads to maintenance difficulties, out-of-date documentation and broken clients. The report also cites rising data creation and “everything-as-a-service” trends as harbingers of an API sprawl.

    Various computing evolutions. New computing trends are also prompting a sprawl, says the report. Business units may be operating on different on-premises or cloud environments. When this hybrid situation occurs, APIs could get dispersed over many locations and become difficult to track. Specific connections may be created to cater to new environments like edge computing and IoT devices, further increasing sprawl.

    Problems With API Sprawl

    A digital economy reliant on APIs also relies on the underlying stability and availability of these services. Yet, “APIs have a shelf life and become unsupported if ignored by the developers,” the paper acknowledged. It could be hard to maintain service reliability for APIs sprawled across a distributed cloud. In addition to reliability issues, there are many other possible repercussions of an API sprawl.

    For one, management and operation at scale become difficult. Not all APIs are based on a specification, like OpenAPI. This means that API documentation may not be as streamlined and accessible, thus hurting discoverability and onboarding. “A simple means to connect these APIs may not be possible with conventional or legacy networking approaches,” the report added.

    As developers are top consumers of APIs, a sprawl could negatively impact their integration experience. Maintaining a swelling library of API dependencies can be cumbersome. Plus, APIs are prone to evolve and version over time and when endpoints are altered without advance notice, it leads to broken clients. Even small changes in API functionality can have a huge impact since all observable API behaviors will become relied upon, whether documented or not. (This is known as Hyrum’s Law).

    But the security ramifications of API sprawl are perhaps the most troubling. A whopping 91% of enterprises experienced an API security incident in 2020. Malicious API traffic also rose by a staggering 300% in mid-2021, Salt Labs found. Increased API use will undoubtedly cause more frequent attacks.

    “Unmanaged API sprawl is a security breach waiting to happen,” said the report. APIs typically adopt API keys for authentication and authorization, but this method is prone to misuse. Credentials are often exposed or misconfigured. Bad actors can use APIs to steal loads of data and computing power. As a result, sprawl is undermining trust in API connections and responses.

    Mitigating The Sprawl

    With the growth of the API economy slated to reach the billion mark in the next ten years, figuring out secure, stable inter-cluster communication will become more critical. According to F5, existing solutions (gateways, ingress controllers, service mesh and forward and reverse proxies) are not an adequate response.

    “We believe the solution should hence involve an intermediary (proxy) device focused on solving inter-cluster connectivity, security and integration challenges.” The writers call this solution API Gateway 2.0, which they describe as a bi-directional application-level proxy. The technology exists to implement such middleware and perhaps it could help companies address sprawl issues.

    Outside of Narayanan’s and Wiley’s recommendations, there are plenty of actions API owners can take to avoid adding to the sprawl. Here are a few:

    • Treat the API as a product
    • Improve developer experience
    • Use spec-driven development
    • Ensure up-to-date documentation and code libraries
    • Use consistent endpoint naming
    • Set clear guidelines for versioning and deprecation
    • Go beyond API keys with OAuth and OpenID Connect

    For more context, check out Continual API Sprawl: Challenges and Opportunities in an API Driven Economy for free without an email gate here.

  • ArgoCon ’21: Get More Done with Kubernetes

    ArgoCon ’21: Get More Done with Kubernetes

    Argo is an open source suite of projects that helps DevOps teams deliver software more rapidly and safely. Argo Workflows, Argo CD, Argo Rollouts and Argo Events are open source tools for Kubernetes to run workflows, manage clusters and do GitOps right. On Dec. 8, the Argo community will come together at ArgoCon ’21 Virtual Event to share their Argo journey, some less known details about the project and offer a glimpse into what the future may hold.

    The agenda for ArgoCon ’21 features technical sessions, demos, deep-dives and case studies covering a variety of topics, including:

    • How GitOps workflows relate to DevOps movement and how it paves the way toward a successful DevOps adoption
    • How to use Argo Workflows, Events, CD and Rollouts to create a flexible, customizable developer experience for CI/CD
    • Argo CD production best practices and how to customize your Argo CD setup
    • Building medical grade AI with Argo Workflows
    • Bridging into Python ecosystem with cloud-native distributed ML pipelines
    • How to enable and empower teams to manage their own Argo-based CI/CD pipelines across multiple multi-tenant clusters
    • How scalable is Argo
    • GitOps-ifying your K8s management with Argo CD and Codefresh

    Speakers

    ArgoCon ’21 features outstanding DevOps and GitOps leaders from the Argo community, including:

    • Alex Collins, Argo project lead at Intuit
    • Alexander Matyushentsev, Argo project lead at Intuit
    • Anaïs Urlichs, SRE at Civo
    • Aya Ivtsan, senior computer scientist at Adobe
    • Christian Hernandez, senior principal technical marketing manager at Red Hat
    • Dan Garfield, co-founder and chief open source officer at Codefresh
    • Edgar Magana, senior principal software engineer at Splunk
    • Hannah Seligson, developer advocate at Codefresh
    • Hong Wang, founder and CEO at Akuity
    • Hui Kang, platform engineer at Salesforce
    • Ian McGraw, lead platform engineer at Arthur.ai
    • Jesse Suen, Argo project lead, co-founder and CTO at Akuity
    • J.P. Zivalich, CTO at Pipekit
    • Kelsey Hightower, principal engineer of Google Cloud Platform at Google Cloud
    • Mike Bowen, director open source program office, cloud native engineer at BlackRock
    • Omri Fima, chief architect at Diagnostic Robotics
    • Pratik Wadher, SVP, product development at Intuit
    • Ravi Lachhman, field CTO at Shipa
    • Ryan Umstead, BlackRock
    • Saad Malik, CTO and co-founder at Spectro Cloud
    • Sergio Méndez, DevOps engineer at Yalo
    • Severin Ryberg, senior infrastructure architect at ACCURE Battery Intelligence
    • Siamak Sadeghianfar, product manager at Red Hat
    • Venkata Gunapati, staff software engineer at Intuit
    • Yuan Tang, founding engineer at Akuity

    Giveaways

    There will be six sponsor giveaways at ArgoCon ’21:

    • A mouse pad and wireless charger giveaway by Red Hat
    • A $100 Amazon gift card raffle by Akuity
    • The Kubernetes 2021: Operationalizing for Scale study by Dimensional Research (sponsored by Spectro Cloud) is freely available for download
    • A $200 gift card raffle by Shipa
    • A brand new Quest 2 and Pipekit swag package giveaway
    • 15 Argo Codefresh branded Bluetooth speakers giveaway

    For more information on how to participate on the raffles and giveaways, visit the ArgoCon ’21 website.

    There will also be a virtual exhibit hall to interact with partner sponsors and download information. All attendees will receive a link to access the virtual environment on-demand once the live event is over. To see the full agenda and to register, please visit the ArgoCon ’21 website.

  • 22 Engineering Leaders Weigh in on APIs

    22 Engineering Leaders Weigh in on APIs

    Every business today is a digital business … and if it’s not, it will be soon.

    A recent survey by Deloitte found that digitally mature companies are better able to navigate rapid change and, as a result, perform much better financially.  Nearly two-thirds of survey respondents asserted that organizations that don’t digitize in the next five years will be “doomed.”

    Application programming interfaces (APIs) are the cornerstone of these digital transformation efforts. APIs aren’t just technical conduits that let applications communicate with each other. In a software-driven world, they allow developers to build services more quickly, enable innovation, open up new markets and drive competitive advantage.

    A new State of the API report from Postman, which surveyed 28,000 developers and other tech pros, offers a vivid snapshot of APIs’ central role in modern organizations. It showed that API-fueled innovation has helped companies respond to the pandemic; that the global API ecosystem is growing; that developers are spending more time than ever with APIs and that almost everyone believes investment of time and resources into APIs will increase or stay the same in their organizations for the next 12 months.

    To learn more about APIs’ status in the enterprise, I’ve spent part of 2021 interviewing 22 engineering leaders from various companies for my podcast, Breaking Changes. I wanted to dive deeper into how and why these leaders view APIs as critical to business strategy and hear what lessons they’d share with others.

    Here, in 20 bite-sized chunks, are the recurring themes:

    Business. The greatest value generated from API operations is rarely directly from any single API, but around the partner enablement APIs make possible. Over and over, companies see that building an API ecosystem helps extend their business’s reach and deepens the connection with partners and customers.

    Strategy. Without proper planning, an organization will always be stuck in reactive mode when it comes to the applications that API operations are powering. It’s much better to have a comprehensive API strategy.

    Third-party relationships. Reliance on third-party systems—the vast supply chain of partners and vendors that organizations share data with via cloud-based technology—is exponentially increasing, pushing organizations to develop a more formal and external API strategy.

    Resources. The resources surrounding API operations need to speak to developers, support them and help make their lives easier as they work with a company’s APIs.

    Governance. It’s important to be cautious in automating governance at the pipeline level before developing relationships across teams and a centralized API governance approach is vital, too. API governance is the practice of applying common rules and security policies for APIs. 

    Management. The management of APIs is a continuous journey and not a destination, requiring a significant amount of planning and attention to reach desired objectives.

    Global. We are all increasingly reaching a global audience with APIs, and every organization’s message and resources need to reflect the increasingly diverse audience.

    Multi-Protocol. The future of APIs is multi-protocol, requiring a diverse toolbox of API protocols and patterns to properly grow ecosystems over the next decade.

    Enablement. Companies need to be enabling their API development teams, not governing them; making sure they have what they need to do APIs well while also failing safely.

    Regulations. API regulations are here and are pushing mainstream enterprises to become technology companies, requiring that they have an API strategy for the future.

    Mobile. The world reached a milestone in 2019: There are now more cellphones than people. APIs are how digital resources are made available on these devices in both leading and emerging markets.

    Opinionated. Informed and opinionated approaches to the API life cycle will make a bigger impact than any single technology used in the infrastructure stack.

    Intelligence. There is a massive opportunity in deriving intelligence in aggregate across the millions of API requests we make each day across core business applications.

    Quality. Organizations should prioritize not just testing APIs but testing the API tests themselves. This ensures a level of quality and reliability across all APIs, regardless of the team working with them.

    Education. Ensuring that teams have the proper ongoing training and education around the API life cycle must be a top-line priority across API operations.

    Authentication. The authentication layer of the API infrastructure holds a wealth of intelligence. Having humans study this dimension is critical for security.

    Data. Challenges and opportunities at the API data layer are massive, and machine learning is giving teams an exoskeleton to support failing and iterating faster during development.

    Privacy. Ensuring that privacy is a basic building block of the API life cycle helps developers focus on what they do best, resulting in higher-quality applications that also respect privacy.

    Observability. API analytics have matured in the last few years; becoming much more about observability across the API infrastructure and helping organizations see and respond to what matters.

    Open Source. Open-source technology is an essential part of growing and supporting an API ecosystem, providing both the technical and human nutrients needed to be healthy.

    By keeping these 20 points in mind, organizations can maximize APIs’ role as an extremely powerful mechanism for leveraging data and application functionality for business success. APIs will underpin digital transformation for many years to come, so it’s important to cover all the bases.

  • A Day in the Life of a Remote Tech Worker

    A Day in the Life of a Remote Tech Worker

  • Sumo Logic Extends Observability Reach to AWS Lambda

    Sumo Logic Extends Observability Reach to AWS Lambda

    At the AWS re:Invent conference this week, Sumo Logic announced that in addition to collecting log data, metrics and traces, it now can collect telemetry data from the Lambda serverless computing service provided by Amazon Web Services (AWS).

    In addition to collecting telemetry data, Sumo Logic also revealed that the Sumo Logic Continuous Intelligence Platform hosted on the AWS cloud can now analyze how functions used on the AWS Lambda service perform during transactions.

    That data can then also be correlated against the performance of other AWS services that Sumo Logic tracks via integrations with the AWS CloudWatch and AWS CloudTrail monitoring services.

    George Gerchow, chief security officer for Sumo Logic, said the company is making extensive use of the OpenTelemetry agent software created under the auspices of the Cloud Native Computing Foundation (CNCF) to make it simpler to instrument IT environments. That data is not only being used to improve application performance but also to surface anomalies indicative of a security breach, he added.

    As part of this ongoing security effort, Sumo Logic this week also added Sumo Logic AWS Quick Start integrations for rapid access to security and compliance insights along with support for Amazon Inspector, a vulnerability management service provided by AWS.

    AWS this week named Sumo Logic as its independent software vendor (ISV) partner of the year. The two companies have a longstanding relationship that began with Sumo Logic’s decision more than 10 years ago to build a monitoring platform hosted on the AWS cloud.

    As IT monitoring tools continue to evolve into observability platforms, Gerchow said it’s becoming much easier to leverage machine learning algorithms and open source agent software to identify the root cause of an issue before it results in a major disruption.

    Most IT teams today are still relying on legacy monitoring tools that only tack a set of predefined metrics to identify when a specific platform or application is performing within expectations. The metrics tracked generally focus on, for example, resource utilization. In contrast, observability combines metrics, logs and traces—a specialized form of logging—to instrument applications in a way that makes it simpler to troubleshoot issues.

    Observability, of course, in one form or another, has always been a core tenet of DevOps best practices. Initially, DevOps teams focused on continuous monitoring as the most effective way to proactively manage application environments. However, it can still take days (and sometimes weeks) to discover the root cause of an issue. Observability platforms promise to make it easier to manage IT environments even as the overall complexity of those environments continues to increase.

    It’s not clear at what rate IT organizations will be transitioning to observability platforms. However, it generally takes only one major outage before IT organizations start looking for better tools to manage their IT environment. As those IT environments become more complex, the probability that there will be a significant disruption is now higher than it’s ever been. The challenge and the opportunity is to find a way to instrument applications more widely; the expectation being that an observability platform will be able to analyze telemetry data to reduce the number of issues that lead to service disruptions.

  • AWS Unfurls Bevy of Automation Tools to Streamline DevOps

    AWS Unfurls Bevy of Automation Tools to Streamline DevOps

    At the AWS re:Invent conference, Amazon Web Services (AWS) added a bevy of tools to its portfolio intended to accelerate the pace of application development while simultaneously simplifying DevOps processes.

    AWS development tools unveiled this week include AWS Amplify Studio, a visual development environment that allows developers to create web application user interfaces (UIs) with minimal coding.

    Ken Exner, head of product for developer tools at AWS, said as an extension of an existing AWS Amplify Studio tool, this latest addition allows developers to customize application UIs at a higher level of abstraction using a library of components while enabling them to drop down to a lower level of coding to customize their application further whenever required. After the UI is designed, AWS Amplify Studio automatically generates the associated JavaScript or TypeScript code for the developer.

    In general, AWS is committed to improving developer productivity by providing, for example, automated reasoning tools that automate processes without locking devs into a layer of abstraction that, ultimately, may create a wall that blocks them from meeting customization requirements, he said.

    Other developer tools launched by AWS this week include updates to Amazon CodeGuru Reviewer that adds a secrets detector that uses machine learning algorithms to identify hardcoded secrets that can be secured using AWS Secrets Manager; a revamped Amazon Inspector vulnerability management tool that provides automated assessment scans, automated resource discovery, support for container-based workloads, improved risk scoring, integration with Amazon EventBridge and AWS Security Hub and the removal of the stand-alone Amazon Inspector scanning agent.

    AWS also revealed it has eliminated the need to manually pull images from the Amazon Elastic Container Registry (ECR) repositories by adding automated synchronization capabilities that keep private and public registries consistent with each other.

    In terms of IT operations tools, AWS added a managed database service to its Amazon Relational Database Service portfolio that allows IT teams to customize the underlying database and operating system using an RDS Custom option, a new table class for Amazon DynamoDB designed to reduce storage costs for infrequently accessed data and a service for RDS, dubbed Amazon DevOps Guru, that extends existing machine learning capabilities to automatically detect and diagnose performance bottlenecks and operational database issues.

    Jeff Carter, vice president for relational databases at AWS, said that as databases are increasingly managed by DevOps teams rather than traditional database administrators (DBAs), the cloud service provider is moving to reduce friction by automating as many manual tasks as possible. That’s critical at a time when IT teams are deploying a much wider range of purpose-built databases that are optimized for different classes of applications, he added. The overall goal is to rely more on automation to reduce friction across the IT environment, said Carter.

    Overall, there are two classes of AWS customers: The first are development teams that are increasingly relying on managed services provided by AWS to automate a wide range of DevOps processes. The second tend to be enterprise IT organizations that prefer to manage cloud services themselves.

    Mitch Ashley, a principal for Techstrong Research, an arm of the parent company of DevOps.com, noted that, in many cases, enterprise IT teams are managing DevOps workflows that span multiple clouds. As such, they tend to prefer tools and platforms that can be applied to both multiple clouds and on-premises IT environments. The more those organizations rely on managed services provided by a cloud service provider the more difficult it becomes to consistently manage DevOps workflows across multiple IT environments.

    It’s too early to say just how much IT organizations will rely on managed services going forward. However, at a time when more organizations are using more IT platforms than ever, it’s clear the DevOps management challenge increases with each new platform introduced into the IT environment.