devopsy.kinsta.cloud

Category: DevOps Practice

  • Declarative Compliance With Policy-as-Code and GitOps

    Declarative Compliance With Policy-as-Code and GitOps

    In recent times, the quest for greater agility, faster releases, enhanced scalability, security and performance brought forth the advent of several automation tools, technologies and frameworks. Software development has evolved considerably over the years to mitigate these challenges.

    Monoliths have been split into microservices for improved scalability, maintenance and faster releases. Since DevOps is adept at bridging the gap between development and operations teams, now these two teams can work together to improve productivity and speed up release cycles.

    The Need for Compliance

    One of the critical strategies for business success is to stay compliant since processes are always improving and evolving. Compliance ensures that your DevOps processes and practices are consistent and meet regulations or regulatory standards.

    An organization with a compliance-driven DevOps culture can cut down on operational costs, improve efficiency and reduce risks considerably.

    What is Declarative Compliance?

    Declarative compliance is defined as the process of managing and enforcing compliance in which policies are defined using code, and the system’s desired state is declared rather than explicitly programmed. To enforce compliance policies, you don’t write scripts or programs; instead, you use a high-level language or configuration file.

    Declarative Compliance provides several benefits:

    • Increased efficiency
    • Better managed compliance
    • Minimized risk of human errors
    • Enhanced security
    • Faster deployment
    • Better collaboration

    With declarative compliance tools, the system’s state is defined as a set of policies, and the tool makes sure it matches the desired state. The tool will automatically fix anything out of compliance, bringing the system back into compliance if there’s a deviation.

    Tools for Declarative Compliance

    Some of the popular declarative compliance tools are:

    • Open Policy Agent (OPA)
    • Terraform
    • Kubernetes Policy Controller
    • Chef Compliance
    • AWS Config

    Declarative Compliance for Infrastructure and Application Deployment

    Often, declarative compliance is used in infrastructure management and application deployment, where policies are written as code and enforced across multiple environments. In addition to managing compliance at scale, declarative compliance tools ensure that the system’s desired state stays consistent across multiple systems and applications over time.

    With declarative compliance, organizations can manage compliance more efficiently, effectively and securely since they can define policies as code and enforce them automatically. In addition to reducing errors, downtime and security breaches, this approach enables organizations to stay compliant and maintain their system consistently.

    Introduction to Policy-as-Code

    A policy-as-code approach allows you to write programs that govern security, compliance and rules throughout your application’s life cycle. It entails defining and applying code to manage and automate policies. It shares the same ideas as infrastructure-as-code (IaC) and other DevOps practices like continuous integration and continuous delivery (CI/CD).

    A policy enforces constraints and restrictions to prevent unauthorized access to resources such as databases, storage, services, etc. By codifying policies, you can enforce rules and constraints throughout the development life cycle to prevent non-conforming resources from being deployed by integrating these policies into the software development life cycle.

    compliance policy-as-code

    Figure 1: Policy-as-Code at work!

    Why Policy-as-Code?

    Typically, businesses have the required processes, tools and procedures to guarantee the deliverable’s quality. Performance, scalability, ease of use and accuracy are typical quantifiable quality metrics used to ascertain the quality.

    You can examine the quality of your application using these metrics, but a manual procedure will take time and is prone to mistakes. A better alternative is to leverage policy-as-code.

    The traditional policy enforcement processes have certain drawbacks. It is challenging to perceive that your policies would be entirely adhered to, i.e., they will not be broken. It is cumbersome to check against a list of policies manually.

    Moreover, the traditional policy enforcement method cannot scale in today’s agile world, where organizations have been growing continuously with more and more employees, services and teams.

    Here are the key benefits of policy-as-code at a glance:

    • Faster and more efficient deployments
    • Automated governance, approvals and policy enforcement
    • More reliable and secure applications
    • Centralized policy management
    • Leverages version control
    • Better visibility of policies
    • Enhanced collaboration

    How Does it Work?

    Policy-as-code involves three key elements:

    1. The policy itself, which contains the required code to model the decision-making process
    2. The data and information about the environment, service or application
    3. The query that is responsible for triggering the decision-making process

    Here’s how policy-as-code works:

    1. Policies are defined as code in a human-readable format
    2. These policies are then translated into machine-readable code using Rego or HCL
    3. The policies are integrated with the IaC pipeline
    4. A policy engine evaluates the infrastructure and the application resources against these policies
    5. The policy engine accepts the policies as input, processes them and then produces a query result
    6. The policy engine reports violations (if any) to the respective stakeholders, such as the development or security teams
    7. If violations are reported, the necessary remediation is performed automatically or manually

    Types of Policies

    • Security policies: Security is enforcing adequate technological controls to safeguard company assets.
    • Compliance policies: Compliance policies are designed to help organizations adhere to governance standards. Typical examples include PCI DSS, GDPR, HIPPA, etc.
    • Operational excellence: This encompasses policies pertaining to service degradation or interruption.

    What is GitOps? How can GitOps Help With Compliance?

    GitOps is a subset or extension of DevOps encompassing a collection of practices that can simplify deployments using Git. It is an operating model for cloud-native applications that leverages Git as a single source of truth for declarative applications and infrastructure. Using GitOps, you can implement a standardized policy-as-code approach across your CI/CD and GitOps pipelines.

    The key principles of GitOps are:

    • The state of the system is captured and stored in a Git repository
    • The entire system is configured declaratively
    • Automated deployment

    You can create policies in Git and store them in a particular repository, which is essential since Git offers sophisticated change management tools like version control. Only the compliance team would have admin access to the repository.

    It’s easy to enforce policies with Git, and you can apply them to any execution engine, like CI or CD. With GitOps, you can audit and document every action that affects data consumption.

    You can expose system changes, optimize deployment and use a version-controlled infrastructure. By managing compliance with GitOps, you can make sure that the changes are visible, verifiable and auditable.

    GitOps

    Figure 2: GitOps at work!

    Using GitOps as the Execution Engine in a GitOps Process

    There are several benefits to using GitOps as a process execution engine, including:

    • Increased automation: GitOps automates software delivery, reducing manual intervention and increasing deployment accuracy and speed.
    • Better transparency: GitOps records all changes to the system in a transparent and auditable way so that you can track compliance easily.
    • Maintaining consistency: GitOps keeps the system in the desired state defined in the Git repository, reducing non-compliance risk.
    • Fosters collaboration: All changes to the system are made through the Git repository, so GitOps encourages collaboration across teams.

    Using Policy-as-Code and GitOps: Policy-Based Compliance

    By using GitOps and policy-as-code, teams can manage compliance-as-code in an auditable, declarative way and enforce it via GitOps, ensuring that the desired state is always maintained. This can help your teams to codify compliance requirements as code, store them in version control and automatically enforce them through GitOps.

    Organizations can improve the consistency and reliability of their systems by defining and enforcing compliance requirements using policy-as-code and managing their deployment processes with GitOps. While this can minimize compliance violations, it can also maximize efficiency.

    By following policy-based compliance declaratively, businesses can define policies as code and enforce them automatically and repeatedly. By adopting such an approach to compliance management, you can better manage compliance in complex systems in a consistent, scalable, agile, transparent and collaborative manner.

    For example, you can leverage policy-as-code to define policies for infrastructure provisioning and configuration management to ensure that the infrastructure complies with organizational policies and industry standards. Typical examples are defining policies for network configuration, access control and resource allocation.

    Use Cases: Using Policy-as-Code and GitOps in a Production Environment

    Infrastructure Provisioning

    Open Policy Agent (OPA) can define policies for infrastructure resources like AWS Elastic Compute Cloud (EC2) servers. You can use GitOps to manage the deployment of infrastructure resources by storing IaC files in a Git directory. When the IaC files are changed, GitOps is adept at automatically deploying the changes while OPA checks for policy compliance.

    Deployment

    You can use policy-as-code tools such as OPA to create application deployment policies requiring specific security settings, while GitOps manages the application deployment by storing manifests.

    Manage Kubernetes Clusters

    You can leverage policy-as-code to better manage your Kubernetes clusters. So, you can write your code to define policies to manage Kubernetes resources, such as nodes, pods, clusters, etc., declaratively.

    Compliance Management

    You can use policy-as-code to establish and implement rules and regulations, such as those pertaining to security, to help businesses ensure that their systems comply with industry regulations and standards.

    OPA can be used to set compliance policies, like restricting access to sensitive data to authorized users only. You can use GitOps to manage the deployment of compliance policies seamlessly.

    Trusted Delivery: Policy-as-Code Integrated With GitOps Workflows

    Trusted application delivery is a software delivery approach emphasizing security and trust in the software supply chain. This ensures that any software delivered to the clients is devoid of security flaws, malicious code and other attacks.

    This approach prioritizes security and trust at the development, testing, packaging and deployment stages and integrates security and compliance checks into the GitOps pipelines. This quality check reassures clients about the product’s quality and ensures that any modifications adhere to predetermined rules and principles.

    Trusted application delivery allows for quick deployment while securing apps with automated guardrails. These guardrails are implemented using policy-as-code. By enforcing these guardrails, you encourage frequent deployments while ensuring that the reliability and security of the application are not compromised.

    Conclusion

    Declarative compliance with policy-as-code and GitOps is a best practice for achieving greater compliance, reliability and agility in software development and deployment processes. You can simplify defining and enforcing compliance policies using declarative configuration files and automation tools. Using Git as the single source of truth for all configuration changes, an organization can achieve greater efficiency, visibility and auditability in the compliance process.

  • Synopsys Preps Extensions to Polaris SaaS Platform

    Synopsys Preps Extensions to Polaris SaaS Platform

    Synopsys plans to extend the capabilities of its Polaris Software Integrity Platform for securing application development environments by adding dynamic application security testing (DAST) tools along with the ability to scan code used to provision infrastructure.

    As a software-as-a-service (SaaS) platform, the Polaris Software Integrity Platform was created by combining the static application security testing (SAST) tool Synopsys gained with the acquisition of Coverity in 2014 and the software composition analysis (SCA) tool it added to its portfolio by acquiring Black Duck Software in 2017.

    In 2015, Synopsys acquired the Seeker tool for analyzing the code created using interactive application security testing (IAST) tools from Quotium. The company last year acquired WhiteHat Security from NTT to add a DAST tool.

    Patrick Carey, senior director for marketing strategy for Synopsys, said that the company would add the capabilities of those latter two platforms to create a comprehensive set of best-in-class security tools that are easily accessible via a SaaS platform.

    At a time when more organizations than ever are moving to secure software supply chains, Carey said it’s critical to provide developers with more visibility into vulnerabilities across the entire software development life cycle (SDLC). Some organizations are opting to achieve that goal by embedding tools within the SDLC themselves, but many organizations would likely prefer to invoke a SaaS platform that they don’t have to maintain, noted Carey.

    He noted that a SaaS platform makes it simpler to point a collection of integrated tools at a code repository to surface those vulnerabilities.

    As DevSecOps workflows continue to evolve and mature, more responsibility for application security is shifted left toward developers. The challenge is that many developers have limited cybersecurity expertise, so it’s critical that they have access to high-quality security tools.

    In addition, a SaaS platform helps reduce the total cost of achieving that goal by providing multiple capabilities within a single integrated platform, said Carey.

    Naturally, most of the focus today is on reducing the number of vulnerabilities that find their way into production environments as applications are being built. However, there are already large numbers of unaddressed vulnerabilities in applications that have already been deployed. Organizations will need to find ways to address that technical debt in addition to providing developers with tools that help prevent those code vulnerabilities in the first place.

    As such, there is no one central place through which application security can be managed, noted Carey. Instead, application security needs to be addressed across every phase of the SDLC, he added.

    It’s not clear how quickly organizations are adopting DevSecOps best practices, but as calls for increased liability for software development increase—as outlined, for example, in the U.S. National Cybersecurity Strategy— organizations should expect DevSecOps workflows to become a requirement. Otherwise, organizations will be severely penalized for vulnerabilities that find their way into code. In effect, there is now a race to embrace DevSecOps before any proposed liability legislation becomes the law of the land.

  • How To Keep Engineers Happy and Reduce Burnout

    How To Keep Engineers Happy and Reduce Burnout

    Software development can be a mentally taxing profession. It requires intense mental focus, often under heavy workloads, with unclear job expectations and a lack of control. And with recent labor shortages due to layoffs and resignations, teams may also be understaffed, making matters more complicated. These issues can compound and lead to prolonged excessive stress—that’s when burnout hits.

    In fact, one report found that 82% of developers surveyed indicated that they had experienced burnout in the last six to eight months. Some top causes of burnout include increased workload, a poor work culture and personal reasons. But whatever the cause, for 73% of these developers, burnout is negatively impacting their productivity or personal life. So, what can leaders do about it?

    I recently talked with Randall Degges, head of developer relations at developer security company Snyk, to learn how to address ever-present burnout issues in software development teams at scale. Snyk, which provides popular automated vulnerability detection tools, recently closed a staggering $196.5 million series G funding at a $7.4 billion valuation. With that kind of momentum, they must be doing something right. According to Degges, there are a handful of tactics to consider to prevent burnout, like ensuring developers are heard, not assigning tasks without context and making improvements to the DevOps workflow.

    Understanding Burnout

    Burnout is an ever-present issue, explained Degges. It’s common within any high-stress job where you’re on the hook for constantly delivering new features to support the business. Specifically, in software development, burnout can quickly arise when developers don’t feel they have a voice. Or worse, when they’re tirelessly producing software they know won’t ever see the light of day.

    A paper entitled Burnout in software engineering: A systematic mapping study took an interesting systematic approach and analyzed decades of research into burnout in software engineering. It found the following qualitative work-related factors to be most correlated with burnout. (Interestingly, the paper suggested machine learning might soon help produce more quantitative indicators of burnout.)

    causes of burnout diagram
    Causal-relationship of burnout, taken from ‘Burnout in software engineering: A systematic mapping study.’ (Deciphering this diagram gives me burnout!)

    7 Ways to Fight Burnout

    Since we know burnout is a pervasive problem, how can engineering leaders minimize the issue? Below, we’ll outline some specific strategies to consider and how they might help.

    1. Give Developers a Voice

    First, Degges encouraged leaders to ensure developers have a say in what they’re building. He recalled working within a large enterprise, sitting in a cubicle and having absolutely no say in the features his team was working on. This often included code they all knew would never make it into production.

    Slaving away on irrelevant work is never motivating. Instead, a good internal culture provides a comfortable atmosphere where developers can raise issues early on, said Degges, hopefully before leadership puts targets destined to fail on the roadmap.

    2. Provide More Context

    On that note, when decisions are made, don’t throw orders over the wall in a vacuum. Instead, Degges recommended providing more context into what you’re all creating. Giving visibility into the reason behind decisions will align everyone on project roadmaps and create a shared purpose. This helps foster a culture of empathy and listening to others, said Degges. (David Shackelford affectionally calls this attitude HugOps.)

    3. Optimize Communication Patterns

    Optimizing communication patterns for your working environment is also vital to encourage a healthy culture, said Degges. For example, this could equate to optimizing for the physical reality of distributed teams—no one should have to clock into daily standup at 4:00 a.m. If you are working under timezone constraints, Degges recommended adopting more asynchronous communication.

    For instance, his team relies heavily on Slack and provides dedicated channels for sharing ideas. They’ve also incorporated automation to summarize verbose Slack communications with a recap of conversations. In general, removing unnecessary meetings and optimizing communication patterns could greatly reduce the baggage an engineer might carry in their day-to-day work.

    4. Practice Internal Developer Relations

    As they say, practice what you preach. Many tech-driven companies have adopted a developer relations (DevRel) division to build a community with external developer consumers. But are you practicing DevRel internally? According to Degges, DevRel teams should act as a bridge between external and internal teams.

    “Our goal is to not only to think and empathize with customers but also to encourage internal people to be authentic community members themselves,” said Degges. This can equate to empowering internal employees with resources like mentoring, blog writing or speaking opportunities, he said.

    Additionally, other groups are emerging to help increase internal developer satisfaction. For example, platform engineering and developer productivity engineering (DPE) are being applied by many organizations to improve developer experience around internal tools and processes. Naturally, this will require working closely with internal developers to source feedback and make quality improvements.

    Another method to upkeep engineering morale is through automating away toil. And security vulnerability discovery is one area full of painstaking manual tasks. Half of applications have high-risk vulnerabilities, and new exploits in open source libraries are being discovered almost daily.

    As more software supply chain risks emerge, it’s challenging to keep up with the pace of vulnerability discovery and necessary software patching. Therefore, implementing real-time automated vulnerability detection, said Degges, can help remove many manual processes and free up energy to devote to other areas.

    6. Hire Based on the Work, Not the Interview

    Before developers even get the job, they can be put through high-stress on-the-spot tests during the interview process. Experiences here can set the tone for the entire engagement with the company. But activities like white-boarding algorithms are pretty disconnected from day-to-day work, said Degges. In his opinion, engineers should instead be hired based on the quality of their past performances and the work they’ve produced. This could be artifacts like their personal blog or contributions to open source projects.

    7. Apply Artificial Intelligence (AI)

    Lastly, what’s a tech blog post in 2023 without a mention of AI? Generative AI like ChatGPT has rocked the programming world and is being integrated into more development environments and engineering workflows. The future of AI looks very bright and is poised to help developers produce more with less time and resources.

    Of course, automated code generation will require sanity checks, but these tools can certainly help many ancillary functions. For example, Degges’ team is using ChatGPT to spin up content outlines and automate keyword research, helping support their educational content production.

    Soothing the Burnout

    Even with new tools emerging and a newfound emphasis on developer experience, burnout is still prevalent. A full 58% of developers and security professionals said they are currently experiencing burnout, according to a 2022 Cobalt study. This is, perhaps, due to the rapid release cadences of today’s software delivery.

    “In the past, it took months or years for new software updates to roll out. Now, developers only have a matter of weeks,” wrote Bill Mann, CEO of Styra, on Forbes. “Suffice it to say the stakes are high for today’s software teams.”

    Burnout is psychologically taxing and can negatively affect employee happiness and retention. Therefore, it’s an excellent idea for engineering leaders to take it seriously and consider what positive cultural changes they can make.

  • Creating a Growth Loop for Your Open Source Project

    Creating a Growth Loop for Your Open Source Project

    Every open source project hopes to achieve escape velocity, attracting enough users and contributors to become a successful, sustainable project that expands over time. But how do you attract developers and create the cycle of adoption that can lead to a thriving ecosystem?

    It obviously starts with building a great product that developers love to use, and there’s an element of luck in finding this sweet spot and breaking through. But there’s also a lot that you can control. I was fortunate to have helped create an open source project, Streamlit, that expanded quickly and our company was acquired last year. I wanted to share what worked for me in the hope that it also can be helpful for you.

    My guiding principle is to meet developers where they are and make it as simple as possible, at every step in the journey, to discover your project, learn how to use it and build great products. Depending on the type of open source project you’re creating, providing a simple way to share artifacts and use cases built from your code—in our case, this was applications—can also be a powerful way to raise awareness.

    I think about this in terms of a “growth loop” with multiple points of entry where developers can get interested and get started. These entry points include developer forums, source repositories, your documentation page, learning assets, your website and more. The point is to eliminate friction at every point in this loop and make it as easy and fun as possible to start building and sharing with your code.

    For organizations trying to get their open source project off the ground, here are some key learnings about what worked for my company in building this growth loop and some tips for driving continued adoption and innovation.

    Be Visible in the Open Source Community

    We made it a point to go where developers are already at, including forums like Reddit and Stack Overflow and conferences like ODSC and FOSDEM. Engaging with developers to understand what they really need from your project and how to reduce points of friction is key. We were always grateful when people gave us feedback and we reached out to any developer who engaged with our product, whether they had 10 followers or 10,000. I’ve found that developers really appreciate it when you “show up,” and you learn an incredible amount that can help your project succeed.

    Give Developers What They Need

    Don’t hold back features or functionality that developers will find useful. Developers should have access to everything they need and feel confident that they won’t run into a significant paywall that prevents them from scaling. Building a successful open source project is a bottom-up process, and that means eliminating barriers to entry and making your code as easy to use as possible.

    Where developers go, enterprises typically follow. If you can build a tool that developers love so much they’re coding with it in their free time, they’re likely to take it into the workplace. If your goal is monetization, this is where you’re most likely to have an opportunity to recoup your investment.

    Enterprises require particular features around security, access management, IT support, scaling, service level objectives (SLOs) and other areas—capabilities they are often willing to pay for. So, consider a model where you give developers everything they need to become super users and look for adjacent areas where you can charge for enterprise-type features or enterprise implementations that businesses need.

    Make Sharing Incredibly Easy

    If developers build great things with your code, it’s helpful if they can share those things with the community to show what’s possible and drive interest. You can think of this as a type of advertising. Most developers aren’t won over by traditional ads, but discovering cool things that other developers have done with your code can be a great way to attract new users and fuel the cycle of discovery and adoption.

    So think about the natural output from your project that developers could share with the community. Can they put code on GitHub? Share an app or visualization they built? Post a speed comparison? What can they put on Twitter if they love your project and want to show what they’ve achieved with it?

    Our project is for building data apps, so early on, we added a feature that allowed people to take a screen recording and share their work on Twitter so others could see it in action. Later on, we added a free cloud offering so they could directly share apps with the community. Think about what works for your project and how you could make sharing easy.

    Create Amazing How-Tos and Tutorials.

    These can be important entry points on your growth loop, and it’s best to provide them through developers’ preferred channels. YouTube is a go-to for tutorials of all types, and you can also leverage Coursera, Educative and other e-learning platforms. Someone on my team created a Coursera course for our project that has more than 20,000 enrollees, and one of our Code Camp videos has been viewed over a million times.

    Get creative. If your project is starting to see traction among students, think about providing educational materials that professors can use in the classroom. Online interactive courses are not only a great way to learn programming, they make it easier for educators to manage coursework and see who’s keeping up. This is an excellent way to promote your project and capture new developers.

    Documentation is another important entry point. Our documentation page is starting to look like an e-learning course itself, with guides for getting started, sample code and sample apps. Documentation pages can be listed by search engines, and we found it was worth the engineering time to make this happen.

    Contribute to Other Open Source Projects

    Giving back to the community is a good practice for anyone who cares about software, but it can also benefit your project: Contributing to open source projects helps ensure that your software will be compatible with the tools your target developers use. As well as ensuring interoperability, this can help raise awareness of your project in relevant developer communities.

    Reaching critical mass with an open source project is about understanding what your target developers need, being smart with your resources and removing friction wherever you can. This concept of a growth loop worked for our project, and I hope it helps you to achieve success with yours.

  • Things We Should Acknowledge, Part Two: Skills Are Less Relevant

    Things We Should Acknowledge, Part Two: Skills Are Less Relevant

    This is perhaps the toughest issue for all of us to acknowledge: The longer you are in the space, the more skills you have filled out on LinkedIn (or wherever you are tracking your career for HR reps to look at) but the skills you possess today are, by definition, a subset of the skills you will possess even a couple of months from now if you change roles. Note that I said “change roles,” not “change employers.” The fact is that we use an insane number of tools in the development, DevOps and operations environments, and we’re addicted to adding new ones (that might be a hint about part three of this series). Changing roles within a company can involve minor changes, or it can take you to a whole new environment where no one cares that you are a Java genius; they want to know how fast you can pick up Python and be truly productive. Or they nod at your talk of configuring servers and ask if you can do that with Ansible on containers. It’s just not as important anymore for most employees to have a specific skillset. That’s a major contributor to the problems I talked about in part one.

    Of course, there are roles in IT that involve far less change. Email administration, for example, isn’t all that different from the way it has been since, well, forever. We have changed how/where the server is configured and many orgs have added SSO or multifactor authentication, creating new issues, but it’s all evolutionary. When NoSQL systems came out, it looked like a major change in DBA work, but it really wasn’t. Within a year, reality put SQL over top of NoSQL stores and DBAs kept chugging along.

    But DevOps has impacted both development and operations in profound ways. We pretty much standardized on Java, with all of its warts, because it was object oriented, general purpose and could be compiled if needed. Though embedded largely stayed with C, there was no serious competition in the enterprise for Java and Java-like (.NET) languages. Knowing them meant you could do everything but real-time/embedded development jobs—and even some embedded work shifted to Java as prices for miniaturized compute power dropped.

    But that changed. The inflection point was JavaScript, and from there, we went a little bit crazy. When JavaScript took off, full-stack developers needed to know one front-end and a couple of backend languages. Other languages still existed—the staying power of both Perl and PHP is somewhat surprising to me; they’re both good languages but have limitations. But still, no serious competition. But Node.js took JavaScript to the server, Ruby was popular for backends, Python came along for quick scripting … And now we have apps written in those and a dozen more languages floating around.

    I chose languages to fit the whole story into a blog because they see less change than the operations side, where we’ve gone from annual big releases to rolling deployments updated frequently and all the support tools changed in the process. Infrastructure has changed, data stores have changed, mobility has changed, UI targets have changed—basically, it’s a chaotic mess.

    And you’re in the heart of it, keeping the data flowing and the org running. It’s astounding, and you deserve more kudos than you’re likely getting. So approach hiring differently. Some orgs are already there, but skill-based hiring is still too prevalent. Hire broad technical knowledge and a desire to know more. As someone I know honestly said in a job interview once, “No, I never used it, but it is just another programming language. I’ll be productive in a week.” She got the job and delivered as promised. For all but those few corner-case roles, hire for potential and achievements and skills will follow.

  • CodeSee Extends Reach of Code Visualization Platform

    CodeSee Extends Reach of Code Visualization Platform

    CodeSee updated its code visibility platform to enable instant mapping of application services, directories, file dependencies and code changes.

    CodeSee CEO Shanea Leven said CodeSee Enterprise 2.0 should extend the appeal of the platform beyond developers to include software engineers managing DevOps workflows.

    CodeSee Enterprise 2.0 provides real-time visibility across multiple repositories regardless of whether the application is a monolith or based on a microservices architecture, she noted.

    In addition, a Code Automations module enables developers to set granular conditions and actions down to the function level, requiring checklists to be complete before allowing developers to move on.

    CodeSee Maps now includes the ability to click to view code directly, while a streamlined Review Maps module has decreased code review time by 30%. Developers can easily share maps to discuss proposed changes or keep necessary code changes as automatic permanent artifacts.

    Additional languages including C#, VB.NET, ASP.NET, Blazor, Java, JavaScript, Go, Python and TypeScript are now supported.

    Finally, CodeSee is making available a dedicated solutions engineering team that provides guidance to connect an organization’s Datadog or OpenTelemetry trace data to CodeSee in less than an hour without storing code or having to first dive deep into traces, noted Leven.

    The core open source CodeSee tool was created because it takes too long today for most developers to pore over a codebase to understand where they might contribute. The larger a codebase becomes, the longer it takes for a developer or software engineer to be onboarded to a new project.

    The overall goal is to improve developer productivity, noted Leven. That’s become a critical issue because many organizations are hesitant to hire additional developers during uncertain economic times; they first want to ensure they maximize the utilization of the teams they already have.

    Application development, of course, is now a team sport. The days when a lone developer cared for an entire application are long over. The issue is finding a way for teams of developers to collaborate more easily in an era where many of them may never have met in person.

    In the longer term, Leven said she also expects that generative artificial intelligence (AI) platforms such as ChatGPT will supplement code visualization tools. Many developers are already starting to use these tools to help generate code faster. That code should, in theory, have fewer errors because it has essentially been copied from other previously validated sources. CodeSee provides a means to visualize that code before deploying it to ensure it will function as promised in increasingly complex application environments.

    One way or another, the tools being provided to developers to boost productivity are advancing. It’s not clear what metrics should be employed to track developer productivity, but the amount of code simultaneously moving through DevOps workflows is going to increase. The challenge DevOps teams face now is keeping track of all the dependencies that exist in that code as application environments are continuously updated.

  • Database Virtualization – Driving the Next Big Transformation

    Database Virtualization – Driving the Next Big Transformation

    Data is both the most important asset and the biggest challenge for most corporations. It impacts every aspect of the business from financial considerations to AI/ML training. Data is used by development and DevOps teams, engineering teams, quality assurance, business intelligence teams and database administrators. Despite its central role in operations, traditional forms of database delivery remain complex, costly and time-consuming.

    In many cases, enterprise data is growing exponentially, increasing management challenges and driving up the cost of data storage. Duplicating data to each new environment as use cases arise is overwhelming DBA teams and incurring a significant storage footprint. This has resulted in data becoming a significant bottleneck for DevOps teams, highlighting the importance of finding a quick and automated solution for the replication process across non-production environments.

    What’s Changed?

    In the past, development teams would work with the same dataset for upwards of six months. This left DBA teams with plenty of time to meet all database delivery needs. However, as technology became more sophisticated and consumers grew more demanding, the data delivery life cycle was cut down dramatically. Today, with agile development, new features may be added every few days, if not hours. Coupled with QA teams’ requirement for testing data continuously, AI/ML teams requiring millions of data points for accurate training, and data analysts constantly needing fresh data for organizational growth, this means DBA teams are flooded with demands they are incapable of managing.

    What Now?

    Database virtualization is an automated and easy solution to the dataset bottleneck. By simplifying the creation and distribution of database replicas, DevOps teams will be able to create fully functioning virtual environments via a self-service process–allowing DBA teams to focus on more critical duties relating to production databases.

    Speed Up

    The first—and most obvious—advantage of this approach is the increased speed at which data is delivered. To keep DevOps teams productive and avoid the use of old, stale data, new environments need to be produced on demand. By integrating the production of new environments into the CI/CD pipeline, enterprises will be able to automate the process and provide DevOps teams with full autonomy. By cutting out the dependency on the DBA, teams will be able to access the data they need, when they need it.

    Scale

    Agility is fundamental to the success of database management. Each team requires an independent environment, unaffected by other use cases. The creation and supply of virtual databases allow for quick provisioning. Once created, these databases look identical to the original physical database and function completely independently of each other. Any changes made to a vDB remain unique to the environment it’s in.

    Save

    As the need for data grows, so does the cost of storage. Creating a physical copy for each team and use case results in a significant storage footprint. This means that these datasets come with a hefty cost. However, the creation of virtual databases led to a dramatic decrease in storage needs. Where traditionally, physical datasets can weigh upward of several TB, vDBs only weigh a few hundred megabytes. Further, virtualization solutions also compress the weight of physical copies – such as the golden copy – actively decreasing the weight and cost of storage across all fields.

    Secure the Database

    Even after the database delivery problem has been solved, enterprises are faced with another major issue – security. With so many different teams accessing data in virtual and low-level testing environments, the risk of ransomware and other cyberattacks increases dramatically. Generally, the investment in securing non-production environments is significantly weaker than in production.

    A common approach to address this issue is masking sensitive data so it is rendered useless to outside sources. However, traditional masking requires a data expert to manually sift through all existing data and implement data masking algorithms on any sensitive information found. This time-consuming solution creates a new data bottleneck. By automating the process and applying it to the golden copy, corporations can be confident that all vDBs created are completely secure and privacy compliant.

    Looking Ahead

    Database virtualization is the future of database replica management. Virtual databases are delivered in a matter of minutes, significantly accelerating the speed of delivery. These environments are easy to use, independent, and can be maintained via a self-service portal. The ongoing sync between the original database and the vDB ensures continuous access to the latest data without the need for timely or expensive processes.

    Effective database virtualization will allow well-established enterprises to more easily compete against those agile startups that face significantly fewer barriers to entry.

    Image Source: joshua-sortino-LqKhnDzSF-8-unsplash

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

    Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    Dice.com

    NerdRabbit
    New York, New York
    DevOps jobs
    $120,000 to $150,000

    LinkedIn

    Tata Consulting Services
    Sunnyvale, California
    AWS DevOps Engineer
    $105,000 to $130,000

    Monster.com

    VANET, Inc.
    Irvine, California
    Software Engineer
    $100,000 to $300,000

    SimplyHired

    Renaissance
    Remote
    DevOps Engineer 2
    $100,000 to $125,000

    Indeed.com

    GEICO
    Remote
    Senior DevOps Engineer
    $82,000 to $185,000

  • npm is Scam-Spam Cesspool ¦ Google in Microsoft Antitrust Thrust

    npm is Scam-Spam Cesspool ¦ Google in Microsoft Antitrust Thrust

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: The npm registry suffers spam infestation, and Microsoft makes Google sad. (more…)

  • Debunking Myths About Reliability

    Debunking Myths About Reliability

    “Our service should always be up.” Some myths just won’t die.

    Engineering for reliability is well understood by engineering leaders, less so by bosses demanding unreasonable uptime with minimal resources and immense feature pressure. Business leaders tend to waffle between ignoring reliability with a hand wave and freaking out after an outage. How can they understand the reality of reliability and forget the myths?

    Myth One: Our Service Should Always be Up

    Reality: Our Service is Engineered to Consistently Exceed Expectations

    Some might imagine reliability like a light switch—it is either on or off. But reliability is about consistent repetitions and managing the risk of minor, occasional, recoverable failures. Can we consistently exceed customer expectations?

    Understanding customer expectations is challenging because you can’t ask people what they expect from you (unless you have a peculiar customer). So instead, we want to define and quantify the impact of our service not working and how that could negatively impact customer experience based on observed behavior. For example, “People abandon their shopping cart when our checkout experience takes more than 10 seconds to load, which means we lose money.”

    We reflect business impact in our reliability goals and then engineer our system and processes to meet this goal. We might change how we do releases, ensure tests pass or avoid changes during times of peak usage. These engineering decisions use reliability as a business metric based on customer expectations.

    Myth Two: Innovation is More Valuable Than Reliability

    Reality: You Need to Balance Innovation and Reliability Engineering Work

    The constant drive for new features over reliability is the most frustrating myth. “We need to release new feature X, or we won’t have customers; we can worry about reliability later.” In reality, most customers care about reliability; they don’t mention it until it’s a problem.

    You can have the most fantastic whizbang feature in the world, but if it doesn’t work–and Murphy’s Law will make sure that it doesn’t work at the worst possible time–no one can use it, no one will be impressed and it will turn your technology into a laughingstock.

    Innovation excites customers, but trust comes from reliability, which you must earn through hard work and clever engineering. Depending on your business context, you may require reliability more than ever. If you are facing headwinds, you may need to scale back your ambitions regarding innovative features. Still, you can’t scrimp on reliability or customers will justifiably leave.

    Myth Three: Five Nines is Normal and Incremental From Four Nines

    Reality: Five Nines is Expensive—10X the Cost of Four Nines

    No one–not even massive cloud providers or telcos–can consistently deliver at 99.999% across all their services by accident. Reliability at that level–less than six minutes downtime per year!–is an engineering marvel. A bridge or a dam might look simple after completion, but the engineering required to create a reliable physical infrastructure is immense, as everyone knows. Why is it so hard to understand the complexity, design, engineering and redundancy required to deliver a highly available and performant digital system? Further, it’s easy to think that 99.999% is just a bit more than 99.99%. After all, it’s “just one more nine.” Remind your less-technical counterparts that each nine requires ten times more effort!

    Why is it so expensive to deliver? Because the failure tolerance (also known as an error budget) is 1/10th the size but the risk of missing the goal increases exponentially. You’ll need more redundancy, careful testing and certification of releases, increased on-call rotations, extra hardware or cloud capacity and extensively tested backup plans to achieve this goal.

    Worst of all, higher reliability will slow you down. You can’t innovate or deliver updates as fast when you need to ensure absolute uptime.

    But what if there was a limit to how much reliability we need?

    Myth Four: More Reliability is Always Good

    Reality: Reliability Engineering Has Diminishing Returns

    There is a point at which being “too reliable” is terrible for business. It’s expensive to build all that redundancy, testing, responding to tiny glitches and all the rest. And most of your users won’t notice. We must avoid the large blowups that put us in the headlines and manage expectations everywhere else. The significant outages that can impact thousands, if not millions of customers come from this reductionist view of reliability as a by-product of conscientious work rather than an engineering problem with well-defined tolerances and thresholds. You earn the trust of your customers by properly engineering reliability into your delivery process.

    Consider tardiness at meetings. If you wanted to be 99% on time, you’d need to join a one-hour call within 36 seconds and at 99.9%, you would need to enter a Zoom within 3.6 seconds of its start time, a timescale so small you don’t even notice it. You would have to do this for every meeting you attended, no excuses–bio breaks, last meeting ran long, someone at the door, etc. None of these things matter when defining and measuring reliability. This metaphor also provides a common sense way to think about risk and error budgets. Your other meeting attendees can’t possibly notice (or care) if you’re 3.6 seconds late, no matter how prestigious or impatient the other party is.

    You could apply this same reasoning to catching a flight, picking up your kids from school, completing an exam, building a woodworking project or any human endeavor. The concept is so intuitive to daily life that even pointing it out seems absurd. But this is the fundamental concept from which reliability engineering stems. To build a reliable system, we must define acceptable failure boundaries. Otherwise, we will spend precious time and resources to eliminate the 3.6 seconds of delay that no one cares about and miss the more significant issues–like being present and engaged in the discussion.

    Busting Reliability Myths

    Understanding reliability is vital for engineers and business people alike. It all comes down to intentionally designing a customer experience, keeping up with expectations and, in some cases, even promises. Right-sizing reliability lets you find the perfect balance between delivering excellent service and efficiently running your organization.

    Image Source: Indira Tjokorda via Unsplash