devopsy.kinsta.cloud

Category: DevOps Practice

  • From Frontend to Full Stack: How I Leveraged Modern Dev Tools to Make the Leap 

    From Frontend to Full Stack: How I Leveraged Modern Dev Tools to Make the Leap 

    A few years ago, I had an idea about how to make publishing recipes on the internet more accessible and profitable. As a front-end engineer-turned-product manager, I could write code as long as it involved JavaScript. But I needed someone who could handle the whole stack, especially the back end—you can’t build a platform with frontend tools only. So I searched for a technical co-founder, poked around online forums and talked to friends. Eventually, I found a candidate (let’s call him Tom) who seemed like a good fit. He’d built a recipe app in the past and was comfortable with both the front and the back end. It seemed great. We struck a deal and dove in.

    Tom started building the API and database while I made prototypes with a no-code builder called Bubble. I loved using Bubble. Without writing any code, I could build things fast and see what I was creating. Eventually, though, there were too many limitations with the no-code approach, including the lack of a Git repository for version control and history. So we decided to switch to full code with React Native mobile apps. 

    A few years prior, I’d switched career paths and became a product manager, but it had been a while since I had dug deep into writing code. And although I had a background in React, I felt insecure about building the frontend of a whole web platform on my own. But alas, you do all sorts of scary and undesirable things when you’re a startup founder!

    After jumping into it, though, I quickly remembered that I really enjoyed coding. I picked up React Native pretty quickly and started building screens. Tom took care of the data while I took care of the UI. All was good.

    Well, all was good until it wasn’t. We were a few weeks away from launch day and were finishing up. We even had a few early-access bloggers lined up to launch with us. And then, surprise! Tom decided to take a job as the CTO of another startup. His parting words were, “You’re technical enough; seems like you can handle it.” 

    I was floored. Aside from the front end, I had no idea how anything worked. We had cloud functions running on Pulumi hosted in Google Cloud, and I had no idea how they worked. I didn’t know how to train the machine learning AI models we used for ingredient recognition. The most I knew about SQL was that you used it to get stuff from a database. And to top it all off, our recipe scraping API was written in Python, a language I had no experience with, running a version that did not work on my M1 Mac. What in the heck was I supposed to do?

    Well, I guess I did what any engineer would do. I figured it out step by step. When the Postgres database ran out of rows on the free Heroku plan, I figured out how to migrate it. When the recipe scraper API needed updating, I learned how to code in Python and deployed it with Docker. It was an incredibly stressful time. I was up against a looming launch deadline with a codebase teetering on collapse at any moment—mostly because I didn’t know how to hold it up.

    The silver lining was the GraphQL API platform, Hasura. It was by far the easiest part of the stack for me to grasp. Understanding how our data was laid out was easy enough—I could simply explore it like a spreadsheet. I wasn’t sure how to build GraphQL queries, but the tutorials on the Hasura website helped me learn the basics and the query builder in the console did the rest. It’s no-code, but better—a no-code console makes it super easy to add new tables and modify permissions, and a Git repository keeps everything in check and easy to reference. 

    Once I understood how Hasura and the rest of the stack worked, I was able to finish the screens that I needed for launch.

    Today, the company has grown significantly. We have a new co-founder, but she’s nontechnical. After figuring out how to run the entire stack, I decided I was technical enough to be the primary engineer. I’ve learned how to use all sorts of Hasura functionality that makes our lives so much easier—listening for events, firing off actions in our API to run business logic, and much more. Through all this, I now consider myself a full-stack engineer. 

     

  • Prioritizing Product Security With DevSecOps

    Prioritizing Product Security With DevSecOps

    Building software with strong security can no longer be an afterthought for organizations. The need for a reliable cybersecurity posture has proven vital amid the constant attacks we’re seeing across industries, all over the world. Product-specific security is top of mind for most companies, and a successful product security practice requires integrating with the ethos of the engineering team—DevSecOps. Consider a three-fold strategy, which has proven successful again and again in the industry.

    It consists of first creating a culture of security, with all stakeholders understanding the priorities, strategies, and business value. Second, teams must integrate security with the software development life cycle (SDLC). It’s not enough to shift left–organizations must shift their thinking from shift left to shift everywhere. And lastly, monitoring product systems at all times is non-negotiable. In a 24/7/365 world, there must be teams to watch over security 24/7/365. Part of the engineering deliverable becomes providing data to these teams so that they can have all eyes on the products, always.

    A Successful Practice Starts With the Culture

    In building a strong culture, the goal is to make the entire engineering team a part of the mission. This includes putting a product security team in place to support engineering needs as well as providing continuous education with resources available for addressing any eventuality.

    While every organization has its own unique needs, considerations and approaches to creating a product security team, these five roles are highly recommended:

    1. Technical Product Manager (TPM): Responsible for driving security priorities in planning
    2. Security Architect: Sits on the Architecture Review Board and ensures architecture and design have security processes incorporated
    3. AppSec Engineer: Responsible for implementing the security development life cycle
    4. Adversarial Engineer: Hammers on the attack surfaces to identify weaknesses
    5. Security Champions: Engineers from each Scrum team with an interest in supporting their teams with security fundamentals; they are instrumental in scaling out the overall product security effort

    Creating and executing an engaging security education program is also pivotal to winning over the hearts and minds of employees. Consider initiatives such as in-depth security boot camps, vulnerability-of-the-month presentations, capture-the-flag (CTF) contests and security consultation programs.

    Integrating Security Into the SDLC

    From a high level, all SDLCs consist of planning, development and release, and each phase should have its own set of security controls.

    Planning: The product security TPM is responsible for working with the product management team to determine security priorities–derived from sources like risk reduction, executive priorities, customer requests, security findings and regulatory requirements. Determining what is most important is key.

    Development: Much of the heavy product security lifting comes during the development phase, which includes implementing threat modeling, security baselines and automated scanning.

    Threat modeling: The process of identifying potential threats in a system. Once identified, the threat modeling team can create mitigations, and results are integrated into penetration testing to verify that the threats did not make it into implementation. It’s most efficient to test models during the architecture phase before coding begins to avoid rework and Band-Aids. For example, a company may have a policy disallowing access to port 22 from the Internet. Threat modeling automation can detect violations in the design and notify the owning team asking them to close the port. Tools that support this use case include IriusRisk and ThreatModeler, as well as Microsoft Threat Modeling Tool, which is a free option but does not include automation.

    Security Baselines: Teams should create secured images that are fully patched, meet security policies and have mandatory security agents preinstalled. Doing this for both OS images and containers allows developers to get a level of security for “free” and avoid ongoing security reviews. Baselines should automatically update and flow into CI/CD systems.

    Automated scanning: To keep up with Agile processes, scanning should be automated and have a low number of false positives. In fact, it is better to report on fewer issues than kill credibility with a lot of noise. Continuous tuning will increase the fidelity rule sets and helpful tools include static application security testing (SAST), software composition analysis (SCA), and dynamic application security testing (DAST). Automation is critical to securing applications that use a CI/CD development process and providing security right at the code push is invaluable–developers get immediate feedback, not a ticket weeks later when they likely have forgotten important context. Integrating these tools into build pipelines is a solved problem.

    For example, the below YAML demonstrates automating source scans for embedded secrets (e.g., passwords and keys).

    sast:secrets-detection:
    extends: .sast:secrets-detection
    stage: deployment:development

    Release: When teams reach the release phase, adversarial testing becomes an important component. Penetration testing can be in-house, outsourced or both. An internal red team will have a better understanding of the technology and, therefore, have a broader scope than external security partners. For example, they test across product boundaries, test processes and people and test live production systems. For a mature product, consider a bug bounty program. This will invite white hat hackers to find weaknesses in exchange for cash. But “mature products” is essential; otherwise, the number of payouts will become overwhelming.

    24/7 Monitoring

    As mentioned earlier, 24/7 monitoring is key, especially for any cloud-based technology. A dedicated security operations center (SOC) can help, scrutinizing for threats against the company, employees and customers. The SOC’s responsibilities are much broader than those of a product security team, and they allow product security folks to sleep well at night. Security teams should configure systems, load balancers, public cloud platforms and networks to feed data directly into the security information and event management (SIEM) system. The SOC can then respond to custom automated alerts that identify any data anomalies that are being streamed to them.

    As products–and technology overall–become more advanced, the way organizations secure them must do so as well. The perfect security approach varies from business to business, but empowering IT, engineering and security teams is a beneficial first step to identifying, optimizing and integrating security for both employees and customers.

  • CloudBees Survey Sees VSM Driving Digital Transformation

    CloudBees Survey Sees VSM Driving Digital Transformation

    A global survey of 600 senior executives conducted by CloudBees suggested a high correlation between investments made in value stream management (VSM) and digital business transformation success.

    The survey found 90% of respondents agreed that organizations that committed to VSM are likely to be further ahead in terms of achieving their digital business transformation goals, with 92% noting VSM also creates more time to focus on innovation.

    Hope Lynch, senior director for platform at CloudBees, said the challenge is that too many executives equate VSM with legacy Six Sigma processes rather than application development and deployment processes that advance digital business transformation initiatives. The survey found that while 90% of senior executives claimed they were familiar with the VSM concept, only half could select the correct answer when asked. In fact, two in five respondents confused the concepts of value stream management and value stream mapping. Nearly two-thirds (72%) of respondents admitted there is still confusion about what VSM is.

    VSM traces its lineage back to lean manufacturing methods, which called for each step of a manufacturing process to be continuously measured. As software development has evolved and organizations realize how dependent they are on software, there is a growing appreciation for the value of monitoring things such as the impact of missed software development deadlines on the business. At their core, VSM platforms aggregate DevOps metrics and then match them to a set of key performance indicators (KPIs) defined by business leaders. Once those metrics are gathered and correlated, organizations can apply advanced analytics.

    Eventually, the goal is to apply algorithms and other forms of predictive analytics that will enable organizations to more easily identify the precise impact software development processes have on the business, noted Lynch.

    In the meantime, the CloudBees survey identified manager commitment as the main roadblock to VSM adoption, with 35% of respondents saying getting mid-level employees committed and 31% saying getting senior leadership committed are the biggest roadblocks. That’s a significant issue because more than half of respondents (56%) said C-level executives make VSM investment decisions.

    Survey respondents also reported that scaling VSM efforts throughout the organization is a significant issue for a third (33%) of respondents.

    In general, many VSM initiatives were sidetracked by the COVID-19 pandemic, noted Lynch. However, organizations will now be looking to evaluate the results of those efforts, she added. Digital transformation plans are in place and going well, according to 96% of survey respondents, with only 18% reporting that they are moving, but not fast enough.

    However, all software development processes could be improved by surfacing real-time insights, added Lynch. Most senior executives today don’t have a lot of visibility into the software development processes that are driving multiple digital transformation initiatives, she added.

    In theory, VSM platforms should one day help narrow the divide that has long existed between IT and the rest of the business, as dashboards make it simpler for everyone involved to appreciate the cascading impact a software development decision has across the entire business.

  • Shift Left Testing in Microservices Environments

    Shift Left Testing in Microservices Environments

    By now, it’s common knowledge that the later a bug is detected in the software development life cycle (SDLC), the longer it takes and the more expensive it is to fix that bug.

    In 2017, the Ponemon Institute found that it cost around $80 on average to fix a defect detected early in the SDLC and around $7,600 to fix the same defect if it was detected in the production stage.

    Relative Cost of Fixing a Defect Based on Development Stage

    Design

    Implementation

    Testing

    Maintenance

    Cost of Fixing Defects

    1x

    6.5x

    15x

    100x

    While these numbers can be debated, it’s clear that early detection of defects can save organizations significant time and money while enabling those organizations to deliver higher quality software more rapidly. For these reasons, many organizations are implementing shift left testing practices, including in microservices environments.

    Shift left is a set of practices intended to find and prevent defects early in the software delivery process instead of during the testing and maintenance stages. The idea behind shift left testing is to improve software quality by making actionable test signals and feedback available as early in the SDLC as possible.

    Here, we’ll highlight how shift left testing, in combination with test automation, can better enable both innovation and fast delivery of high-quality software. We’ll also address some of the challenges involved in implementing shift left testing in microservices environments, as well as introduce a valuable strategy for successful microservices implementation.

    Shift Left Testing Benefits

    The waterfall model of development involved the explicit passing of responsibilities between highly specialized design, development, QA, and release teams. It also involved lengthy feedback loops.

    Scrum and agile methodologies made the entire SDLC more flexible and nimble by introducing sprints and allowing more frequent iterative development and delivery. Further, DevOps and DevSecOps focus on removing the silos between development, operations, and security through tooling and automation. As a result, the time to market and quality have improved dramatically. Adding shift left testing into the mix better positions teams to handle the broad range of responsibilities from the design stage through the maintenance stage as effectively as possible.

    Shift left testing focuses on prevention rather than detection. Shift left benefits include the following:

    • Increase efficiency by eliminating bugs earlier in the SDLC.
    • Reduce human errors and associated costs.
    • Increase delivery speed and reduce the time between releases.
    • Improve the quality of software.
    • Gain a competitive advantage.

    It may be helpful to think of shift left testing like this: Test early and in parallel with development, instead of the traditional waterfall model of development followed by QA testing. 

    Shift Left Testing Challenges in a Microservices Environment

    By structuring an application as a collection of loosely coupled, independently deployable services, a microservices architecture enables frequent delivery of large, complex applications.

    While shift left testing practices bring many benefits, shift left testing also comes with three challenges that organizations must address when implementing it across a decentralized, microservices-based stack:

    1. Building and managing each microservice separately reduces development time. However, implementing shift left testing at the microservices level means shift left practices must be applied to each individual microservice.
    2. Microservices developers can choose the programming language and service dependencies best suited to their development needs. However, as a result, shift left testing practices need to be implemented and managed across different programming languages and dependencies.
    3. To ensure that microservices function as expected, teams usually run integration tests, end-to-end tests, and performance tests that assess functionality in the context of the larger stack. Shift left testing is especially challenging in the microservices context because of the complexity of testing against the stack.

    Microservices, each with their own SDLC pipeline

    Pipeline Scaling Considerations

    Several classes of automated shift left tests can be integrated into the DevOps pipeline, including unit tests, integration tests, performance tests, and security tests. Different microservices will have different testing needs.

    The key is to apply shift left testing practices to each microservice while maintaining the flexibility and decentralization of those microservices. The ideal workflow system and DevOps pipeline implementation must simplify pipeline creation while enabling development teams to manage the pipeline and requiring minimal input from DevOps and platform teams. In other words, shift left practices must be tailored to the specific needs of the microservice in question to make the testing process scalable and easy for development teams to manage on a continuous basis.

    As just one example, GitHub Actions and similar CI/CD pipeline tools enable development teams to use GitOps-based practices to configure, manage, and administer the entire pipeline for their microservice while leaving infrastructure testing and testing that relies on or affects other parts of the system to DevOps teams.

    Test Environment Considerations

    Test environments are environments where unit tests, integration tests, performance tests and security tests can be run on an isolated collection of microservices and configurations. The nature and quality of these test environments directly correlate with the effectiveness of shift left testing.

    A cloned production or staging environment can be used for testing when the number of microservices is low and overall complexity is relatively low. However, once an organization reaches a threshold number of microservices or a threshold level of complexity, runaway infrastructure costs and operational burdens make such environments untenable.

    In response, some organizations rely upon smoke testing (minimal viable stability testing) and mock-based integration testing early in the SDLC. While such tests can prove useful in the near term, deferring essential testing to a later stage in the SDLC often introduces test signal and validity problems in the staging environment and can lead to delivery delays and serious issues with software quality.

    So how can shift left testing be used successfully in a microservices environment? The solution involves the use of scalable, ephemeral test environments.

    Ephemeral Shift Left Testing Environments

    The availability of high-quality test environments is a common bottleneck for organizations looking to scale shift left testing practices. Using a few high-fidelity staging environments for signal testing can slow the release process. More to the point, it tends to slow down the entire SDLC and increase stack instability over time.

    Ephemeral test environments offer a fundamentally different approach. These sandbox environments use isolated environments that enable users to test independently without impacting other users. In particular, they enable isolation while sharing resources (e.g., external cloud dependencies and hosted databases) using application-level multitenancy instead of the resource duplication approach in traditional environments.

    The idea is that microservice test environments can be used to test changes against a much larger stack of services. Because multiple tests can be conducted at the same time, test coverage and resource efficiency vastly increase.

    High-quality sandbox environments can spin up in seconds and scale to tens of thousands of microservices. In fact, each pull request, branch, or commit can have its own test environment without significantly increasing infrastructure and operational costs. Any change, no matter how small, can be tested in isolation to ensure performance and stability against the entire stack.

    Shift Left Testing Solutions for Microservices

    A microservices-based organization looking to scale shift left testing across the organization must first address several challenges. Considerations include the need to apply testing practices to separate microservices, the need to accommodate different programming languages and service dependencies and the need to test against the stack.

    The right tooling and automation are needed before shift left testing practices are scaled across the organization. In addition, a cultural shift may be required in order to help developers understand the ways in which shift left testing can expedite and improve the core functionality that they are developing.

    To apply valuable shift left testing practices in a microservices environment, organizations should consider using ephemeral sandbox environments. Using the right set of tools in a sandbox environment can enable development teams to test microservices throughout the SDLC. In doing so, organizations can ensure those services follow security best practices and perform exactly as intended, both independently and against the stack–all while reducing costs, speeding time to market and improving software quality and user satisfaction.

  • Index Tracks Impressive Growth of API-First Startups

    Index Tracks Impressive Growth of API-First Startups

    APIs are the building blocks of modern software, and their use continues to grow around the globe. A full 89% of investment into APIs will increase or stay the same over the next 12 months, Postman’s 2022 State of the API report found. And the evolving API-first approach is ushering in many innovations and startups into the field. This trend is epitomized by the latest research from GGV Capital, which maintains an API-First Index of leading developer-first software companies.

    The API-First Index tracks API-first companies that have raised $50 million in funding or more. And as of Q3 2022, the number of businesses that met this threshold increased to over 60. Just in Q3 alone, $350 million in funding was raised by companies on the index, and, in total, the index represents over $14 billion in funding. Undoubtedly, interest and investment in this space are expanding and will continue to increase in the year ahead. It’s a good area to keep tabs on, as the emphasis on modularity impacts how software is pieced together.

    I recently met with Tiffany Luck, partner at GGV Capitol, who oversees the API-First Index. Below, we’ll explore the index a bit more and get Luck’s viewpoint on why these findings represent a unique moment for API-first technology and business models as a whole.

    The API-First Index

    The API-First Index focuses on companies that serve niche functionally through APIs; also known as API-as-a-product. The list includes an assortment of API-based tools from FinTech, communications, health care, commerce and other sectors.

    For example, Stripe is likely the most well-known API-as-a-product, as it is commonly integrated as a way for businesses to accept payments. There’s also MessageBird, an API for sending notifications across any channel. Other examples include Mapbox, an API for geolocation data, Mux, an API for creating live and on-demand video experiences, Nylas, which offers a popular email API and Alpaca, an API for stock and cryptocurrency trading and many others.

    Currently, the index doesn’t include the entire API platform landscape of tools, which ranges from areas like API management to testing, documentation, monitoring, design, security and observability. There are also middleware integration platforms to consider as well. With these included, the list of well-funded API-related toolsets would surely be even longer.

    Why the Excitement About API-First?

    The number of people deciding to build API-first companies is rapidly increasing. These API-based web components are interchangeable and represent the future of software development. “The world is going toward web and cloud components,” said Luck. The trend represents a global movement with much effort reflected in Europe, Asia and the U.S. Furthermore, API models seemed resilient throughout ongoing economic uncertainty and the patchy VC funding of 2022. “APIs are holding up very well compared to the broader VC landscape,” said Luck.

    So, why is there so much excitement around APIs? Well, people are looking for best-in-class structural blocks that can be used to build common features like geolocation, AI, weather, payments or specific CRM functions. Niche APIs can swiftly deliver particular features without the bulk of a large platform. Another reason is that they’re relatively sticky—API-based delivery models are efficient at building a self-service platform that expands over time, which is a very keen prospect to investors, said Luck.

    Unicorn APIs Solve Hard Problems

    In evaluating API-first companies, Luck looks for those offering B2B solutions to super-hard, shared problems. This equates to reusable functionality that is difficult to build, maintain and scale individually. Furthermore, the solution must solve a need that’s common across companies but that is not core to their products.

    For example, Luck has her eye on a few impressive API-first use cases. Stream, for example, powers in-app user-facing features. Knock offers solutions for handling notifications. But many of the top most-challenging areas include financial services, as this functionality is difficult to build from a regulatory and connectivity perspective. Examples include Pinwheel, which aggregates real-time income and employment data. Method API does something similar but for aggregating consumer debts.

    Luck described APIs as a “modern way of accessing what would otherwise be technically inaccessible.” We’re seeing this action unfold in open banking, as new standards around the globe now mandate that consumer financial data be programmatically accessible, often in the form of an API connection.

    Developer Experience: The New Competitive Advantage

    In a competitive market surging with powerful developer-facing tools, non-functional aspects such as developer experience (DX), security, performance and reliability are becoming more of a focus. According to Luck, developer experience is probably at the top of the pyramid as it represents a catch-all for many of the aforementioned traits. Specifically, developer tools with high-quality DX have excellent documentation and must maintain backward compatibility to avoid breaking changes on the consumer side. “Ultimately, what you’re implementing impacts that customer’s end customer. Customers want zero negative impact and zero disruption,” she said.

    The API-First Index is good for the industry as it helps educate the public on private investing rounds. And as discussion around it increases, it brings awareness and visibility to the benefits of building API-first.

  • 5 Tips to Consider When Introducing Infrastructure Automation

    5 Tips to Consider When Introducing Infrastructure Automation

    Although we discuss infrastructure automation often, the fact is that a lot of manual processes still exist throughout DevOps. Whether it’s rewriting scripts to spin up new servers, updating cloud-native policies or configuring computing environments, manual toil is an all-too-common reality. To make matters worse, engineers commonly create lengthy scripts that work in isolation but are too bulky to be reusable by other developers working with slightly different constraints.

    More infrastructure automation is the key to increasing deployment agility, but setting aside time to automate your daily workflow can easily take a back burner, especially when developers are already behind on their long list of to-dos. So, for the time-pressed, where should they start? What exactly should be automated? And how can we effectively reuse and share automation to help other areas of the business?

    I recently connected with John Laffey, technical product marketing director, Puppet. Laffey has thirty years’ worth of tech support and engineering operations knowledge under his belt and was “doing DevOps before it was called DevOps.” He shared with me five helpful tips to think about when introducing more infrastructure automation into your workflows. Below, we’ll expand on each of those points and consider other best practices around automation as the industry moves toward platform engineering.

    1. Automate Things That Interrupt Your Day

    First, Laffey recommended starting with the things that are easy to fix. These are the tedious actions that interrupt your day. Perhaps a disk runs out of space and troubleshooting takes 15 or 20 minutes. Minor interruptions like this might seem like part of the job, but they definitely add up over time.

    2. Automate Things You Can Reuse

    Secondly, try not to automate things for an ultra-specific purpose, said Laffey. Instead, focus on building automation that is more generic and can have more value to others. This follows the DevOps mantra of teamwork and collaboration. One-off automation that is too verbose or specific to a certain condition won’t apply well across groups.

    3. Automate Stuff That Doesn’t Change Much

    Next, automate the infrastructure that isn’t prone to change. These are fairly reliable things that you’re sure will be part of your systems for the foreseeable future. Because if a component does end up having to be updated often, supporting the automation could be more hassle than it’s worth. “Don’t trade fixing problems for fixing automation,” said Laffey.

    4. Start Small

    Here’s some sage advice applicable to most new projects with a big scope—start small. Laffey recommended starting by automating the easy stuff that has proven immediate value. These tasks should be modular and do one specific thing, like rebooting a machine or draining a log file. “Automate things in chunks that you can put into a toolchain.”

    Too often, teams try to automate everything all at once. But by taking big bites, you can quickly get off track and potentially add to technical debt. If you keep things modular, you’ll find it easier to change and reuse this automation.

    5. Share and Be Open

    This last tidbit is more of a cultural change—don’t code so that you’re the only one who knows about it! Sometimes, engineers don’t necessarily want the world to know about the automation they create. They might worry about automating their job away. Yet, Laffey said, these worries are misplaced. There will always be more work to do, and automating toil away can free up time to help you focus on more exciting projects.

    So, Laffey encouraged DevOps and SREs to be open about the automation they create. Ensure they’re well documented and stored in a centralized repository to allow other departments to reuse your code. This can avoid the duplication of having five different versions of the same script written in different languages within the same organization!

    Next Step: Move From Scripts to Industry-Standard Tooling

    It can be challenging to argue for investing time and energy into infrastructure automation, especially when engineers are hard-pressed to keep the lights on. Yet, small gains in automation can make huge waves in the long term.

    Eventually, said Laffey, you’ll want to move away from using scripts within your toolchain, which he describes as a low-end DevOps maturity. But as you update your process, you’ll want to replace these scripts with more modern industry-standard tools like Terraform or Puppet’s DSL. He recommended slowly revamping scripts in a toolchain as you update them one by one. “Start where you are and avoid a monolithic approach.”

    In essence, platform engineering, Laffey explained, is wrapping DevOps principles at a higher level to make them more efficient. And we will likely see an increased emphasis on infrastructure automation as platform engineering gains more mindshare. “It’s going to become a necessity.”

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted as part of an effort to serve our audience better.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.
    Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    Dice.com

    APN Software Services
    San Jose, California
    Principle Engineering DevOps
    $190,000 to $200,000

    ZipRecruiter

    Craft Machine
    San Francisco, California
    Senior DevOps Engineer
    $160,000

    Monster.com

    Citibank North America
    Tampa, Florida
    App Dev Tech Lead Analyst
    $156,000 to $177,000

    Indeed.com

    Brighthouse Financial
    Remote
    Principal DevOps Engineer
    $133,000 to $168,000

    SimplyHired

    Hellebore Consulting Group
    Fairborn, Ohio
    Senior DevOps Engineer
    $125,000 to $185,000

  • Codenotary Extends Dynamic SBOM Reach to Serverless Computing Platforms

    Codenotary Extends Dynamic SBOM Reach to Serverless Computing Platforms

    Codenotary has extended the reach of its platform for automatically generating software bills of materials (SBOMs) to serverless computing platforms running software constructed using functions.

    Codenotary CTO Dennis Zimmer said because serverless apps are dynamically created, it’s not possible to generate SBOMs using traditional approaches. The TrueSBOM platform makes it possible to create an SBOM in real-time by adding one line to the source code of any application. The TrueSBOM for Serverless edition of the platform now extends that capability to the serverless computing frameworks made available in the cloud by Amazon Web Services (AWS), Microsoft and Google.

    That approach makes the SBOM a part of the application itself rather than a text file that is stored separately, he noted. Each time an application is updated or a scanner discovers a new vulnerability, a new SBOM is automatically generated, added Zimmer.

    Awareness of the need for SBOMs has skyrocketed since an executive order issued by the Biden administration made it clear that federal agencies would require them from software providers starting next year. Many enterprise IT organizations are likely to follow suit as part of a larger effort to better secure software supply chains in the wake of a series of high-profile cybersecurity breaches.

    Most internal DevOps teams already have a good handle on what software components are used within the applications they deploy. The issue is that the organizations that use that software can’t easily verify what components are being used, noted Zimmer. That’s problematic because an organization may prohibit deployment of a specific software component because of a known vulnerability. TrueSBOM allows organizations that use an application to maintain control over their software environment regardless of what software artifacts are used to build closed or open source components, Zimmer added.

    It’s unclear how most organizations will operationalize SBOMs now that more of them are being created. Ideally, organizations should be able to approve only software with components that have been verified as secure.

    As SBOM mandates become more stringent, the number of application providers that are anxious to comply with rules for securing software supply chains should increase. The issue now is finding a way for both the developer and consumer of that software to streamline a verification process that, in its current form, is too cumbersome to manage effectively. As a result, many organizations are now putting together dedicated teams made up of cybersecurity professionals and software developers to manage SBOM processes, said Zimmer.

    Of course, there is a massive amount of code that has already been deployed that today doesn’t show up in an SBOM. While many emerging applications—created using artifacts such as functions and containers—are likely to be increasingly instrumented to dynamically generate an SBOM, the number of legacy applications that might need to be similarly instrumented is several orders of magnitude greater. As a consequence, the amount of effort required to generate SBOMs for every application in use might be greater than teams first anticipated.

  • Survey Surfaces Agile Development Challenges

    Survey Surfaces Agile Development Challenges

    A survey of 3,220 software development professionals published this week found that among the 80% of respondents that embraced Agile practices for building software, just under half are tracking on-time delivery (47%) and business objectives achieved (44%).

    The survey, conducted by Regina Corso Consulting on behalf of Digital.ai, a provider of a value stream management (VSM) platform. The survey also found a total of 19% of respondents were not sure what metrics were used to track software development progress.

    At the same time, 40% of respondents noted they are dissatisfied with Agile at their organization; a clash with company culture was listed as the leading cause (41%) of unsuccessful delivery. Half of respondents use a combination of Agile, waterfall and/or iterative processes, but 27% noted this combination of frameworks was a struggle for them.

    The survey also noted that just over a third of product and application owners (36%) that participated in the survey have adopted Agile.

    Derek Holt, general manager of Agile and DevOps at Digital.ai, said that percentage suggests that Agile principles are now being adopted beyond the engineering teams that construct software. As that appreciation for Agile principles continues to grow, the organizations these individuals work for also tend to start tracking value stream metrics, he added. That transition typically begins with a few pragmatic metrics before the organization matures enough to justify the adoption of a VSM platform, Holt noted.

    In time, however, that VSM platform not only helps organizations identify bottlenecks but also identifies potential quality issues and risks to the business, Holt added.

    Overall, the top three top Agile development priorities identified by respondents are to accelerate time-to-market (52%), delivery predictability (44%) and lower risk (31%). However, from a business perspective, the top three priorities are company goals (54%) and end customer satisfaction (43%) followed by time-to-deliver (35%).

    More than half (51%) of respondents, however, also noted that DevOps teams are not incorporating business feedback regularly. Well over half (58%) said the most valuable improvements to DevOps teams would be end-to-end visibility and traceability from business initiative through development, test and deployment to end users, while 55% said the ability to measure cycle time, wait time and bottlenecks would be an important improvement.

    Regarding Agile tools used, Scrum led the pack at 87%, followed by Kanban at 56% and Scrumban at 27%. The Scaled Agile Framework (SAFe) framework has been adopted by more than half of respondents (53%), while Scrum@Scale/Scrum of Scrums was adopted by 28% of respondents followed by Lean Management at 8%, according to survey respondents.

    The top benefit of Agile is increased collaboration (69%) followed by better alignment with the business (54%) and better work environment and increased visibility in the application development life cycle at 39% each.

    The debate over the merits of Agile frameworks, of course, remains fierce and no two organizations may ever have the same level of adoption. However, as the pressure to deliver quality software faster in the age of digital business transformation continues to mount, the more crucial it will become to align Agile and DevOps best practices.

  • Migration to AWS Cloud: Do’s and Don’ts

    Migration to AWS Cloud: Do’s and Don’ts

    What if one day, you were told to migrate to AWS/GCP/Azure/any other cloud provider? It would likely introduce some stress, to understate the case, right?

    I’ve seen quite a few migrations in my career and, in this article, I would like to share some thoughts to help DevOps engineers, architects, managers or anyone else who might be involved in such a migration. I’m focusing on migration to AWS mostly because I have a specialization in this subject and because it’s fairly common, but I think these experiences are applicable to any other cloud provider. For this case, I’ll cover migration from on-premises to a cloud because these types of migrations are much more difficult (in my opinion) than cloud-to-cloud migrations. I’ll also share my own perspective on do’s and don’ts of migrations. But please filter them through critical thinking and remember that everybody has their own background and experience. What’s good for one might be bad for another.

    What is a Cloud Migration?

    Let’s start by defining what migration is. How do you know if it’s a migration? I like the following definition: 

    Cloud migration is the process of moving a data center’s capabilities to IaaS providers.

    “Capabilities” is the keyword. A business does not migrate servers/virtual machines/hardware/data or anything else. A business migrates its capabilities.

    There are seven migration strategies (or seven Rs): Relocate, rehost (lift and shift), replatform, repurchase, rearchitect/refactor, retain and retire.

    For example, I’ve seen situations when a company was creating a new product that would replace an existing one hosted on-premises. The product would be designed to be well-architected and cloud-native from the very beginning and would be hosted in a cloud. Is that migration or greenfield project type? From the definition above, we can say that this is a migration. This is a rearchitecture strategy for migration.

    What if we buy a new SaaS subscription instead of some software we host on-premises? This is also a migration: A repurchase strategy.

    Let’s dig deeper into these seven migration strategies:

    Relocate. We host some workloads on-premises and move them as-is into the cloud. This is possible in limited cases; when we migrate cloud-agnostic workloads or when a platform might be cloud-native. Examples: Relocate a Kubernetes cluster from on-premises to the cloud or relocate VMware machines with VMware Cloud.

    Rehost. This is also known as lift and shift. We convert virtual machines/servers on-premises to virtual machines in the cloud. Examples: Migrate on-premises virtual machines to EC2 instances with CloudEndure, or create EC2 instances, install the software and apply the same configuration as on-premises.

    Replatform. We migrate a workload to a cloud-native platform without rearchitecting systems. Examples: Migrating PostgreSQL to AWS RDS or Docker containers to ECS.

    Repurchase. We replace some custom/legacy system with SaaS. Example: Replace custom on-premises email system with SendGrid or replace CRM with Salesforce.

    Rearchitect. We change your application’s architecture to be cloud-native and use managed cloud services. That also includes rebuilding from scratch. Examples: Change the code of your apps to upload files to S3 instead of local storage or containerize an app and migrate to ECS.

    Retire. We decide that a piece of the workload will be eliminated if not needed. Example: Log servers are no longer needed as migrated applications use CloudWatch.

    Retain. We leave it on-premises. Usually, it’s temporary. Example: A huge Oracle database with a lot of functions and customization is not moved to AWS because of the massive costs. It’s decided to keep it on-premises until it’s replaced with another solution in the modernization phase.

    I like the following picture that shows the high-level stages of each migration type. It may be a helpful cheat sheet to approach migrations.

    Picture Source: https://aws.amazon.com/blogs/enterprise-strategy/new-possibilities-seven-strategies-to-accelerate-your-application-migration-to-aws/

    Migration Don’ts

    I don’t recommend the following when performing a migration:

    Migrating too quickly. A business might think, “We have to migrate to the cloud quickly. Let’s do lift and shift right now!” It’s better not to start right away. Are you sure that rehosting strategy will really be faster than some other strategy? In one instance, we tried to migrate on-premises VMs, but it didn’t work because the OS on the machines was quite old. Some OS-level dependencies didn’t work on AWS’ hardware. This required a lot of configuration work, custom code and testing and the initial estimates were completely wrong. It eventually turned out to be a replatform. 

    Assess what you have, analyze it, analyze the business case and think. During this engineering work, we should stop and think before we act.

    Migration without clear goals. Why are you migrating? If you can’t answer the question, you probably need to find an answer or not migrate at all. I’ve seen situations in which businesses wanted to migrate to the cloud to reduce costs. They performed a lift and shift migration without doing a total cost analysis. Only after the fact did they realize that it cost even more than when it was in a colocation center. This migration may be considered unsuccessful.

    To avoid this, define clear goals and define the target state that achieves these goals. If costs are the driver, make the proper calculations and plan accordingly. Cost optimization will be the main thing to focus on when you plan your target state. If availability is the reason, measure it, define SLOs and focus on availability in a target state.

    Black box migration. Don’t treat workloads you migrate as a black box. It is almost critically important to know what exactly you migrate, what technical requirements it has, what dependencies it has and how it works. That information will allow you to choose the correct strategy for workload migration. If this is not done, there’s an excellent chance that you will choose the wrong path and the result will be lost. Capturing information about your systems is the most important task when planning.

    Don’t forget well-architected. Migration is stressful. That’s why people tend to try to get through it quickly on as short of a timeline as possible, focusing on the only thing: Making everything work in the cloud ASAP. They set aside well-architected framework pillars. This is a mistake; it may lead to a different set of negative consequences of different severity. Don’t ignore well-architected framework pillars, or you won’t know about potential issues or risks until they have already materialized.

    Use one strategy for everything. It’s always easy to make a single overarching plan for all workloads. For example, deciding to do lift and shift and then rehost everything in the cloud. But what if some things could be retired? What if some components could be replatformed with little effort? You will save significant resources by analyzing all of the components of the workload and deciding on a unique strategy for every component. Collect data and plan properly.

    Cloud Migration Do’s

    Do perform a cloud readiness assessment. If a person migrates to a new location, they should consider:

    • That the climate will be different
    • The language spoken in that location may be different
    • The cost of living will be different and perhaps more expensive

     When a company migrates to the cloud, they should ask questions including: 

    • Have we considered all the risks?
    • What must be changed for the workload to actually work?
    • Do we have the human resources required for all of the work?
    • Do I have the budget to accomplish everything? 

    The Cloud Readiness Assessment is a long-form interview about varying processes in the company that identifies whether a company is ready to live in the cloud. It’s based on the Cloud Adoption Framework (CAF) and allows us to understand what should be changed so that a migration can happen successfully. AWS has a Cloud Readiness Assessment tool which covers all of that. I urge you to use this process for all workloads being considered for migration.

    Create a business case. Describe the business case. What are the drivers for migration? What problems are resolved with the migration? What goals should be achieved by migration? Be specific: If you intend to reduce costs, then calculate the sum. If you intend to improve availability, then define a clear SLO. That will be the number-one driving force for your migration. All other artifacts should be based on it.

    Automate discovery and data collection. Use automated data collection and analysis to create a portfolio for migration. Don’t base your analysis on manually created spreadsheets or verbal communication. Let machines collect data about inventory and usage statistics. The more precise data you collect, the better. AWS Migration Evaluator service will help you to do this in most cases.

    Portfolio management. Create a portfolio of workloads to migrate, analyze them, choose a migration strategy for each, choose some for the pilot/first wave migration, estimate timelines and prepare a plan. Execute the migration after that. Repeat until the migration portfolio is empty. Share the data. Have subject matter experts review it.

    Found CCoE. This is part of the CAF mentioned above. Before executing the migration, create a team of people responsible for creating a strategy and defining a good standard of living in the cloud: A Cloud Center of Excellence. It should include people with a variety of subject matter expertise including management, operations, platform, people, finance and every other department consuming or providing services in the cloud. These people will prepare a landing zone for the migration, hard policies and softer policies for the landing zone; preparing the company for life in the cloud. Don’t fear revisiting any policy.  Make changes as new factors come to light.

    Well-architected framework reviews (WAFR). Do well-architected framework reviews (WAFRs) during migration waves and after them. That will allow you to avoid foolish mistakes and to get a high-level overview of what you have done from a technical perspective.

    Modernization plan. Prepare a modernization plan along with the migration plan. Migration isn’t the end of the story. To be effective in the cloud, you will have to do well-architected reviews and modernization. The first modernization is likely to be huge; that’s why it’s important to plan and to schedule budgets and resources in advance. Failure to do this will lead to increased operational costs, not decreased costs.

    Summary

    If we want to reduce the risk associated with a migration and to make it less stressful, we do not rush. You may believe that rushing will save you time and other problems will be managed later, but experience tells us that this is wrong. The most likely outcome is that you will end up resolving all the issues encountered in the most stressful situation possible: When you cut over to your new cloud-based production environment. Instead of resolving issues in a well-considered way and in advance, the choices will be made under duress. Proper engineering principles must be applied here. Picking up a bridge and moving it from one river and plunking it down on another never works. The key to migration success is to measure, analyze, design and plan. Migrating a workload to the cloud is hard, but I hope these do’s and don’ts will make it easier.