devopsy.kinsta.cloud

Category: DevOps Practice

  • How Does That Stack Up?

    How Does That Stack Up?

    There’s an entire class of successful innovations made up of things that people didn’t realize they needed. Things that were an obstacle they found a way around and didn’t think about until the innovation showed them an easier way. Think of things like virtualization (hardware was the bottleneck, but “That’s just the way it is,” until it wasn’t) or web monster shopping sites (“They don’t have that in the stores; I guess I’ll live without,” until you could order the weirdest or smallest things from the likes of eBay and Amazon).

    My rule here is that I only talk about products and services if I use them and find value in them. And even then, I try to do it sparingly. But in the spirit of the innovation story you just read, I’m going to point you at one of the coolest new websites on the net. I don’t know the company; have not done business with them, couldn’t even name a principal. But I have spent a surprising amount of time on their site, both in the Feed section and Browsing Stacks.

    The site is Stackshare.io, and the reason I think it is so innovative is what and who it is aimed at: Listing companies’ DevOps stack and offering advice on different parts of the DevOps stack.

    The Browse Stacks section is for presenting what other organizations are doing, so you can get good ideas for your own stack. But the developers of the site appear to have recognized the other useful aspect of this information: Matching organizations to prospective employees based on technologies. You can go look and see that Company X is using GitLab, for example, and that can then influence your job search.

    The Feed section is just a running discussion that uses categories to ask questions or offer insight and advice about decisions a given organization came to—so, “Chose X over Y” is actually a standard subheading for posts, as is “Need advice on Z” or “Recommends A”. It makes for great reading, and no doubt while reading through, you’ll have some advice to offer one of the “Looking for advice” posts.

    The Stackshare.io participants seem to range in size from startup to massive, and that is one of the things I hope they change as they grow—the options for looking at DevOps stacks are “Popular” and “Trending”. We all want more filters than that—organization size, number of apps, stuff like that—to help us zero in on environments that are similar to our own. My little company does not have the needs that Facebook does. And while it’s true that some startups use more tools in their toolchain than Facebook, you get the idea.

    Another issue I had with Stackshare.io’s site (that I fully understand, but still wish could be different) is forced registration/login to see some data. It’s not even particularly sensitive data—I know what ELK and Azure Data Services are, so putting the definition/description behind a registration page/login is kind of silly. But every free site needs data to pay the bills, and I suppose I’m glad it is info that’s also available elsewhere that is behind a reg wall.

    The company offers APIs for integration to your site/tool and private company instances of StackShare to help you manage one of my other gripes—the five (or fifty!) different DevOps stacks many organizations have allowed to develop in-house.

    So if you are job hunting, Stackshare.io can help you find companies doing what you would like to do or know you’re good at. If you are an organization trying to improve or set up a new toolchain (which is how I found the site), it has a lot of information to digest, though not all of it is easily accessible at this point. Either way, keep kicking it and keep an eye out for tools that do the job, no matter where you are looking.

  • The Next Phase of Pandemic Tech Transformation

    The Next Phase of Pandemic Tech Transformation

    Last spring, we were inundated with headlines, tweets and Medium posts applauding the rapid digital transformation that businesses were going through—almost overnight—due to the pandemic and the shift to remote work. It made for an interesting feel-good narrative at the time, but it didn’t paint the full picture. Companies were thrown into the deep end; many barely staying afloat by implementing quick-fix, surface-level changes that they assumed would only be needed for a short time. Turns out, the “transformation” touted was just the tip of the iceberg.

    More than a year later, the workforce is still in flux. The reality is, some form of distributed, hybrid work is likely here to stay. Employees will continue to expect remote options and customers will want seamless, digital ways to interact with businesses, pandemic or not.

    As a result, companies must move past the temporary, band-aid solutions of last year and make those sweeping changes permanent. It’s time to move forward with true, sustainable change: Technical, experiential and, most importantly, cultural. Every company will need to become a modern technology company capable of delivering first-class digital solutions to customers, partners and employees—or risk becoming obsolete.

    Transactional Vs. Transformational Change

    The good news is just about every company understands the value of leveraging technology. A recent 2021 State of the CIO survey found that 82% of CIOs said they implemented new technologies, new IT strategies and/or new methodologies over the past year. The issue is how companies go about their technology deployments.

    People talk about digital transformation all the time, but it’s been used so much that it has become an unhelpful misnomer. More often than not, the change that companies are implementing isn’t truly transformational, but rather transactional or just slightly incremental.

    When companies have a transactional mindset, the focus is on immediate, simple and predictable changes. The purpose—or, more importantly, the outcome—is simply to continue business-as-usual. This is reminiscent of many companies last year turning to technology just to keep the lights on. A transformational mindset, on the other hand, aims to disrupt this business-as-usual and fundamentally change the paradigm in how the company operates through ways of working, use of technology and culture. It’s purposeful, aiming to solve business problems holistically and spur long-term growth.

    To be successful in today’s climate and to gain a competitive edge with customers, companies must strive to be transformation-focused.

    How to Move Toward True Transformation

    Here are three examples of how companies can shift from a transactional mindset to a transformational mindset.

    Cloud Migration vs. Modernization

    Often, when companies talk about their use of cloud, they tout that they’ve “moved to the cloud.” But all too often, it’s treated as a one-time event; a destination to reach and a box to check. At the outset of the pandemic, we saw so many organizations quickly mobilize to shore up their digital strategies and ensure they were thinking about the limitations of doing business in a virtual context, but their cloud strategies didn’t evolve to rethink these paradigms, as well. They simply accelerated their migration strategies to get out of their physical data centers and into the cloud.

    The opportunity to leverage the cloud is so much bigger than that. It can unlock greater efficiency, flexibility and growth and underpin new digital experiences and revenue opportunities—exactly the kind of strategies most companies envision. For example, health care leader InnoCare was struggling with outdated clinic management software. Instead of just moving the old bits to their cloud provider, they adopted a broader digital strategy to modernize their delivery capabilities and reimagine both the patient and clinician experience, setting new industry standards in the process. This is what we refer to as modernization and it is what leads to transformation.

    Software Engineering vs. Product Engineering

    With the rapid growth of software, it’s no surprise that software engineers are in high demand. The U.S. Bureau of Labor Statistics projects that software engineering job openings will grow 22% through 2029. And while software engineers will play an irreplaceable role in helping build innovative solutions, transformational-focused companies know to bring more people under the tent. By combining software engineers with other cross-functional experts who bring a different level of artistry, design and audience understanding, the end result is a more engaging, customer-centric product that evolves and grows over the entirety of its useful lifespan.

    This is exactly what fitness giant Nautilus did. They brought together a team of engineers, data scientists and machine learning experts to help bridge the gap between their equipment manufacturing prowess and their digital experience and software development goals. Collectively, they produced a total platform that integrated hardware, software and data to create a dynamic workout regimen that impressed users and fended off competitors.

    Moving Beyond Proof-of-Concept to Long-Term ROI

    A proof-of-concept (POC) is a rite of passage in the tech world. It’s how a company or team demonstrates that their idea is feasible and can work in a real-world application. But POCs alone are insufficient; they are transaction-minded and don’t focus on achieving lasting business outcomes or generating deeper ROI. Rather, they mostly focus on technical feasibility. The key to upleveling toward transformation is by focusing on the processes, not just the end product. When testing features in a real-world setting, make sure that you’re proactively measuring not just that they work—but that they contribute to business imperatives, meet customer expectations and strategic outcomes. And if they don’t, go back to the drawing board and optimize with those higher-level goals in mind.

    Culture is Key

    As you can see, the difference between a transactional and transformational mindset is more about culture than technology. A culture of innovation and modern ways of working leads to transformation that fundamentally shifts the way technology is conceptualized and created. It isn’t solely about gaining efficiencies or cutting costs—instead, the focus is on making the lives of customers, employees and everyday users better. Transformation-focused companies believe in the promise of technology advancements, but they believe that it also can make operations smoother, customer relationships more meaningful and increase revenue when implemented with the right combination of skills, disciplines and organizational buy-in over time. It’s those companies who will thrive in this new era.

  • SRE’s Guide to Pragmatic Incident Response

    SRE’s Guide to Pragmatic Incident Response

    In my past experience as an SRE, I learned some valuable lessons about how to respond to and learn from incidents. If you want the TL;DR, I’ll summarize them here:

    • Declare and run retros for the small incidents. It’s less stressful, and action items become much more actionable.
    • Decrease the time it takes to analyze an incident. You’ll remember more, and will learn more from the incident.
    • Alert on pain felt by people – not machines. The only reason we declare incidents at all is because of the people on the other side of those machines.

    Now, let’s dive into each of these lessons a little deeper, and explore how they can help you build a better system for pragmatic incident response.

    1. Focus on the Small Incidents First

    The bad habit of ignoring small issues often leads to bigger issues. You should run retrospectives for small incidents (slowdowns, minor bugs, etc.) because these often have the most actionable takeaways—instead of shooting the moon and creating “rearchitect async pipeline” as a Jira ticket that never happens. Focusing on low-stakes incidents and retrospectives are a great introduction to behavior change across your organization.

    Let’s look at an obvious example of how a focus on small incidents can have an outsized impact. Case in point—my apartment. I live in an old candy factory retrofitted into apartment units. We have an elevator (thank God), but some of the buttons don’t light up when you press them. The LED display doesn’t match up with the numbers on the floor buttons. Yes, the elevator goes up and down, but overall, you can tell that things are wrong with it.

    One day, I came home and noticed an “Out of Order” notice on the elevator door. Was I surprised? Not at all. Of course an elevator with mislabeled buttons and broken LEDs would stop working!

    This isn’t dissimilar from an important software lesson—ignoring small issues often leads to much bigger issues. What starts as an inconvenience can lead to a complete breakdown. This is why you should focus on the small incidents first. I hear a lot of companies say, “We need to fix incident response! Every time we have an incident, it’s just chaos!”

    They are talking about unexpected downtime during a high severity level incident and they want to improve it.

    I say that’s the wrong type of incident to use to fix your incident response system and process. Focus on the small incidents first. Running retros for small incidents can help you build strong incident response models, because they have the most actionable takeaways and they are the best way to change behavior through repetition and practice.

    If you have a high-stakes, 12-hour incident and you run a one-hour retrospective you’re not going to get the results you want. You need to start small. Run retros for bugs that were introduced; maybe a bad data migration that didn’t really impact anything but took up a couple of hours of your day.

    Heidi Waterhouse captured this idea really well in this piece on reliability. Every airplane you’ve ever flown on has many tiny problems, Waterhouse said, “… like a sticky luggage latch or a broken seat or a frayed seatbelt. None of these problems alone are cause to ground the plane. However, if enough small problems compound, the plane may no longer meet the requirements for passenger airworthiness and the airline will ground it. A plane with many malfunctioning call buttons may also be poorly maintained in other ways, like faulty checking for turbine blade microfractures or landing gear behavior.”

    I couldn’t agree more. Extrapolating to software: The small things are typically indicators of bigger issues and could cause catastrophes down the road.

    2. Track Mean Time to Retro (MTTR)

    It’s important to think about what you measure in your organization. You should be measuring how you’re improving, and the most important metric here is mean time to retro (MTTR). Everyone should be tracking MTTR. It’s a great statistic to improve incident response in your team because it helps you understand the delay between incidents and retrospectives.

    The easiest way to have a bad incident retro is to wait two weeks. It’s better to get into a room quickly and hash out what happened than wait a long time until you’ve got everything perfectly prepared.

    Tracking MTTR can help you hold prompt and consistent retrospectives after incidents. Set a timer and make an SLO or SLA for yourself that says, “This is how long we take for retros.”

    Retro time will vary depending on the severity of the incident itself. If it’s a SEV1, clear everyone’s schedules, because you need to have retro within 24 hours of the incident. For SEV3 incidents, you have much more leniency.

    I also like tracking the ratio of retros to declared incidents. This is a metric that should go up; you should see your ratio of retrospectives to incidents increasing. You can break down that number by severity, as well. If you have a low retro ratio for a SEV1 versus a SEV3 incident, that might be okay at first (remember, start small) but you want those metrics to eventually become equal.

    3. Alert on Degraded Experience with the Service, and Not Much Else

    The severity of incidents is directly linked to customer pain. We would not declare SEV1s if there weren’t a lot of people feeling a lot of pain.

    Alerting on computer vitals is an easy way to create alert fatigue and burnout. As your company starts to scale, you are going to use more CPU; you’re going to use more memory. Tying alerts to computer vitals is not a good sign.

    If I run, my heart will beat faster—it’s just doing its job. Paging people at 2:00 a.m. because disc capacity is at 80% even though you won’t run out of space until next month is a good way to lose great teammates. I have worked with people who left their previous companies strictly because they got paged too many times for stuff that didn’t matter.

    This is why you need to alert on a degraded experience with the service and not much else. A CPU burning hot at 90% is not necessarily a bad thing, but you need context to decide. Create SLOs that are tied to customer experience and alert on those. People experiencing problems with the service is the only thing you should alert on—for the most part.

    One of the best ways I’ve seen to think about this came from Soundcloud developers, who explained that you should alert on symptoms, not causes. My fast heartbeat is not necessarily a problem. But if my elevated heart rate leads to lightheadedness and I fall—that’s a problem. So, I need to be able to alert on something like that. Paging people at 2:00 a.m. because disc capacity is at 80% and you won’t run out until next month is not good. But paging people because you know that disc capacity problems cascade into other, systemic problems is. You can apply the same thought to other potential causes of an outage. Paging alerts that wake you up in the night should only be based on symptoms.

  • The Key Benefits of Observability for SREs

    The Key Benefits of Observability for SREs

    In today’s technology landscape, organizations strive to champion innovative ideas, techniques and technologies to achieve success and outshine their competitors. For this reason, site reliability engineering (SRE) has become one of the fastest-growing enterprise roles and a set of organizational practices for fast and reliable software delivery. 

    SREs use various tools and practices at their disposal to manage services at scale, such as observability. Observability is the ability to infer a system’s internal state(s). It provides actionable insights into when errors occur within a system and, more importantly, why they occur. For SREs, this actionable data can be important to providing secure and reliable applications. 

    To gain more insights into the benefits of observability for SREs, I asked SKILup Day participants and DevOps Institute Ambassadors to weigh in. Here’s what they shared: 

    Sponsor, Priya Satheesh, CEO, Instana 

    “As their name implies, site reliability engineers are tasked with keeping the applications, architectures and websites up and available. Observability solutions put alerts in front of them at the first indications of trouble, with proper context and information so they can take action before issues become incidents and start affecting their customers. In addition, AIOps and machine learning help them to predict events and standardize responses for common occurrences to reduce response and repair times.”

    Helen Beal, chief ambassador, DevOps Institute 

    “Embracing observability supports SREs’ goals by:

    • Reducing the toil associated with incident management—particularly around cause analysis—improving uptime and MTTR.
    • Providing a platform for inspecting and adapting according to SLOs and ultimately improving teams’ ability to meet them.
    • Offering a potential solution to improve when SLOs are not met and error budgets are overspent.
    • Relieving team cognitive load when dealing with vast amounts of data–reducing burnout.
    • Releasing humans and teams from toil while improving productivity, innovation and the flow and delivery of value.
    • Supporting multifunctional, autonomous teams and the ‘We build it, we own it,’ DevOps mantra.
    • Completing the value stream cycle by providing insights around value outcomes that can be fed back into the innovation phase.”

    Mark Peters, technical lead, Novetta

    “True system observability means any process or event within any operational or development phase can be monitored by SREs. Systems with built-in observability provide SREs, as subject matter experts, the opportunity to examine events from different perspectives. The more observable the system, the more different places can be examined to help improve the overall flow. SREs can then create the feedback and design experiments to observe from different perspectives. If the event is showing functional difficulties and the only monitoring point is the input/output of the function and there is no process observability, SREs have to go back to the beginning to find events. If they can observe the entire process chain from commit to deployment, one can find the different areas. Think about the A/B cycle: Limited deployments are planned, and the load shifted gradually to ensure the new system can handle the increased functions. If the SRE cannot observe the function, then they cannot help improve reliability.”

    Ryan Sheldrake, field CTO, Lacework 

    “If you can ‘see’ or observe the complex and ever-changing environments that SREs are tasked with managing, creating and perhaps destroying, you can manage the associated risk posture associated with those environments and corresponding workload.“

    Vishnu Vasudevan, head of product engineering and development, Opsera

    “The top benefit for site reliability engineering (SRE) teams is the opportunity to work collaboratively with the business stakeholders and technology stakeholders to set goals and continuously improve. Based on these goals, SRE’s can create an approval process, which will help them understand where they are with respect to their own planning, security policies, quality metrics and operations practices. The ability to measure every release and verify the project helps improve speed to market and other factors once a robust observability practice is achieved. With DevOps orchestration across all the DevOps tools, applications being released and the different teams involved, SRE teams can then turn observability practices into a competitive advantage.”

    Sushant Mehta, senior manager, application development, Diyar United Company

    “Using observability, SREs can achieve a number of objectives in their day-to-day work, like identifying the root cause of production issues, faster resolution of issues and striving for self-healing infrastructure setup with no-code.”

    Tiffany Jachja, engineering manager, Vox Media

    “Every team that delivers software should be taking responsibility for their contributions to a software service. For SREs, having observability into an application means having the systems in place to help developers gain insights into how software applications function. Ultimately, it’s about building better software.”

    Parveen Arora, co-founder and director, VVnT SeQuor

    “SREs teams can leverage observability to get the following benefits:

    • To keep SLOs intact by detecting customer-affecting issues faster and rolling back before an issue affects the SLOs.
    • To have real-time health updates and transparency of information with regard to a service’s status. 
    • To create better workflows for debugging, optimizing workflows and resolving issues rapidly.
    • To simplify root cause analysis and investigation of hypotheses.”

    Jose Adan Ortiz, solutions engineer, Akamai Technologies

    “SREs are mainly concerned with the reliability of systems and usually monitor practices of toil, SLO, error, budget and stability to maintain systems working as expected.

    You could ask, ‘How can an SRE achieve such an integrated view of all systems components?’ The answer is observability. Without observability tools, it was impossible for an SRE to provide an effective response, fast RCA and analysis of data to improve SLOs.”

    Supratip Banerjee, solutions architect, Principal Global Services

    “It can provide several advantages, including:

    • Detecting customer-impacting errors faster and reverting before SLOs are broken.
    • Fostering transparency and delivering real-time updates on the status of a service. This allows SREs to be more productive and saves a lot of time.
    • Developing better methods for debugging, optimizing and quickly addressing issues.
    • Allows for feedback loops, which are crucial in the SRE role.
    • To avoid downtime, the relevance of observability rises in real-world production systems. There should be proper notification processes in place.”

    Maciek Jarosz, DevOps and process expert

    “The benefits are quite great, I’d say. Unless one is living under a rock with one local system, then it’s rather inevitable that there will be dependencies here and there. And if you’re an SRE, then I’d say that you’d rather know a bit more about dependencies from other branches of your system—or other systems altogether—than less.

    Learn more about observability and similar topics by registering for an upcoming SKILup Day. 

  • DevOps World 2021: 3 Sessions for Successful Scaling

    DevOps World 2021: 3 Sessions for Successful Scaling

    Your business has grown from a startup of five people to a full-blown organization of 100 … congrats! But, now what? The practices that worked for a smaller company may have worked as you first started growing, but now it’s time to scale operations to fit your new needs. Just like a kid outgrowing their clothes, your organization needs a new “wardrobe,” and scalability is key for long-term success.

    This year’s virtual conference—scheduled for Sept. 28-30—will offer a number of breakout sessions, panels and keynotes focused on scalability within enterprises. If you’re ready to scale within your business, here are a few sessions from this year’s conference to help get you started.

    From Startup To Enterprise—Sustainably Scaling

    Many organizations struggle to scale DevOps culture as they grow. Now it’s time to say goodbye to the startup phase and hello to thinking like an enterprise.

    Alston Rousseau, director of software delivery at Campspot and Kailene Cloud, director of talent acquisition at Bonusly, will walk attendees through the investment needed to sustainably scale operations, development and culture. Plus, they will share tips for finding more value in existing commitments to money spending, time and diversity in DevOps and how this approach applies to organizations with and without high compliance requirements.

    Scaling up Car Configuration: From a Statically Hosted Front End to Universally Rendered Services

    Over the past year, tech company Code Orange made the transition from a traditional API/SPA split to a much more technically challenging server-rendered SPA with full cache handover.

    In this breakout session led by Niels Hoiting, a developer at Code Orange, attendees will learn more about Code Orange’s transition from API/SPA split to server-rendered SPA while analyzing the pitfalls that can arise in the transition and how to optimize compute spend to best handle a specific workload.

    Leveraging CloudBees to Scale Satellite Ground System Emulation in Multifaceted Virtual Environments

    Aerospace and defense technology manufacturer Raytheon Technologies has successfully implemented CI/CD pipelines for the test and near-operational integration of both software and hardware for a global satellite ground system.

    In this breakout session led by Casey Jackman, deputy of GPS Product Factory at Raytheon, attendees will discover how Raytheon uses simulated satellite control stations in virtual private clouds for CI, test and check-out. Additionally, Casey will share how the team met challenges in initial pipeline generation between multiple disparate target environments with AWS and on-premise data centers.

    Think it’s time to scale your organization? Register now for DevOps World 2021 to get started.

  • How to Achieve AWS Competency Accreditation

    How to Achieve AWS Competency Accreditation

    For businesses in the early stage of their ventures, proving that they have the expertise and technical know-how when it comes to cloud technology can be difficult. An acceleration toward digitization due to COVID-19 and an increasingly crowded cloud market, dominated by the big players, means that simply shouting about your skills in cloud engineering and DevOps isn’t enough to get noticed.

    But what if you can work alongside these giants? Joining forces with expert brands and entering into endorsement partnerships can propel your business forward by supplementing it with seasoned expertise. Accreditation, if you can achieve it, can give you a stamp of approval and reassure prospects that you have the relevant knowledge to support their digital transformation efforts. Collaborating with industry leaders on skills accreditation programs is a great way to demonstrate your capabilities and provides clear goalposts to work toward when it comes to building and learning.

    So, how do businesses achieve this status and what are the long-term benefits of the process?

    Become an AWS Partner

    Achieving accreditation as an AWS DevOps Competency Partner takes a lot of hard work and investment into building the technical skills required to reach such a high standard–logging and evidencing every action your team takes is no mean feat. From beginning to end, the course takes around six months. The process begins with researching and selecting which specific competency your team is interested in achieving. It’s important to understand which competency your business is most suited to, so doing your homework is key. From there, there’s a lot of work to evidence, certificates to achieve and reviews to undergo. Becoming an accredited AWS partner brings a multitude of benefits to your brand, so it is well worth the investment.

    Top Tips for the Accreditation Process:

    Create detailed accounts of your work: To become a certified DevOps partner, businesses need to evidence four case study examples; two public and two private. As such, it can be useful to bring in a tech writer to consolidated your notes and observations throughout the process. We began, for example, with a technical exploration from the team to understand the best case studies to put forward; our work had to be clearly evidenced with strong examples and the documentation to back it up which took a considerable amount of time to pull together. Maintaining clear and detailed accounts of previous work can prevent any delays. This information is firstly analyzed by AWS to check that it met the specific requirements before being passed on to be examined by a team of external auditors, so every detail counts when it comes to showcasing your work.

    Prepare the marketing team: As part of the process, businesses are required to have certain marketing collateral in place across their website to aid with lead generation. For the DevOps competency certification, we needed to evidence three technical blogs–businesses applying for the certification need to ensure that the content best reflects their expertise in the given competency. Once the accreditation is finalized, the partnership should be displayed on the site. Even if a business is tech-agnostic, demonstrating your skills in a particular type of cloud engineering is highly valuable to your brand and AWS is a respected platform to use.

    Advance technical training in the workplace: Having members of the team qualified in cloud engineering is another requirement for accreditation. For DevOps, we needed eight people on the team to have achieved their associate certifications and four to have achieved professional certifications. To fulfill this requirement and ensure we’re on top of our game when it comes to cloud training, we’ve invested in third-party tools, provided access to LinkedIn Learning programs and we give our people additional time off from client-facing work to prepare for their exams. Building and learning in tandem is the fabric of our culture, and a similar approach would benefit any company looking to achieve certification status.

    The Benefits of AWS Accreditation

    Being able to showcase a close association with a major player such as AWS opens up a number of doors for smaller businesses. Operating within this kind of business ecosystem provides the opportunity to demonstrate a business’ DevOps and technical skills to an enormous global audience which can drive new business conversations and drive quality leads. Confirmation that your business is a trusted partner for a major cloud organization will ensure that prospects have confidence in your capabilities and you’re taken seriously as a technical player in the market. Building confidence within your teams and supporting them in their learning is an investment that will pay huge dividends.

    Working alongside AWS as an accredited DevOps partner gives us a springboard to talk about our work and engage with other players in the market. The accreditation is a fantastic talking point and definitely gets you noticed in the cloud community–plus, you will be taken more seriously as a partner.

    Investment in Cloud Tech is Key for the Future

    As businesses everywhere begin to operate in an increasingly digitized environment, technical skills and cloud expertise are a necessary investment for any business. Working toward coveted industry accreditations while prioritizing long-term skills training is imperative. It’s a competitive, buoyant market for cloud engineers and DevOps at the moment; opportunities are at their fingertips and if they aren’t feeling fulfilled by their current work, they’re likely to look elsewhere. Investment in training and nurturing growth in this way breeds employee engagement and loyalty. For a company to thrive technically, it needs to host a wide range of diverse talent and be able to retain it. Learning opportunities should always be the key focus.

    The shortage of this talent means it’s critical to advance the business’s cloud expertise by pursuing recognition from major global players like AWS. Providing advanced training and certification opportunities for your people will ensure long-term talent retention while making the business a leader in cloud engineering. That’s pivotal for long-term success.

  • Building DevSecOps With Value Stream Management

    Building DevSecOps With Value Stream Management

    As teams develop software, testing for potential security risks and flaws is mission-critical, just as it has always been. Yet, cyberattackers continue to exploit code vulnerabilities as open source code remains the foundation for the majority of commercial applications across all industries. Security firms are churning out solutions at an aggressive rate, and both project managers and their teams hope they can purchase some bolt-on tool or license a service to take care of security, painlessly. In the end, vulnerabilities still occur.

    The most effective pathway to secure software code has always been to eliminate vulnerabilities. The question today is, “What is the best way to achieve that goal given such an accelerated, threat-laden landscape?” Before we dig into that answer, let’s take a closer look at the problem.

    Software Security Is Everyone’s Issue

    Even the largest organizations, from Adobe to Microsoft, have been humbled by the number and scope of code vulnerabilities that have been overlooked—and therefore escaped into public release—due to the frenzied pace of software updates. Here’s just one example:

    On May 11, 2021, Adobe issued a security update for Adobe InDesign that addressed multiple critical vulnerabilities. “Successful exploitation could lead to arbitrary code execution in the context of the current user,” the security bulletin read. “Arbitrary code execution in the context of the current user” is cybersecurity speak for “an attacker can execute arbitrary commands or code on a target machine or process.” Arbitrary code execution vulnerabilities are known to be one of the top attack vectors that lead to system penetration resulting in data breaches.

    Given the enormous effort companies and their software teams are exerting today, and the importance of avoiding software flaws in the current threat landscape, how is this happening? It doesn’t help to have publicly available lists of software vulnerabilities posted on the dark web and CVE (ironically, a database of vulnerabilities launched to help companies improve their cybersecurity). Hackers — from solopreneurs to nation-state teams — are having a field day with this information.

    Rearranging the Puzzle Pieces Doesn’t Guarantee Security

    In an attempt to resolve this challenge, industry professionals are coining new terms and calling for new approaches. One I have seen promoted recently is “SecDevOps” rather than “DevSecOps.” With SecDevOps, they assert, security is first—conducted at the start of the project and maintained all the way through.

    This approach isn’t new—it’s essentially a variant of shifting left, which for more than a decade has been an established mechanism in software testing for finding flaws and “failing faster.” In my experience, while failing earlier and faster with any software endeavor has considerable value, it is not automatically going to highlight all security challenges and enable their resolution.

    It is just as important for DevSecOps teams to collaborate and work together, not only to support development and operations, but also to ensure security, compliance and governance are built into every piece of code.

    Insight Is Preferable to Blind Interpretation

    Unfortunately, achieving that collaboration productively comes with challenges. DevSecOps teams need rich, real-time visibility into code vulnerabilities, early, so they can act on that information and update code as it moves through the pipeline. To do that, they need actionable insight—not a motley assortment of data that’s open to interpretation.

    In value stream management, we have seen firsthand while working with our clients that several techniques can be applied to eliminate vulnerabilities more quickly and effectively. All of these approaches hold value, whether a firm is seeking to deliver a new product to market or secure its current software pipeline.

    • Amplify feedback loops. Continuous feedback loops—from DevSecOps tools, for example—give developers more detailed insight into their code’s vulnerability.
    • Run security scans. Give testers the ability to run security scans on their code even before it is committed to identify security vulnerabilities as early as possible.
    • Leverage reporting and analytics. Gaining early access to as much insight as possible makes teams and management more confident regarding swift action to resolve code deficiencies.

    These approaches also support governance and compliance initiatives. As a bonus, they boost software quality, as well.

    The Business Case for Built-In Security, Governance and Compliance

    As mentioned earlier, these activities are integrally connected with the effort to eliminate code vulnerabilities. In case you are thinking, “This is obvious,” take a moment to reflect on this statistic: 60% of data breaches involved vulnerabilities for which a patch was available but not applied.

    The sheer scope of the problem also underscores the lack of attention to security and governance—and any compliance requirements that accompany it.

    Half of websites in production have high-risk vulnerabilities, and 82% of these are located in application code.

    The high percentage of errors in source code suggests it was not checked for vulnerabilities during development, signaling that developers focused on app functionality and gave short thrift to security. This often means they don’t have a program for governance and compliance, either.

    In my experience, a primary reason for these failures is that teams don’t have actionable information from the activities mentioned above, such as real-time feedback from DevSecOps tools. Developers need this insight when they are resolving sprint backlogs. Information relating to high-level security issues must be available so it can be addressed, with issues prioritized and resolved, and not left sitting in siloed DevSecOps tools.

    In many ways, DevSecOps is in the same stage of transformation as DevOps was a decade ago. Teams are frustrated by needs they do not know how to meet, and they hope for fast resolution in the form of tools and solutions. As with virtually every other software challenge over time, there is no replacement for getting in the trenches and fixing the problem rather than placing strategic band-aids.

    If you are not subject to compliance mandates and therefore dismiss the value of software governance, consider this: If your team misses a software vulnerability that results in a major data breach because you didn’t have the insight to identify it—or the governance to ensure it was addressed—your firm may not recover. The imperative for software security—and the governance that helps ensure it stays that way—should be sufficient for organizations to take vulnerabilities seriously.

    Effectively applying the insight gleaned from software code reporting and analytics is the surest way to eliminate vulnerabilities. As an added benefit, the reporting and analytics that informs the software effort can also make the case to upper management for the value of launching future improvement initiatives.

  • How to Optimize Data in Software Development

    How to Optimize Data in Software Development

    Data can drive significant business value for every company. Globally, we are collecting more of it than ever before, and it keeps growing—from 45 zettabytes in 2019 to a projected 175 zettabytes by 2025, according to IDC. However, companies are struggling to monetize all the personal, behavioral, engagement and transactional data from customers and operations and turn it into actionable intelligence to drive their digital strategies. Leveraging insights from your data is critical for success and the development of new digital solutions to better serve customers, grow sales, improve operations and more.

    Data helps organizations develop the right software for the right customer at the right time. Data-driven organizations are 23 times more likely to acquire and retain customers than their peers, and are growing at an average of more than 30% annually, according to Forrester.

    Organizations with a deep and dedicated focus on data and analytics tend to follow three common paths to successfully drive their software development initiatives.

    Discovery Matters

    It is nearly impossible to draw value, understanding, and insights from the massive amount of data businesses collect without a proper discovery system in place. Having a comprehensive process of collecting data from various databases and silos and consolidating it into a single source that can be easily and instantly evaluated is critical for any software development project.

    During client engagements, data scientists perform deep-dive research into clients’ data pools and workflows to support client roadmaps. We recently partnered with a global e-commerce company to create a recommendation system for their customers. The discovery process allowed us to verify the feasibility of recommended products using their existing data before moving ahead with the project. Using machine learning, the team was able to clean and analyze information from multiple angles. With that knowledge and confidence, the rest of the development project proceeded quickly knowing the data was in place and the project was viable.

    Align KPIs

    Software development metrics help organizations and their partners track and measure how an application is performing and how effective the development team is in its work. IT organizations rely on a range of KPIs to fully understand software engineers’ progress, as well as gauge software quality including performance, engagement and user satisfaction. While each engagement is different and requires a different set of KPIs, ensuring that the organization and development team are on the same page is crucial.

    During the e-commerce project, success was measured through conversion and user retention rates. Other KPIs that are regularly used include developer productivity and performance metrics like cycle time and team velocity, and software developer metrics like response time, usability and UX data, among others.

    Deploy Agile

    By using agile methodologies at all stages of development, developers are able to focus on business results, not just architecture and new technologies. A similar agile approach can also be applied to data management. When it comes to data science, it’s all about extracting useful information from raw, structured and unstructured data and implementing machine learning models to analyze vast amounts of information. This process requires more creativity and is a key driver for understanding potential risks and failures over the course of any given project.

    With agile, data models and analyses are broken down into manageable bite-sized chunks that can be built and tested quickly—the same way software is developed. If the analytics do not yield the expected results, data scientists see the failures immediately and can correct their course. They could ask the team to analyze different data sets, adjust the models, tweak algorithms or make an educated decision about the project’s viability.

    A multinational credit company took this approach to the next level when it used agile to develop a new, big data hybrid cloud platform. After anonymizing, aggregating and storing information in a data lake, the team quickly realized the database it was planning to use was limited to a certain number of partitions and entities. By running agile sprints and developing and testing the minimum viable product (MVP) continuously, the team was able to seamlessly switch to a new database that had more scale and functionality.

    The value of data is only going to increase with time. Data-driven organizations embrace the use of data and analytics to gain unique business insights to support their software development projects and overall business. By focusing on the data in each and every development project, companies have a tremendous opportunity to gain a competitive advantage and grow their businesses.

  • CloudBolt Survey Shows DevOps Frustration in the Enterprise

    CloudBolt Survey Shows DevOps Frustration in the Enterprise

    A survey of more than 200 senior IT leaders at organizations with more than 1,000 employees published today finds only 4% consider their organizations to be experts when it comes to continuous integration/continuous delivery (CI/CD).

    Conducted by CloudBolt Software, a provider of a platform for many IT environments, the survey also found only 11% consider their CI/CD infrastructure to be reliable. Another 69% said their CI/CD infrastructure was somewhat reliable, while 21% said their platforms were somewhat unreliable or not reliable at all. A full 85% also report they regularly test the infrastructure they rely on to drive DevOps processes.

    Overall, the CloudBolt survey finds only 5% of respondents report their organization is deploying code multiple times a day. Most respondents (85%) said their organization only deploys code on a weekly or monthly basis.

    A full 70% said that, on average, it also takes multiple days or even weeks to set up a pipeline. Nearly two-thirds (63%) said there are too many manual processes involved when setting up pipeline infrastructure. More than half (56%) also noted they have no automated processes for post-deployment tasks.

    More than half (53%) said it is also challenging to proactively detect infrastructure issues, while 55% said it is difficult to create a consistent CI/CD pipeline environment across multiple development teams. Improving those pipeline environments requires speeding up the process of provisioning infrastructure through automated processes (70%); continuously detecting infrastructure issues to reduce testing challenges and failures (62%) and simplifying remediation of infrastructure issues proactively (56%), according to survey respondents.

    The CloudBolt Software survey specifically identifies issues organizations encounter when employing Terraform to manage infrastructure-as-code (IaC). Only 5% of respondents said they were “very satisfied” with using Terraform and another 75% reported they are somewhat satisfied. Top challenges include difficulty using it in on-premises infrastructure deployments (62%), lack of Day 2 management for deployed resources (53%) and governance (42%). Only 2% of respondents said they were employing Terraform to deploy 75% or more of their infrastructure. A full 90% noted that Terraform requires more custom-coded integrations with other critical tools and systems to work properly.

    Finally, 29% said their organization lacks specific Terraform expertise. The most sought-after Terraform capabilities are enabling non-Terraform experts to easily deploy infrastructure using Terraform; ensuring that only Terraform plans with IT “guardrails” are executed and the ability to visualize and cost-optimize infrastructure deployed by Terraform. More than a third (35%) want to simplify deployment of on-premises infrastructure when using Terraform.

    Jeff Kulkowski, CloudBolt CEO, said the survey makes it clear there simply isn’t enough expertise available to implement DevOps  best practices at scale without enabling IT administrators—that often lack programming expertise—to manage infrastructure and DevOps platforms. That requirement creates the need for an IT management platform based on a set of visual tools that enables DevOps professionals and traditional IT managers to work more collaboratively, he added. DevOps teams can manage IaC whenever they desire, but it’s also crucial to enable IT operations staff to manage IT infrastructure alongside those teams.

    Obviously, DevOps as an IT discipline still has a long way to go before it is pervasively adopted across enterprise IT organizations. The challenge now is to remove many of the low-level barriers that conspire to block IT modernization initiatives.

  • So Many Options, So Little Time

    So Many Options, So Little Time

    So many language/framework options, so little testing time, that is.

    Recently, two projects I was working on drove me to check out a breadth of languages and/or frameworks to handle cross-platform development. While Unity and Unreal have traditionally been my game platforms of choice and I typically use Xamarin for cross-platform (though native development is more common for us on both Apple and Android), I decided I needed to do the occasional sweep to find out what is new out there.

    The two projects are a game on which I’m helping a new game developer learn and a graphics-heavy application that is unsuited to web development. Both would work best in a cross-platform environment; both would benefit from a game engine’s graphics manipulations. But both could be done in any language that supports graphics, as there isn’t a heavy physics requirement to the game.

    So I went looking at what’s available now and how it fits. From React Native to Godot to Flutter and even Kivy, I kicked the tires on each, got them running, ran and traced samples, wrote some code of my own, even talked with friends that were using them to get their opinion.

    Alternate frameworks languages

    The one thing that came through loud and clear? There is no way that security and testing are keeping up with this language/framework proliferation. Kivy has a language (more of a specification than an actual language, but still—source code) that can run on top of Python (it’s not required to use the framework, but it’s kind of assumed you will if the app gets beyond a certain size); Godot has a language that is integral to the framework and that you use pretty much everywhere. React has a ton of add-on code and both Flutter and Dart are new languages.

    I like stuff from each of them. I don’t like that we’re going to use them in ever-increasing amounts and we’re going to expect testing to stay on top of those implementations. Each of these platforms/languages/environments has a large following that is doing some of the testing, but since they’re all relatively new, “How much testing?” is a valid question. And tests have to be done in those languages, implying that those writing need to know them and so do testing teams.

    Where did we end up? We haven’t made a final decision yet, but taking input from the person I’m mentoring, coworkers and those who are using each of the above, we’ll probably end up using Godot. For 2D apps, it is the bomb. Our team is small, so I can offer my knowledge of Godot to shore up testing and everyone agrees that its Python-like language is acceptable, while the graphic design element is nicer than the competition. Both applications will benefit from Godot’s graphics processing capabilities and the game will benefit from game elements built into the Godot Engine.

    Here is an interesting side note: All of the products we looked at are open source. Both of the default products that we (and just about everyone else) would choose for the game are commercial. That’s interesting, but we don’t need the weight of those commercial solutions for a simple 2D interactive game/app.

    For you all, the questions might be harder to answer. With larger projects come larger requirements for source size, test, you name it. Choose carefully, with an eye on the future. Personally, I think React Native, Godot and Flutter would all be good choices for cross-platform applications, with Godot the best solution for cross-platform 2D games. And all three are likely to be around with a good size following for a long time, making them safer choices. That is less clear with Kivy, though the popularity of Python appears to be driving growth in Kivy use.

    Whatever you choose, plan for all phases of development, not just writing code. That’s true of everything, but new cross-platform tools are complex beasts, so it is worth reminding you all.

    And hey, if you have a cool app you developed with one of these tools, point me to it. Always good to see what you all are doing, and nice to see the breadth of solutions these languages can create.