devopsy.kinsta.cloud

Category: DevOps Practice

  • Q&A: Database Security in Financial Services with Yaniv Yehuda

    Q&A: Database Security in Financial Services with Yaniv Yehuda

    As financial institutions are adopting DevOps best practices to speed up application delivery the database remains the stumbling block causing delays, introducing risks, slowing down release cycles and wasting valuable internal resources. At the same time, compliance and audits are managed manually, complicating the process even further.

    It is time to bridge the gap between the database and the CI/CD process in order to achieve faster release cycles and simplify compliance, security and auditing.

    Yaniv Yehuda, co-founder and CTO at DBmaestro, answered a few questions regarding security challenges in the financial industry and DevOps.

    Yaniv is one of the distinguished speakers presenting at this year’s FinConDX Virtual Summit. During his session “Database Devops For Financial Industry: Security And Audit,” Yaniv will take a deep dive into DevOps best practices and the issues related to the database.

    What are some of the main challenges financial industries are facing when it comes to security and the database?

    In the modern world you need to move faster, you need to automate, but at the same time you need to bring security to the front. You can’t deal with security and compliance after the fact, it is just going to slow everything down. As everyone is trying to accelerate, compliance and security must be baked into the process. What the financial industry is looking for is a way to make sure security and compliance are dealt with automatically, while at the same time aiming for the productivity and process improvements when dealing with data. What financial institutions are struggling with is how to set up processes and procedures in their day to day while achieving compliance and security as a byproduct, instead of generating additional workloads.

    How can DevOps help? How does financial services companies benefit from DevOps?

    DevOps is a way to build the process and the communication between the stakeholder and all the technologies in order to automate the processes. Once you automate the process you can plug in your documentation, your policies and your pre-checks to prevent things from breaking. It’s not a person who does this – it is a process.

    What can we expect from your session?

    Understanding DevOps best practices, how they apply to the database and what DBmaestro offers out of the box to achieve end-to-end automation that includes the database.

    To learn more about database security and DevOps in the financial services industry, attend Yaniv’s session at FinConDX on Dec. 3. To check out the full agenda or to register for free, visit the FinConDX website.

  • Antipatterns That Hurt DevOps Implementations

    Antipatterns That Hurt DevOps Implementations

    Avoiding these DevOps antipatterns can help organizations reap the benefits of implementations faster

    It’s a given that all organizations at some point will suffer from poor development practices when trying to find the perfect balance between software development and IT operations. The good news, however, is that implementing DevOps can help you improve these processes and get them synchronized and working together in harmony.

    However, understanding and mapping out the core issues and the common DevOps “antipatterns” that are holding you back from successful DevOps implementation is essential. This article will focus on the most common DevOps antipatterns that organizations tend to fall into and how to avoid or overcome them.

    What Are DevOps Antipatterns?

    At first glance, DevOps may sound like a pretty straightforward concept. At its core, DevOps is about bringing together separate but functional areas and bridges the gap between software development (Dev) and IT operations (Ops) with the ultimate goal of delivering software faster and with better stability.

    That said, DevOps antipatterns can best be defined as ideas that are counterproductive to the DevOps culture. They include:

    • Blame culture: In this kind of culture, blaming and punishing people is the norm when mistakes are done. It can happen at an individual or organizational level.
    • Root cause analysis (RCA): RCA is a process that involves identifying the “root” cause of a problem or an event, as well as the appropriate actions to take to avoid a recurrence. It’s an iterative and continuous process that must be performed until data is exhausted or all organizational factors have been identified.
    • Human error: Human error is often cited as a root cause in RCA. This is also common in “blame culture” and someone has to be held accountable for mishaps and mistakes.
    • Silos: An organizational or departmental silo defines the mentality of teams that don’t share their knowledge with other teams within the same company.

    4 DevOps Antipatterns and How to Break Them

    Wrong Perception of Your DevOps Team

    The perception you have of your DevOps team matters. It’s essential to consider the team as part of the entire process. Neglecting to provide them with structure, resources, and a positive working environment can increase your chances of DevOps failing. Furthermore, making DevOps teams more productive requires proper implementation of processes and tools.

    How to Break It

    A successful DevOps implementation requires more than just bringing people together. Structuring the team correctly and adding processes is vital if they are to succeed. Providing the necessary resources, technology and information security they need will get the best results.

    Practices such as continuous integration (CI) and continuous delivery (CD) are both concerted processes that can be used to bring development teams together in alignment so software products are delivered quickly, operations are more simple to manage and no downtime is present during deployments that are detectable to the end user.

    Poor Choice of Technology

    Technology plays a significant role in IT; however, the rapid development of new technologies has made it challenging to separate essential and overhyped tech trends. For instance, DevOps itself is a powerful digital transformation tool as it plays a critical role in new software development and problem-solving. Nevertheless, many of the latest IT trends should not be worth your attention unless you and your team are willing to implement it correctly.

    How to Break It

    This antipattern can be broken by top-level and project managers who make the decisions regarding technology choices. Asking questions such as, “What does the organization need?” or, “How well will this technology work for us?” will help when choosing what tech is right for your organization. With thousands of tools and technologies available, you need such a mindset to make the right decisions.

    Unclear Responsibilities in Teams

    You may have come across this problem in many other non-IT projects. However, in both IT and DevOps teams, the consequences can be severe if solutions aren’t implemented quickly. If team members aren’t clear on their roles and responsibilities, this can lead to poor decision-making and unnecessary interference in other people’s positions.

    How to Break It

    First of all, you need to clearly define roles and responsibilities for everyone in the team. A clear definition of roles boosts productivity. For example, having a data scientist in your DevOps team can play a huge role in business success due to their very specific data analytics skills, techniques and tools. In fact, 89% of business managers agree that hiring a data professional will help to improve success at their respective organizations. There just needs to be active and consistent communication between team members for everyone to conduct their own roles and responsibilities efficiently.

    Every team member should follow the strategies and plans in place so timelines can be monitored and effective top management decisions can be made. When implementing complex DevOps architectures that require teams and software to work together, poor decision-making will be an issue if roles are left unassigned.

    Wrong or No Code Deployment

    To put it bluntly, there’s no DevOps without code deployment. With this being the ultimate result, DevOps makes no sense if it fails. This is quite a challenge for several organizations, as having Dev and Ops teams working in harmony can seem impossible.

    How to Break It

    As mentioned above, code deployment is often a challenging task for DevOps teams. However, organizations can break the resultant antipattern by integrating the units correctly to enable a collaborative environment, a great strategy for successful DevOps implementation.

    Such an environment will also make code deployment easier by reducing the need for a series of meetings and reviews, which ultimately delay teams from getting code to deployment.

    NoSQL’s Key Role in Database DevOps

    DevOps processes need to work in a synchronized manner to be successful. Issues with the database can cause it to slow down and affect the delivery of database changes. However, NoSQL’s unique approach is designed to simplify the process of DevOps implementation in the following ways:

    Faster scalability

    Many small companies fail to maintain peak performance with greater workloads. A more significant workload calls for better tools that can help maintain peak performance, subsequently making it easier for organizations to scale out.

    More efficient technical support

    Having technical support that doesn’t hold you back is key to DevOps success. The right NoSQL solution enables you to solve problems before they become full-fledged, expensive and time-consuming.

    Faster adaptability

    The changes that new codes bring for databases can be challenging. NoSQL makes it easier to deliver solutions to client requests, as it organizes them using dynamic indexing. Therefore, it verifies all queries before answering them with the appropriate index.

    Platform testing

    Testing is one of the critical steps you need to go through before actual implementation. Databases should be flexible enough to allow you to test queries without interfering with the real data. Testing helps to determine how new changes will impact the development process.

    Conclusion

    As you can see, there are many ways to implement DevOps systems in your business to deliver products faster, offer more technical support and automate essential tasks.

    All of the requirements we have covered are necessary if a business is to scale and grow. Businesses need to ensure the correct strategies are in place, roles in the company have been clearly defined and common antipatterns have been identified and eliminated.

  • InfluxDB to Revamp Storage Engine for Time-Series Database

    InfluxDB to Revamp Storage Engine for Time-Series Database

    InfluxData has launched an open source InfluxDB IOx project that it said will become the foundation for the next generation of the company’s time-series databases.

    At the same time, the company has made InfluxDB Open Source 2.0 generally available. This version of the company’s database comes in the form of a single binary that makes it easier to secure.

    InfluxDB Open Source 2.0 also provides access to Data Explorer visualization tools as well as a lighter-weight Flux query language purpose-built for time series data.

    InfluxData CTO Paul Dix said while improvements are still being made to the existing database offering, the time has come to start working toward addressing a number of issues via a new project anchored around a storage engine that will make it possible to query petabytes of data across thousands of servers. The InfluxDB IOx project will address that issue by eliminating current restrictions on cardinality, data size and cluster size that are inherent in the current platform, he said.

    The InfluxDB IOx project is written in the Rust programming language and incorporates Apache Arrow, an open source in-memory analytics engine that incorporates a columnar data store that is being advanced under the auspices of the Apache Software Foundation (ASF). That approach will also make it possible for InfluxDB to natively support SQL along with InfluxQL and Flux query languages.

    The current plan is to make InfluxDB IOx an optional storage backend in a future point release. By the latter half of next year, InfluxData plans that InfluxDB Enterprise customers will have a commercially supported version of InfluxDB IOx and InfluxDB Enterprise. InfluxDB IOx will have its own builds and can be run separately. In effect, Dix noted, InfluxDB IOx will make it easier to separate compute and storage requirements.

    Dix said that while InfluxDB has served as the foundation that enables a wide range of monitoring tools to collect data in real-time, current cardinality limits that impact the number of tables that can be joined requires a new approach to add support for distributed tracing capabilities required to observe microservices-based applications. The IOx version will also employ object storage platforms to reduce the total cost of ownership while making it possible to address the requirements of distributed computing environments that now include a wide range of edge computing platforms, he noted.

    The decision to embrace Apache Arrow also opens the platform to end users who prefer to work with SQL and other scripting languages, he added.

    It may be a while before observability tools that leverage InfluxDB IOx find their way into DevOps tools. However, it’s clear that time-series databases will evolve to meet the requirements of cloud-native applications that will be based on large numbers of microservices spanning distributed computing environments. The only issue is to what degree the current rate of deployment of these applications might outpace the current ability of InfluxDB to keep pace.

  • Design Systems: Development Choices for Meeting User Expectations

    Design Systems: Development Choices for Meeting User Expectations

    Modern software development trends involve design systems and modernization, and if we drill down further, the trend within the trend is about creating better user experiences. This means improving usability, emerging technology and performance enhancements. This is easier said than done. What does it mean for developers?

    Employing Design Systems

    What a designer sketches, a developer tries to replicate, often with varying degrees of success. When we introduce multiple screen sizes and form factors, the experience can vary greatly if the collaboration between design and development is not in sync. To combat this, many modern software development practices are beginning to employ the use of design systems to provide an outstanding experience to the end user and streamline processes internally.

    Typically, a design system consists of UI components and visual style guidelines, both in the form of reusable code and reusable design artifacts. The system could include UI patterns, foundational elements and design principles or guidelines. Many go as far as defining what tools should be used for optimal UI and UX. For example, one of the most widely known design systems—and most used, due to its open source nature—is Google’s Material Design.

    With a design system in place, developers and designers are singing from the same sheet of music. This eliminates many delays and inconsistencies and provides prescriptive guidance for both the visual and functional experience of the app. It allows developers to deliver the beautifully styled, feature-rich responsive experiences the end user has come to expect.

    Choices for Modernization

    For .NET developers, this could mean exploring Blazor, Microsoft’s answer to JavaScript, which allows developers to build interactive web UIs using C#. It could mean diving into.NET 5, which is the next step in building a unified platform, allowing developers to use it to target Windows, Linux, macOS, iOS, Android, watchOS, WebAssembly and more. Or, it could mean getting a jump on .NET MAUI, the welcome evolution of Xamarin.Forms, combining the best of Mono and .NET Core and allowing developers to build truly native desktop and mobile apps for any operating system from a single codebase and project system from their preferred IDE.

    For the JavaScript developer, modernization could mean looking toward Vue 3, which is a faster, smaller and more maintainable framework. Or, it could mean exploring Angular 10 and its warnings for CommonJS imports. Or maybe it’s truly understanding and appreciating the downplayed release of React 17, which now allows multiple versions of React on one page, making it easier to use React with other frameworks. Modernization is all about staying relevant, learning and leveraging the latest the industry can offer.

    Popular Frameworks

    For front-end development, React, Angular and Blazor are some of the most popular frameworks at the moment. Blazor is so new enough that you aren’t going to find anyone with years of experience, but it is a framework that is incredibly adaptable and easy to learn if you are C# developer.

    React and Angular are more mature and with their latest releases provide even more compelling reasons to build your web apps with them. The soft skills—the ability to communicate and collaborate—are some of the most in-demand skills today.

    Regardless of your front-end preferences, the proliferation of frameworks and new versions means that developers have more choices than ever for creating rich, interactive, modern experiences for multiple platforms and devices.

  • DevSecOps Implementation: Dynamic Scans

    DevSecOps Implementation: Dynamic Scans

    This is the third installment in this series on DevSecOps. Read the first installment, on static analysis, here and the second installment, on source composition analysis, here.

    One weakness of static analysis is its failure to account for environment and use. Running static analysis on a code base as the only check before production deployment is akin to looking at the engine of a VW Golf before assembly and pronouncing the vehicle fit to use. The way a vehicle runs after assembly, and the way it is—or, more precisely for dynamic scans, could be—used, matter to any product. That is where dynamic scans come in.

    Exercising the product as it runs is a big deal, and that is what dynamic scanning attempts to do. Automating the testing of applications by exercising inputs and watching the results, dynamic scans can detect a variety of issues that static analysis simply cannot. These tools are the source of a lot of the noise in DevSecOps because they’re testing a variety of scenarios with each run, and things that a dynamic scan sees as vulnerabilities might very well be known to not be a risk at all by the dev team. As with static analysis, this has been improving over time, and they’re well worth a look at this point.

    The best use for these tools is as gateways—before going to test, in test and before going to production, set a policy for promotion requirements and then run dynamic scans to determine if the application meets that policy, needs remediation or needs approval to go live with possible issues known.

    With all of that said, here is a starter list of things to look for when considering using dynamic scans:

    • Production Safe: These tools run tests that try a documented vulnerability to see how the application responds. The thing is, if you’re not very careful, this can cause serious problems in production. Since long-term the trend is to eliminate a pure test environment for all but the most stability-conscious organizations, knowing that the product you choose can run in prod without causing outages is an important step.
    • Vulnerabilities Tested: Most vendors offer a list of what they look for. Compare these lists to your organization’s needs and other vendor’s lists. Since choosing a product is unique to each org, I don’t have much in the way of buying advice, but here I will say if the product does not support at least the OWASP top 10, look elsewhere, simply because OWASP is both a recipe for you to prevent attacks and a handy list of easy vulnerabilities for attackers to look for. All of the vendors we looked at for our ASG work do support OWASP, so this might be non-advice.
    • Integration: Dynamic scans need to be run after the product is fully assembled. This is not to say complete, but the various pieces need to be together—the scans generally look at an application, not pieces thereof. This means they can be spawned at any point after build—or, more specifically, after post-build deployment to dev/test/prod. Make certain there is a place in the toolchain to call the scanner from and proper gateways to suit your organization’s needs. Part of that evaluation is how the application is kicked off: Can your build tool call it directly? Is it deeply integrated into your chosen environment, or will you have to do work?
    • Reporting and Gating: What level of reporting is available: Can dynamic reports and static reports be integrated to show that a potential problem reported by static scan is found to actually be a problem by the dynamic scan? Can an overall status report be found (à la C-level dashboards) to show the security posture of a given application? Can alerts be configured for apps that fail certain tests (so you can make informed decisions before going to production, for example)?
    • RBAC: This is applicable to all scanning, but I’ve chosen to include it here because for dynamic (and interactive), it’s a bit more important. Can you limit what an individual can see/do? In the end, dynamic scans are reporting on actual vulnerabilities in your codebase—only those who need to know to fix the issue should be able to see it; others should just see pass/fail overall for the app, not a list of known vulnerabilities in the app. The ability to run these scans should likewise be limited. Who/when is a corporate decision that is as much policy and process as it is the chosen tool, but if the scanner doesn’t support that policy, enforcement becomes more difficult.

    When you get to interactive scanning, it is time to consider how security issues traditionally have been handled versus how they should be moving forward. Most organizations hold security issues separate from other application development issues. It is time for that to end—and indeed, some organizations ended this process long ago, but too many are hanging on to the past. Security vulnerabilities that can result in a risk to the organization need to be run through the system as high-priority bugs. Because they are. We used to think that something making the UI look bad for 5% of customers was more important than a security issue impacting no one. A truly global internet and the rise of automation even among attackers make this a silly stance to take. No one is impacted by security issues at this second, but sometime in the next minute, everyone might be impacted. It’s like saying you don’t need scalability for Black Friday because the servers are handling traffic well right now (written in early November). The issues that dynamic scanners alert on are generally real (removal of false positives has come a long way in the dynamic space) and generally dangerous. If your scan tool can find it, assume attackers’ scan tools can, too.

    As with the other installments, I’ll thank you all for keeping the lights on in a really topsy-turvy IT environment—you are doing us all a service—and mention that all of your hard work is at risk if security at your organization isn’t taken seriously. We quickly identified the threat from the sudden opening of millions of RDP ports on the public internet, and most of you took steps to secure your own. Keep it going. Make certain the apps you are developing and deploying are as safe as you can make them. And keep kicking rear—you are the rock that’s keeping the suddenly dispersed company clipping along.

  • What Children Can Teach Us About Coding

    What Children Can Teach Us About Coding

    Redgate’s Code Club for kids ended up teaching their parents to love coding again

    Like lots of people in IT and other office-based jobs, I’ve been working from home since the pandemic started to hit in March 2020. In some ways it’s been hard because I’m the technical lead on a software engineering team, and continuing to work in an agile, collaborative way can be difficult via Zoom. In other ways, it’s been interesting because we’ve demonstrated it can work, often remarkably well.

    But I’m not just a technical lead, I’m also a father. And for my 8-year-old son, Dougie, things have been tougher. Here in the UK, schools were forced to close for a long time and one of the things he really missed was the lunchtime Code Club classes he’d been taking. Run by teachers, parents and volunteers and supported by Code Club UK, part of the Raspberry Pi Foundation, Code Club is a great initiative to help children to code with a range of really well-planned and thought-out projects.

    So naturally, being a software engineer myself, I thought, How hard would it be to organize an online code club for the children of people at Redgate Software? I mentioned it to a few people at work and when someone suggested maybe we should see if parents beyond Redgate would be interested, I thought, No problem at all. Redgate has quite a few followers on Twitter, so we put out a tweet.

    What could possibly go wrong?

    Panic Stations

    What started as an idea to help maybe 10 or so children of the people at Redgate went crazy. By the next morning we had 1,000 sign-ups from kids from all over the world. Not just the U.S. and Europe, but India, Nigeria and Sri Lanka as well. And the sign-ups kept coming in by the hundreds.

    Fortunately, the Raspberry Pi Foundation had already done a lot of the hard work for us by creating a series of Code Club class projects for beginners, intermediate and more advanced coders. They were delighted at our suggestion of using them in our online classes, and recording them so that they could be watched later as well.

    Redgate also actively encourages staff to give back to the community and when I gave a shout-out for volunteers to help run the classes, there was a huge response with 50 people raising their hands. Within a couple of weeks, we had the sessions planned, with the beginner and intermediate ones using Scratch and a more advanced session learning Python.

    There was quite a lot of safeguarding we had to think about as well around talking to other people’s children over Zoom, and we had conversations with the director of Code Club UK and her team to get their input. We wanted to make sure people weren’t sharing their videos and sound and that kind of stuff, but we managed to work our way through it and ensure we were protecting children while sharing some really cool stuff with them.

    All Systems Go

    On Friday, April 3, Redgate’s Kids Coding Club opened its doors online to hundreds of children aged between 6 and 14, as well as their parents. I wanted the kids themselves to run the sessions for other kids, so they were the presenters and their parents were co-presenters. In the sessions hosted by my son Dougie, he would talk about what we were going to do in the session and explain what things like sprites are. We then worked through the module as a group and if anyone got stuck, they could raise their hands virtually using the Zoom tool and a volunteer would step in to help.

    We ended up running six coding club sessions every Friday for 17 weeks while schools were closed. During that time, we taught nearly 800 children from 20 countries how to code, delivered 1,572 hours of learning, and recordings of the classes on YouTube have racked up thousands more views.

    What We Learned

    What I love about the coding club is that while we were delivering learning, we were learning, too. There were three big lessons for me.

    Children love to code

    A lot of people think children just want to sit down and play Minecraft or Fortnite, but if you give them the right tools, they absolutely love coding. Scratch, for example, is a block-based visual programming language that’s really easy for children to pick up and start coding with, particularly when you give them fun stuff to do.

    Sixty percent of the children we taught were between the ages of 6 and 8 and they loved using Scratch on projects such as Rock Band, Cats, Clone Wars and Boat Race. They coded their own musical instruments, stopped cats falling into holes, saved the Earth from space monsters and navigated a boat to an island. What’s not to love?

    Parents love coding, too

    Around 40% of the kids who signed up had parents who had never done any programming either, so they were both starting together in front of the computer, which was great. Typical feedback from the parents was that they were surprised that beginning to code isn’t hard, it’s really fun. A lot of them sat with their children for an hour after the lesson and continued to just play about with things and change stuff.

    Children can help us be better at coding

    There’s this preconception with coding that you have to get it exactly right for it to work, but what’s great about Scratch and other tools is how much freedom the kids had to change the project they were working on. They’d rather use a parrot than a polar bear. They didn’t want it to jump 10 pixels high, but a hundred pixels high. What happened if they asked it to jump minus 10 pixels? They could just poke the thing and play with it, and it was fun to see the little light bulbs going on in the face of the kids—and their parents.

    That can teach us about coding, too, because we often go down tramlines and do things the same way, every time, even when we say we’re taking a test-driven development approach to coding. Teaching children how to code was a nice reminder that we should be thinking more like them, poking and pushing things to the limit and doing the unexpected. We’ll end up with better code, and we’ll have more fun, too.

  • FinConDX: The State of DevOps in FinTech

    FinConDX: The State of DevOps in FinTech

    The FinTech space has grown dramatically this year as more consumers shift their financial transactions online due to the COVID-19 pandemic.

    Financial services companies are learning how to ship better code, faster by adopting DevOps methodologies. DevOps helps companies develop fast, fail fast and learn fast so they can fulfill customer expectations and deliver quality features faster to market than their competitors. However, the shift to DevOps is not that easy.

    The digital transformation journey for banking and financial services organizations can be quite challenging. That is why, on Dec. 3, the top financial services and tech leaders in the industry will come together at the FinConDX Virtual Summit, a free virtual conference focused on the DevOps, DevSecOps and digital transformation practices and tools you need to thrive in the FinTech industry today.

    FinConDX features a stellar lineup of industry experts, including:

    • Yaniv Yehuda, co-founder and CTO at DBmaestro
    • Viktor Farcic, principal DevOps architect at Codefresh
    • Sladjana Jovanovic, vice president of payments technology, TD Bank Group
    • Shannon Lietz, director of DevSecOps at Intuit
    • Sanjeev Sharma, senior vice president, head of automation and platform engineering at Truist
    • Rosalind Radcliffe, distinguished engineer at IBM
    • Don Duet, CEO and co-founder at Concourse Labs
    • Ravi Lachhman, chief evangelist at Harness
    • Matt Wilson, senior product manager at GitLab
    • Mitchell Ashley, CEO and founder at Accelerated Strategies Group
    • Keith Murphy, senior solutions architect at OutSystems
    • Jonathan Smart, partner, business agility lead at Deloitte
    • John Rzeszotarski, senior vice president of technology infrastructure at PNC
    • Dan Garfield, chief technology evangelist at Codefresh
    • Gary Gruver, CEO at Gruver Consulting and analyst at Accelerated Strategies Group

    The virtual environment features auditoriums with different tracks where attendees can watch the sessions by industry leaders and an exhibit hall lined with booths showcasing the latest solutions from leading DevOps companies. In the exhibit hall, attendees can download resources and interact with partner sponsors including Codefresh, GitLab, WhiteSource, DBmaestro, Harness, Traceable, OutSystems and more.

    Breakout sessions and panel discussions will fall under following topics:

    • Leading digital transformation
    • Emerging technology
    • Security and audit
    • Modernization tools

    Attendees can participate in a scavenger hunt throughout the virtual environment for a chance to win amazing prizes. Participants earn points by visiting booths, watching sessions, downloading resources, networking with speakers in the chat and finding hidden objects in the environment.

    Registered attendees will also receive a link to access the virtual environment on-demand for a limited time once the live event is over.

    To see the full agenda and to register, please visit the FinConDX website.

  • 7 Important Truths About Chaos Engineering

    7 Important Truths About Chaos Engineering

    As a relatively new practice, chaos engineering has plenty of myths surrounding it, from randomly shutting down production systems to requiring huge investments of time and money. There’s a lot of confusion over the purpose, the value and the practice of chaos engineering. This presents a problem for DevOps teams, especially since more than half of the teams surveyed in the 2019 Gartner DevOps Survey listed improving system reliability and release quality as one of their top five DevOps objectives.

    In this article, we’ll clear up some of this confusion by presenting seven important truths to help you make an informed decision about chaos engineering and how it can help your engineering organization.

    Truth 1: Chaos Engineering is not Chaotic

    What comes to mind when you think of chaos engineering as a practice? If the answer is causing random production outages, you’re not alone. Tools such as Chaos Monkey popularized this idea, but for most teams practicing chaos engineering, it is a well-planned, controlled process that aims to mitigate chaos rather than cause it.

    The goal of chaos engineering isn’t to add chaos, but to mitigate chaos.

    It’s true that it involves creating potentially harmful conditions on otherwise healthy systems. For example, we might test our application’s ability to handle load by increasing CPU usage on our servers. With enterprise solutions we have full control over which systems are affected by this test (known as the blast radius), how much CPU we consume (known as the magnitude) and how long it runs for. We can also immediately stop the test and roll back its impact in case of unexpected consequences.

    We also know exactly what conditions we’re introducing into our systems and can revert the changes at any time. Yes, we’re still causing harm, but we’re doing so in a way that helps us learn about our systems and reduces the risk of failures, both real-world and induced.

     Truth 2: Developers Care About Reliability

    Developers aren’t only interested in building new features. While development teams often prioritize feature development, this is likely due to business-driven initiatives resulting in rapid release schedules. Developers—especially those who have responded to incidents or worked through bug reports—understand the value of resilient software. They just don’t have time to build it.

    The problem is that this creates a reliability gap in our applications. Rapidly building features without adequately testing their resilience creates failure modes, leading to problems that developers need to go back and fix at the expense of new projects. The longer we go before finding these failure modes, the more likely we are to experience unexpected behaviors and outages in our applications, and the more expensive a solution will be.

    Increasing development velocity without testing for reliability increases the likelihood of failures, causing our expected availability to drop.

    To prevent this gap from forming, we need to prioritize and provide the appropriate tooling so developers can test their code and improve the resilience of our applications. This leads to our next truth.

    Truth 3: You Have Enough Time for Chaos Engineering

    Chaos engineering saves you the time you would spend responding to, troubleshooting and fixing production incidents. As you build resilience through chaos engineering, you’ll reduce the engineering time, effort and costs spent on incidents and can focus more on your core competencies.

    That said, adopting chaos engineering does require some time investment. Engineers will need to learn the principles and practices, adopt new tools, run experiments and implement fixes. However, this is relatively minor compared to the cost of production outages. Without chaos engineering, your engineers will need to pause their regular work and rush to restore service. This is time they could be spending on feature development and other value-creating tasks.

    A few minutes of proactive testing can uncover defects that would’ve taken hours to fix in production.

    Truth 4: Your Systems Support Chaos Engineering

    You don’t need to use specific hardware, software, or cloud platforms to benefit from chaos engineering. While some tools have specific requirements, most only require access to the host operating system. Some can even target specific resources, such as containers.

    Truth 5: You Don’t Need Comprehensive Observability (But It Helps)

    You don’t need a comprehensive monitoring or observability practice in place to benefit from chaos engineering. Basic metrics provided for free by your cloud provider can still provide important insights into how your systems are responding to experiments, and some chaos engineering tools provide basic monitoring for you.

    That said, chaos engineering provides significantly more value when used with monitoring and observability. We want to know how our systems are responding to our experiments, and the way we do that is by collecting metrics and traces. Even if you’re currently implementing a monitoring solution, chaos engineering can help streamline this process by helping you determine which metrics to track, fine-tune your alerts and avoid alarm fatigue.

    Truth 6: Chaos Engineering Pays for Itself

    Like any new tool or practice, chaos engineering requires an upfront investment of time and money. Solutions need to be evaluated and purchased, and engineers need time to warm up to the practice. However, the value it provides far exceeds these costs. Using chaos engineering can help you:

    • Avoid losses in revenue and productivity due to incidents and engineering time spent resolving incidents.
    • Improve your ability to generate sales by helping you build reliable systems and prepare for periods of increased demand.
    • Save costs by helping you optimize resource allocation and test for non-essential infrastructure.

    Truth 7: You Can Get Started Today

    Getting started with chaos engineering can be as easy as deploying an agent to your systems and choosing a resource to target for experimentation. You don’t need to have a stable, mature SRE practice or enterprise-grade infrastructure. In fact, you can use chaos engineering to help establish your SRE practice or accelerate migration to a new platform.

    The most important truth is that no system is perfect, and there are failure modes hidden all throughout our applications and infrastructure. Chaos engineering lets us uncover these failure modes and transform reliability from myth to reality.

  • 3 Steps to Turn a Data Deluge Into Actionable Intelligence

    3 Steps to Turn a Data Deluge Into Actionable Intelligence

    Digital transformation is creating even more data for organizations to manage effectively. Here’s how to find the intelligence needle in the data haystack

    Data and analytics transform how companies across all industries identify and act on opportunities to gain a competitive advantage. But many organizations are discovering that ever-growing volumes of the former make the latter increasingly difficult. They struggle to turn raw data into a strategic asset that informs decision-making and accelerates growth. Overcoming that struggle requires conceiving and implementing an effective data transformation initiative. Fortunately, there are three steps to implementing sustainable initiatives that any company can follow, no matter its maturity or size.

    But before taking the first step, it is vital to secure the commitment at the board and management levels that the initiatives will be a companywide priority. That is critical to ensuring the allocation of adequate resources and budget dollars at the outset. Failure to secure senior leadership’s enthusiastic support will impede progress if not completely derail the entire project.

    You might not think that would be a difficult sell. But research firm KPMG found that the opposite is true when it examined the fundamental role data analytics will play in the financial services sector over the next 10 years:

    “(Many CEOs are) responding incrementally and, at best, tactically with better front-end digital services for customers or process improvements in the back-office. But they’re not addressing the more fundamental issue – that their very business model will need to change. And fast.”

    If you encounter resistance, here’s the crux of your argument: Your company is a data company. It doesn’t matter if you sell software, cars, clothing, food or any other product or service; the ever-growing volumes of data you’re collecting inform all business decisions. Or at least, they should.

    Curt Hopkins, editor in chief for Hewlett Packard Labs, summed up this point perfectly: “Data has become the new gold that backs the value of companies, though it is a gold that companies must locate, mine and refine before spending. Business has shifted from making things to knowing things, so realizing usable data is the order of the day if a company wants to remain profitable.”

    Step 1: What Data Do You Have?

    Once you have secured the full support of management, you’re ready to take the first step to launch a data transformation initiative: gain an understanding of precisely what you have and where it “lives.”

    Take the time to conduct a thorough exploration of all your raw data sources and where those volumes are stored—likely some combination of individual users’ laptops and other devices, on-premises systems and in the cloud (public, private or hybrid).

    Next, determine who has access to your data and how. If you decide to work with an analytics services provider, do you have the necessary extraction processes in place to deliver raw data to your partner? If not, you will need to build that pipeline or move the data into a cloud-based analytics database.

    Step 2: What Do You Want to Accomplish?

    Once you know what you have and where it is across your organization, the next step is to formulate your initiatives based on your objectives. Classifying all datasets under two broad categories—internal use or external use (i.e. commercialization)—will help narrow down the relevant data for a specific campaign.

    Embed responsibilities for secure data handling and usage across all parts of the organization—don’t put them only on the technology teams’ collective shoulders. Assume you will need to organize, transform and enrich data to realize its full value. Remove any barriers to sharing data under NDA and considering a nuanced build-versus-buy approach. Finally, ensure the initiatives map to management by objectives (MBO) to maintain a focus on using data to help the company achieve its business goals. At this point, you’ll need to answer the build-or-buy question to determine if you have these transformation capabilities internally or need to find them externally.

    Step 3: How Will You Track Progress and Evaluate Outcomes?

    After executing the data initiative, you should regularly measure its outcomes. Hold your teams accountable by setting time-based goals and tracking progress versus plans. Anticipate the need to iterate because harnessing data is a fluid process. Create and maintain a dictionary to keep employees and teams aligned as they progress on their tasks and projects.

    One additional and no less critical consideration is to ensure you strike a balance between gaining the ability to use data assets with protecting them from security threats, both cyber and physical. According to  “The Cost of Insider Threats Global Report 2020” by IBM and the Ponemon Institute, the average cost to recover from a breach is between $3 million and $4 million, and that does not account for the loss of customer trust and brand loyalty. Protecting customer and employee personally identifiable information (PII) and all other sensitive data types and complying with all data rights agreements is paramount.

    Fifteen years ago, British mathematician Clive Humby famously proclaimed, “data is the new oil,” and the analogy holds up today. Like the raw crude that comes out of the ground, raw data only represents potential value. Just as oil has to be refined into fuel to become useful, data must be analyzed and refined to become functional.

    By understanding your supply of raw data, formulating your initiatives and executing and measuring those initiatives, you will be better able to identify opportunities and make more informed decisions that give your organization a competitive advantage.

  • Low Code: A Modern Approach to Software Development

    Low Code: A Modern Approach to Software Development

    Low-code development is not a new concept, but since the onset of the COVID-19 pandemic earlier this year, it has become increasingly popular. Low code helps businesses adapt to ever-changing environments quickly by allowing them to build, customize and deploy apps with little to no coding. It enables organizations to accelerate their innovation, automation and transformation efforts.

    “With low-code applications, business analysts, project managers, operations teams and other business users are empowered to implement solutions for themselves,” said Prodly CEO Max Rudman. “Consequently, technical developers have more time to focus on projects that require specialized skills. Because developers are now able to focus on tasks that really do require programming expertise, they become more productive, too.”

    On Nov. 19, the low-code community will come together for the first Low Code Today Virtual Summit to take a look at low code/no code in today’s new normal and highlight the impact of the rapid acceleration of low code use in the wake of COVID-19.

    Here’s a sneak peek of some of the discussions that will take place during the event by some of the distinguished speakers presenting at Low Code Today.

    Accelerate Digital Transformation With Low Code

    In this interview, ServiceNow CIO Chris Bedi shares the main benefits of low code and talks about the ServiceNow Citizen Developer Program. During his keynote presentation, Chris will share a CIO perspective on rolling out low-code applications at scale, best practices from rolling out ServiceNow Citizen Developer Program and suggested guardrails to ensure a successful and secure implementation.

    Empowering Citizen Developers With Low Code

    During this interview, Kianda Chief Operating Officer Derya Sousa explains how low code accelerates digital transformation and empowers citizen developers. Derya will be presenting at Low Code Today Virtual Summit along with Kianda CEO Osvaldo Sousa. During their session, Derya and Osvaldo will discuss how organizations use low-code/no-code platforms to build fully responsive, easy-to-deploy enterprise applications – all without the need for any coding.

    How Low Code Is Transforming Software Development

    Rackspace CTO Jeff DeVerter joined us to explain how low code is transforming software development and how Robotic Process Automation (RPA) complements low-code application platforms. During his keynote presentation, he will explain the newfound prominence of low code and outline how enterprises can best utilize the technology to accelerate the delivery of software and applications.

    Low Code Today features a full day of keynotes, panel discussions, Q&As, workshops and more from low-code pundits and pioneers. Attendees will get an intimate look at how organizations are embracing low-code technologies to advance their business while improving user experience.

    Some other speakers presenting at the virtual event include:

    • Matt Calkins, CEO at Appian (keynote speaker)
    • Jennifer Sable Lopez, Sr. Director, Community and Advocacy at OutSystems; Malcolm Ross, Deputy CTO at Appian; Marcus Torres, VP Product Management, Platform Business and Foundation at ServiceNow; Sephanie Louis, Global Head of Developer Community, Pega (keynote panel)
    • Lori Breitbarth, Sr. Principal IT Program Manager at MedTronic
    • Dan Stoll, Director of Technical Product Marketing and Sean Fiene, Senior Technical Enablement Manager at Nintex (joint discussion)
    • Mayur Shah, Senior Director Product Management at WaveMaker
    • Purnima Bihari, Technical Product Manager at Security Compass
    • Boris Crismancich, Head of Business Development at Katulu
    • Mitch Ashley, CEO at ASG and Parker Yates, CRO at MediaOps (joint discussion)

    The virtual environment features auditoriums with three tracks where attendees can watch the sessions by industry experts and an exhibit hall lined with booths showcasing the latest solutions from leading low-code companies. In the exhibit hall, attendees can download resources and interact with partner sponsors including ServiceNow, Appian, OutSystems, Prodly, Skuid, WaveMaker and DB Maestro.

    To check out the full agenda or to register for free, visit the Low Code Today website.