devopsy.kinsta.cloud

Category: DevOps Practice

  • The Role of APIs In Digital Transformation

    The Role of APIs In Digital Transformation

    Digital transformation is increasingly propelled by rapidly changing user expectations. Release frequency is increasing, as is the need to connect a growing number of applications and digital experiences. As a result, 83% of IT specialists consider API integration critical to their business, found the State of API Integration Report.

    Application programming interfaces (APIs) hold powerful sway in tech discussions of late. Building API-first could increase internal reusability. APIs have become a standardized machine-readable mechanism for connecting partners. There is also a hot market for API startups, meaning APIs are becoming products in their own right.

    APIs are also enabling change for large enterprises. I recently connected with Bernd Gross, CTO at Software AG, to get his perspective on the role APIs and integrations are playing within the new wave of enterprise digital transformation. Below, we’ll see how connected customer experiences, digital business excellence and a new ecosystem-driven economy are driving the emergence of more API-based strategies.

    Three Drivers of the Connected Enterprise

    Businesses continue to transform into more connected enterprises. Once relegated solely to improving back-office efficiency, digitalization efforts are being applied across the board — to employee management, assets, machines, partners and the supply chain at large, said Gross.

    Amid rising digitalization, connectivity is the glue that unites disparate elements and tools. A fully connected world is “the new flavor of digitalization,” said Gross. To his point, ‘connected experiences’ is a crucial driver for 68% of platform providers, The State of API Integration Report found.

    So, how are APIs fueling the new connected enterprise? Gross identified three key reasons why integration is now fundamental to enterprise digitalization initiatives.

    Software AG Marketecture, used with permission.

    1. Connected Customer Experience

    Many organizations “want to transform into a technology and software company,” said Gross. To participate in the connected digital economy, organizations are striving to create advanced 360-degree customer experiences. Today’s technology needs to “look beyond a PC application,” Gross said. These experiences must cater not only to desktops and smartphones but also to IoT devices and physical assets.

    For example, he explains how one large elevator company architected an embedded connected experience, incorporating digital touchpoints within elevators to showcase signage, entertainment and ads, opening new monetization possibilities. This change was so radical that the mechanical engineering company pivoted its mission statement, reimagining itself as a “vertical transportation company.” Soon, their AI-enabled elevators will predict behaviors and optimize travel times, so they are always on the right floor, waiting for human riders.

    Intelligent building technology is similarly being applied by digital-savvy hotels, Gross explained, which is especially important for contactless visits during the pandemic. A fully digitalized customer experience could involve automated check-in, a room access code sent to a smartphone, an online billing system, physical sensors and more. Enabling a seamless digital hotel experience sounds easy, but it’s technically quite challenging — there are endless integration requirements to connect. In some cases, it would take 24 integration points, noted Gross.

    2. Digital Business Excellence

    Gross recognized the high expectations for modern digital experiences. He described a “new trend of business excellence in delivering more of an end-to-end digital experience.” This means data can no longer be locked up; even traditional companies must transition to platforms to offer software and data to their end users. In the process, interesting things may occur — your customers could become partners, Gross adds.

    He described how one medical company, a leading centrifuge producer, sought to build more digital experiences to complement their hardware. Though the machine functioned seamlessly, there was no easy way to port data results to other applications. This inhibited initiating the fully digitalized workflow required for a smart lab experience.

    In this case, APIs were imperative to opening up back end cloud services to enable smooth user interfaces and end-to-end workflow automation. “Intuitive user interfaces must integrate all services, including cloud and on-premises services, to create and automate workflows,” Gross explained. “To do this, you need integration.”

    3. Ecosystem-Driven Economy

    The network effect of platforms is that they enable an ecosystem to form. And, according to Gross, involving partners in this new digital experience matrix is imperative for success. For example, the API gateway is one area where partnerships help an integration strategy; “You need an API gateway for security and governance reasons,” he stated.

    Throughout his experience overseeing APIs on many projects, Gross often noticed a naïveté associated with defining the ecosystem. “When [clients] engage in the first phase of these projects, they think they know how to do it, but they typically don’t,” he said. Potential API providers often don’t fully realize their monetization strategy, the security implications or the complementary software providers necessary to support a working model. “Partnerships often facilitate a value-add to customers — they enrich the total offering through APIs and ecosystems,” said Gross.

    Similar to how a machinery company may embed a bill of materials into the product they sell, digital partnership ecosystems can open up data so partners can access and innovate with it, Gross said. Frequently, ecosystem partnerships make a lot of sense, as they can better meet consumer expectations. For example, Gross suggested that integrating a ticketing system into a conditional monitoring system would bring compelling value.

    The Next Stage: Self-Service Integration With APIs

    According to the report, 40% of companies said that digital transformation is driving their need for app integration. Truly connected enterprises must embrace digitalization and integration across all aspects of the business. As a result, this is forcing a business shift toward more open ecosystems.

    But building these systems will require newfound knowledge dissemination around developer experience and self-service capabilities. “Self-service has become fundamentally important,” said Gross. It’s not enough to simply open interfaces — you have to also add self-service capabilities to integration tools, he added. This could mean offering low-code options alongside APIs — such drag-and-drop UIs would help facilitate integrations for more citizen developers. A self-service approach to digital platformification is an emerging trend, yet one that’s “not fully at mass market,” described Gross; it’s still in its early stages.

    Open ecosystems are guided by consumers, who often desire connections between your tool and external partners. But, opening up data isn’t just helpful for partner integration; it could also benefit internal use. As Gross described, time-series industrial analytics could empower operators on the shop floor or production facility to make sense of local data. “You need self-service analytics tools to give them that ability,” Gross said.

    Building new customer experiences to cater to various devices and platforms is a big driver for API adoption. As a result, headless styles are emerging to address multi-platform requirements. For these reasons and many more, APIs are fueling new digital transformations across enterprises, powering innovative customer experiences and hiding complexity in the back end. With APIs, smart processes, smart machines and smart partners can all come together.

  • SecDevOps is the Solution to Cybersecurity

    SecDevOps is the Solution to Cybersecurity

    Anybody paying attention to world news will notice that the threats and risks perpetrated by cybercriminal actors, in many forms, is a serious problem on the rise, affecting individuals, organizations and nations daily. The cybersecurity threat space is indeed alarming and growing rapidly. The cybersecurity solution space is struggling to keep up. After all, there is no such thing as 100% security.

    What is the Solution to Cybersecurity’s Problem?

    In my opinion, the industry has long underestimated the problem of cybersecurity, which has resulted in solutions that do not sufficiently stand up to the cybersecurity threat space. What is clear is that solutions must come from rapid software changes that detect and plug security holes as fast as they are found, and continuously deploy innovative defensive solutions that keep bad actors off their guard. To do that requires Agile and DevOps continuous delivery practices coupled with a security-aware mindset.

    What is a Security Aware Mindset?

    To put things in perspective, it is useful to consider the mindsets of design, QA, operations and security.

    Design mindset – create a product or service that customers want, now. The Agile framework addresses this requirement.

    QA mindset – verify a product or service works the way customers expect it to, when it is released, for all production configurations available at the time of the release. The DevOps framework addresses this requirement.

    Ops mindset – ensure a product or service continues to work the way customers expect, for all production configurations, for the life of the product or service. The site reliability engineering (SRE) and ITIL frameworks address this requirement.

    Security mindset – defend valuables for organizations that create or use a product or service against unintended and malicious actions, by people and organizations, both inside and outside of the organizations, for all production and non-production configurations, for as long as there is something of value to protect. The DevSecOps framework was created to address this requirement; as we have seen, this does not yet offer sufficient safeguards for the growing cybersecurity threat problems.

    The security mindset has multiple additional vectors of thought required compared to the design, QA and ops mindset. This helps us to understand why the cybersecurity problem space is bigger and requires more comprehensive solutions.

    Not long ago I considered security assurance to be a subset of testing practices. In 2016, in my blog, “Seven Pillars of DevOps – Essential Foundations for Enterprise Success“, I did not break out security as a separate pillar. At that time, this type of thinking was typical of developers. Security was someone else’s job. Perimeter defenses were deemed sufficient.

    Soon after, I realized that way of thinking was naïve on my part. By the time that I co-authored a subsequent blog, “Nine Pillars of Continuous Security Best Practices,” I added a separate pillar for security specific practices, and also added security practices within each of the other pillars.

    In parallel, DevSecOps became a thing in the DevOps world. The idea of integrating security practices into DevOps makes a lot of sense, because DevOps is the answer to continuous delivery of software, and security needs continuous delivery of defensive solutions. However, simply adding security to DevOps is not a sufficient solution, because DevOps was conceived to address specific problems associated with design, QA and Ops – not security. Simply adding security as another metric for releases is not enough. What is needed is a re-think of how DevOps can be used to deliver continuous security, as indicated in my book, “Engineering DevOps.” This is the same idea behind what is referred to as SecDevOps.

    In my opinion, it is now time for SecDevOps to gain favor over DevSecOps when it comes to integrating security practices with DevOps practices. This represents a much more significant shift than simply putting security first in the name. With SecDevOps, security concerns within the entire development organization and at every stage in the DevOps value stream, are explicitly prioritized over other types of concerns. This makes sense, given the much wider blast radius of security events versus other quality concerns that DevOps was originally designed to address.

    With SecDevOps, security tools and resources are weighted at the highest priority level. SecDevOps and secure coding practices are required training for application developers. Threat modeling is part of the value stream. Security tasks that are required to mitigate security risks are given the highest priority in backlog planning. At each stage in the DevOps pipeline, security concerns rise to the top of the priority list. Release and deployment readiness are weighted at the highest level. In short, SecDevOps explicitly plugs some important security holes that are not covered by DevSecOps. With SecDevOps, security becomes continuous, not just an add-on to DevOps.

    What This Means

    By explicitly plugging some important security holes that are not covered by DevOps or DevSecOps practices alone, it makes sense for SecDevOps to now emerge as the preferred framework to close the gaps between the growing security problem space with the evolving security solution space. SecDevOps offers the required security-first mindset and solves the need for rapid software changes that detect and plug security holes as fast as they are found, and continuously deploys innovative defensive solutions to keep bad actors off their guard.

  • CircleCI Acquires Vamp to Apply AI to CD

    CircleCI Acquires Vamp to Apply AI to CD

    CircleCI today announced it has acquired Vamp, a provider of an orchestration platform for automating application releases. CircleCI at the same time revealed it has garnered an additional $100 million in financing to attain a $1.7 billion valuation.

    Jim Rose, CircleCI CEO, said Vamp will play a significant role in advancing the capabilities of the CircleCI continuous integration/continuous delivery (CI/CD) platform at a time when the rate at which applications are being deployed continues to steadily increase. In fact, CircleCI claims its platform is now being employed to process 2.5 million jobs processed per day.

    The Vamp platform makes it easier to construct and orchestrate a multi-stage release cycle involving, for example, canary releases of an application. It then employs machine learning algorithms to validate the impact, performance and quality of releases based on the service level objectives defined by an IT team. If anything is determined to be amiss, the platform will then automatically roll back a release.

    CircleCI

    While many organizations have successfully implemented CI best practices, the ability to continuously deploy application releases has proven to be a more elusive goal. Each platform on which an application is deployed has unique attributes. The addition of machine learning algorithms that can provide continuous validation of deployments should help drive a sea change in the way applications are being developed and deployed to improve the overall state of CD, noted Rose.

    Those machine learning algorithms, along with other forms of artificial intelligence (AI) won’t replace the need for DevOps teams as much augment them at a time when IT environments, thanks in part to the rise of microservices, are becoming more complex. In fact, with the arrival of containers, Kubernetes and serverless computing frameworks, it’s becoming next to impossible to manage an IT environment without the use of AI.

    Rose added that, as the line between pre-production and production environments continues to blur, it’s critical for IT teams to more closely align CI and CD workflows. Rather than isolating CD from CI within the context of a GitOps workflow, the goal should be to integrate those processes more closely. It is becoming more apparent that the elements of what constitutes an application are more empirical thanks to, for example, configuration files that are now stored in a Git repository along with application code. That tighter integration ultimately makes it easier to test and validate applications before they are deployed. Metadata that can be analyzed to optimize application development and deployment can also be more easily captured as part of a continuous feedback loop.

    That approach is more critical than ever, because the days of the “Big Bang” deployment of an application update are over. In its place are now a steady stream of updates that deliver new capabilities, which need to be continuously vetted, noted Rose.

    Each successive era of DevOps is, naturally, going to be more automated than the previous one. The challenge is recognizing when those new eras arrive before rivals gain an advantage in speed and agility that becomes too big to overcome.

  • Linux Foundation Embraces Lightweight API Gateway

    Linux Foundation Embraces Lightweight API Gateway

    The Linux Foundation announced today that an open source application programming interface (API) gateway known as KrakenD has become the latest project it will oversee.

    Rechristened the Lura Project, the lightweight stateless API gateway was originally developed by KrakenD, which plans to continue to offer an enterprise edition of the platform.

    KrakenD CEO Albert Lombarte said the API gateway it developed provides a more streamlined approach to managing APIs that doesn’t require an organization to install a separate proxy or database. All the reverse proxy capabilities most organizations require is already embedded within the API gateway. That approach also makes it easier for IT teams to decouple back-end server applications from client applications because now an API can be more easily updated without impacting client applications, Lombarte added.

    KrakenD Linux Foundation

    Organizations that already contribute to the ongoing development of the API gateway include 99P, Hepsiburada, Openroom, Postman and Stayforlong. The API gateway also is embedded in a variety of other platforms, said Lombarte; he noted that the number of servers running the gateway now exceeds more than one million per month.

    API gateways have been available in various forms for years, but with the rise of microservices the number of APIs organizations need to deploy and manage continues to rise steadily. Each microservice added to an environment adds yet another API. At a time when microservices-based applications are at the core of many digital business transformation initiatives, IT teams must employ a more flexible approach to managing the interactions between APIs.

    Thanks in part to the role they play in enabling digital business transformations, APIs are also being targeted more frequently by cybercriminals that have come to view them as an efficient way to exfiltrate data or inject malware into an application environment.

    It’s not clear what impact the Linux Foundation’s support might have on organizations’ selection of one API gateway versus another. In fact, many organizations are still trying to navigate when to employ an API gateway versus relying on proxy software to manage a handful of APIs versus a service mesh to manage thousands. The issue comes down to not just how many APIs there are, but also the overall weight of the platform being employed, said Lombarte. A service mesh typically requires more time and effort to deploy and manage, he noted.

    As the number of APIs in an IT environment continues to increase, there may come a day when organizations will have to revisit how APIs are created and maintained. All too often, APIs are still created as an afterthought once an application is created. They tend to be brittle and, as a consequence, still break often. A more holistic approach to managing APIs based on a set of DevOps best practices is required. The challenge is getting DevOps teams to prioritize APIs when there are so many other issues that require their attention. Nevertheless, it’s become apparent APIs are, in many cases, becoming too strategic to the business for their management to be left in the hands of a single development team.

  • Why Browser and API Testing is Shifting Left

    Why Browser and API Testing is Shifting Left

    For developers working with cloud-native technologies, software testing and quality assurance is a key part of product development. For most teams, testing has “shifted left,” so the quality of the code can be assessed as it moves through the CI/CD pipeline. This approach to quality assurance reflects the rise of DevOps and other modern development practices – developers today own their applications from end-to-end, and conducting testing early in the product life cycle is how they find potential bugs, save their organizations time and money and build products that delight their customers.

    More recently, developers have added end-to-end API and browser tests into their CI/CD practices, extending the scope of their testing and quality assurance practices. By testing how code performs on different browsers and how it interacts with important data sources, developers have added another layer of quality testing to the early stages of their development processes. This has brought added benefits to other “shift left” practices, such as unit testing and security testing, and has led to a new understanding of quality assurance.

    Traditional Approaches to QA Can’t Support the Pace of the Cloud

    As companies have migrated to the cloud and adopted agile development methodologies that align with DevOps practices, they’ve encountered conflicting priorities between releasing code quickly and maintaining the highest level of quality on each release. Traditional manual testing techniques were not built for agile workflows, but instead supported older waterfall methodologies, in which testing could be conducted as a discreet instance before code went to production. With agile development, having a specific team in charge of all testing right before production is not the most sustainable approach, as it hinders the rapid pace of development that the cloud enables. This is why testing is “shifting left” to earlier stages of the development process, and why manual testing is being done by people embedded within each engineering team.

    Cloud-Native Teams Have Shifted Left Into CI/CD Pipelines

    As part of these agile methodologies, companies have moved toward leveraging CI/CD practices to ensure code is always in a state where it’s ready to be deployed. As code moves through these pipelines, developers want to ensure they are able to catch bugs earlier in the development cycle. To do this, they’ve taken to testing earlier and more often.

    In particular, developers have increasingly recognized the value of end-to-end testing throughout their CI builds. API tests allow teams to verify end-to-end workflows by chaining HTTP requests and API calls, so they can validate all the layers of their systems from worldwide locations. Browser tests capture the user’s view during critical transactions, so teams can understand how UI changes are impacting web applications. Shifting both left into CI/CD pipelines saves time and money, as fixing a bug earlier in the development lifecycle is much less expensive than trying to fix it at the last minute, when it has grown in complexity and scope.

    It also means many issues are caught by the developers, without the intervention of another engineer manually validating their work. This reduces the number of cycles required to find a bug, allowing the code to be circled back to developers, fixed and re-deployed to a testing environment. When regressions are caught early, developers can focus on the details that will make the product great.

    Visibility of Metrics, Traces and Logs Drives Value

    Running browser and API tests in your CI pipelines is beneficial, especially when teams can correlate these tests with data from across their stack – front end and back end, application and infrastructure. With correlated metrics, traces and logs that can be accessed easily without toggling between different products, teams will have the full context necessary for both detecting and troubleshooting failed tests. Detecting the failed tests allows for the rollback of bad code before it goes to production, and the correlated full stack data can be used to troubleshoot and fix any issues not caught during the development stage. This creates a new culture of quality assurance, in which maintaining quality is a holistic process that involves every part of the development cycle, and draws on data from every part of the stack.

    Consistent Quality Requires Collaboration

    This change in quality assurance goes beyond the practice of shifting browser and API testing left, and represents a fundamental change in how engineering teams collaborate within an organization. Testing happens before code goes to production, continues with robust monitoring once code is live, involves data from across the stack and is the responsibility of teams from various parts of the business. With this collaborative approach, for example, a front end engineer can easily identify an issue in the integration of their latest changes with the way the back end APIs are structured. They can then engage with the back end team owning this service to collaboratively iterate and find the best solution to resolve the issue. It may start with a shift left, but, ideally, it ends with the removal of silos and the opening of communication channels among teams.

    Automated QA Delivers Customer Value Faster

    A fully operational approach to quality assurance means that issues are resolved faster, or even resolved while code is still in development, so they never become issues at all. This creates a virtuous cycle – fewer regressions means less downtime, which makes customers happy, and frees up engineers to build new products, which makes customers happier still. That baseline of reliability and customer experience enabled by effective quality assurance is what allows for new innovation that delivers value to customers.

  • The Evolution of SRE and Developer Roles

    The Evolution of SRE and Developer Roles

    As an industry, we’ve seen cultures and roles shift throughout IT teams — specifically, the role of ITOps site reliability engineers (SREs), various developer roles and the culture of DevOps. Most of these roles operated – or still operate – in silos, and didn’t understand (or care to understand) how each other’s role worked. More importantly, they didn’t know how to work together. But these roles are starting to overlap, driven by the increase in demand for customer experience and, subsequently, in intelligent observability, or observability with AIOps. Before we dive into the ‘how’ behind the overlap, let’s look at how each of these roles performs today.

    Developer

    While the specifics of what a developer does varies by project, the overarching responsibilities haven’t changed much since the introduction of the role. Simply put, developers write code to create and improve product functionality.

    IT Operations and SRE

    Traditionally, an IT operations professional focused on operating the code a developer put into production; aka keeping the lights on. However, we’re now seeing evolution of the role of highly skilled operations pro to that of an SRE who oversees automation and features surrounding the product, like CI/CD pipelines, infrastructure-as-code and the ability to monitor and observe the services and components within.

    QA

    Traditionally, QA would test the code the developer wrote and move it down the deployment pipeline. But we’re moving away from traditional QA. Now, the role sets up an automated framework for the developer so they can automate tests — similar to how the SRE sets up the CI/CD pipeline and observability framework to build, test, deploy, verify and monitor their code.

    DevOps

    DevOps has, in some cases, evolved from a practice and culture into a role. The DevOps engineer prepares the automation and the CI/CD pipeline for the developer to push out their code to test and, ultimately, production. But let me be clear – I believe the idea behind the practice of DevOps is that the practitioner does everything — they own testing frameworks, CI/CD pipeline and infrastructure. They own end-to-end product development and operation, so when anything does go wrong, they know exactly what to do, where to go and how to fix it.

    In the end, if a SRE can develop everything around the product code and knows how to fix the product — isn’t an SRE actually a developer? What if we have a really skilled developer who sets up the automation and observability framework to get their code from their keyboard to the user as quickly as possible — without breaking anything, of course — while also building and improving all the infrastructure and components surrounding the product? Isn’t the developer an SRE, or at least performing site reliability engineering?

    The short answer, in my opinion, is yes. And there’s one guiding force behind this amalgamation, this crossover; ultimately, this evolution: intelligent observability.

    The role of observability with AIOps

    Traditionally, and with the use of many tools, DevOps practitioners and SREs have had to — manually — battle floods of data coming in from all their endpoints, services and products. This can feel like a never-ending struggle. It can lead to a lot of toil and time spent on maintenance, when what developers really want is focused time to do what they love – write code, create and innovate. Not to mention, the result has a negative impact on the customer experience.

    But now, new tools are surfacing to support the everyday needs of SREs and DevOps practitioners. For example, intelligent observability, or AIOps, applied to observability data like events, metrics, traces, changes and logs, gives teams clearer insight into not only where complex data is coming from, but why an issue is happening and what action they need to take to resolve it. They can spend less time operating and more time innovating.

    As these roles continue to evolve, it’s important SREs and DevOps practitioners remain agile and willing to adjust to keep up with the latest technologies supporting their day-to-day workflow. AI can be the key to keeping the incident count down and the customer experience up.

    Looking Ahead

    If you’re currently in one of these roles, consider how you are practicing. Whether you’re a developer, a QA engineer, a DevOps practitioner, or an SRE, have you evolved your role to fit your company and customers’ needs right now? As these roles overlap with the application of intelligent observability throughout your development life cycle, it’s possible to deliver better products and fix reliability issues faster. It’s up to you to decide if you want to embrace the evolution of your role and intelligent automation or stay put and risk falling behind.

  • Why We’re Swimming in Dashboards and Drowning in Data

    Why We’re Swimming in Dashboards and Drowning in Data

    Knowing how to use a dashboard is table stakes in the enterprise. Information technologies generate countless volumes of data from multiple sources, and dashboards are meant to harness all of that information, reduce the noise and display the stats you need to do your job. Yet businesses are still struggling to unlock the value of their data, with over half of global businesses (57%) claiming they can identify and capture just 70% of their data, according to a study conducted by IDC and commissioned by Qlik.

    As in “The Rime of the Ancient Mariner,” there is water, water everywhere, but not a drop to drink. Enterprises are inundated with dashboards and still struggle to make effective decisions with the information in front of them.

    The Limits of Analytics

    There are many reasons why a dashboard may communicate ‘bad’ or ‘useless’ information. Analytics can’t do it all. A data pipeline may be parched with too little data or flush with the wrong kind of data, limiting the volume and/or value of outputs that can be drawn into the dashboard. The organization may be heavily siloed, rationing the range of information available to the dashboard. Or, the tooling itself might be inadequate for leaders to quickly respond to opportunities and disruption.

    When a decision-maker looks at a dashboard, they want to see things as they could be, as well as how they are. That requires a dashboard that depicts the data itself, but that also generates views of, for example, how it looks when sales activity in the Southwest is increased in the face of a weather event or other externalities. Results don’t have to be taken at face value – they can be analyzed more carefully and intuitively to uncover meaningful insights.

    Investigatory analysis is what modern, interactive dashboards and applications are all about. They support non-linear exploration and search and provide advanced features, such as augmented analytics, to help subject matter experts explore data and generate insights without relying on IT, data scientists or expert analysts. Interactive dashboards also make it possible for business intelligence to be truly self-service, simplifying aspects that were once limited to experts working in much more complex environments. They give all employees access to the insights they need to make better decisions, regardless of their skill level, essentially democratizing analysis across organizations.

    How to Choose the Right Dashboard

    For any business looking to really expand decision making across the business, here’s how to find a dashboard suited for investigatory analysis:

    Accessibility and timeliness: In the past, business users who wanted insights had to submit a request to an analyst, wait for their turn in the queue and go back and forth answering clarifying questions before they could review a dashboard and make decisions. The process took too long, and insights came too late. With modern, interactive dashboards, users can explore all their data using interactive selections (filters), search, and NLP (natural language processing) powered by AI. In the end, they can ask and answer all their questions leading to informed decisions.

    Flexibility: Modern dashboards are not just static displays of content with limited filtering. They are rich analytics applications that provide unmatched interactivity and exploratory power. Business users aren’t limited to rigid, linear analysis on partial subsets of data. Instead, they have the freedom to explore all their data however they like. Analytics are always in context together, giving users peripheral vision, and all data relationships are exposed – including what’s related to your selections and what’s not.

    Performance: Interactivity should be driven by outcomes. Novel techniques to optimize and store data, index and understand relationships and deliver high-performance calculation all help to go beyond the limits of query-based and RDBMS architectures. Interactivity should also be enhanced with AI to deliver next-level analytical power, in new forms, across a full range of user experiences.

    A Crucial Tool in the Stack

    Turning data into business value is what a dashboard should—and can do—for the business. When a business user looks at a static dashboard or visualization, they usually end up with more questions than answers. To make an informed decision, they need to go through a process of questioning and evaluating – and then questioning again – an interactive display of business-critical trends until the right insight is fully uncovered. This iterative process is unique to every user and their lines of thinking, and modern dashboards allow all users, beyond data experts and analysts, to build visualizations and queries.

    In addition to creating business value, interactive dashboards help rally key stakeholders in organizations and allow them to act quickly to changing market needs. Take, for example, a major luxury fashion retailer; parent company of three well-known apparel brands. Their businesses hinge on having the products consumers want and need, when they want and need them. Their assortment planners need to analyze their sales mix and inventory mix by product attribution to identify mismatches and, more importantly, understand the drivers behind the mismatches to make more accurate inventory recommendations. Interactive dashboards allow them to investigate their data in different ways and ultimately inform strategic decisions for the business moving forward.

    In addition to empowering users to transform data into business value, dashboards ultimately drive the need for data literacy. Sharing dashboards throughout all levels of an organization fosters a data-driven culture by encouraging exploration of data – and that’s how new discoveries and innovations can be made.

    So while some say the dashboard is dead, I say long live the interactive dashboard.

  • The Basics of DevSecOps Adoption

    The Basics of DevSecOps Adoption

    Embracing DevOps is essential, but keeping the software delivery and deployment pipeline safe has become more critical than ever. While attackers are keen on getting into your Docker containers or Kubernetes clusters, securing your images, pods and clusters can deter attackers. Securing your artifacts, deployment workloads and production environments remains critical today. While we have many tools to get these jobs done, it is vital to understand DevSecOps and its related factors.

    DevSecOps has become a mass movement today. But what makes it so important?

    What is DevSecOps?

    DevSecOps addresses software development security via the mindset that everyone is responsible for security at every stage in the development process.

    DevSecOps is a culture and an evolution where security is given the utmost priority in the software development life cycle (SDLC). Security is everyone’s responsibility, and inspection is done at each stage of the development cycle. Basically, implementing a process for inspecting, scanning and reviewing code by developers at predetermined checkpoints to immediately tackle any identified bugs or vulnerabilities. One of the most challenging parts of the transition from DevOps to DevSecOps lies in building more open and communicative teams.

    Why DevSecOps?

    Using DevSecOps, security becomes an integral part of the planning, coding, building, testing, deployment and production stages. DevSecOps makes continuous testing the default for developers and security professionals to deliver more secure features to the market more quickly.

    In many organizations, most development, operations or DevOps teams see security or security teams as a roadblock to getting things done faster; delaying features. In traditional DevOps implementations, the security aspect was missing, DevSecOps was introduced to stress the priority of security in DevOps.

    The Importance of DevSecOps

    Let’s take Uber’s security breach in 2017 as an example. Personal details and information of 57 million customers and over 600,000 drivers was exposed. Uber paid a $100,000 ransom to hackers in an attempt to get them to delete the breached data. This breach happened because engineers failed to secure and update the credentials they were using on GitHub.

    Similarly, in 2018, Tesla’s Cloud was hijacked and used to mine cryptocurrency by abusing a vulnerability in the company’s Kubernetes cluster. A significant amount of FedEx data was also exposed in the same way, affecting 119,000 individuals.

    Logz.io’s annual DevOps Pulse 2020 survey revealed that, out of 1,000 respondents, only 3% indicated their development methodology as DevSecOps, hinting at the slow pace of DevSecOps adoption in the industry.

     

    Image source: Logz.io

    DevSecOps Best Practices

    Adopt a Shift Left Approach

    Shifting security left helps organizations secure each stage of the development life cycle. This refers to starting testing earlier in the SDLC. The goal here is to enhance quality, cut down long test cycles and reduce the likelihood of unpleasant security surprises at the end of the development cycle or in production.

    Include a Threat Modeling Mechanism

    It is essential to implement a threat modeling mechanism in DevOps workflows since it helps developers and security professionals to view software from an attacker’s perspective. It makes developers more careful while writing code and encourages them to more closely follow code writing and review best practices. Threat assessment or threat modeling will also help identify potential vulnerabilities in system design and architecture, and acts as a crucial part of successful security integration in the SDLC.

    Mandate Developer Training

    Training developers on the importance of security is a critical best practice. Making sure each developer who writes, tests and deploys code understands that security is part of their job description is a priority. Training developers on security best practices, how to embed security, check for vulnerabilities and use appropriate security tools will improve overall security knowledge and increase its application within the organization. 

    Use Available Security Vulnerability Tools

    A vulnerability is considered a flaw in the code, application dependencies, external libraries, licenses used or operating system (OS) packages. There are a number of static and dynamic application security tools that can help the development team to find and address bugs and vulnerabilities as early as possible in the development pipeline. Many are automated to reduce the time it takes for developers to find these flaws.

    Sponsor Bug Bounties

    Bug bounty programs and rewards motivate external and internal security experts to experiment with the platform, analyze, think outside the box and report any major bugs and security flaws found. 

    Security-as-Code

    Security-as-code is the next evolution in DevOps, where security is fully embedded in the development processes. This practice implements security and secure coding practices in the earliest stage of development. With security-as-code, developers will be able to identify vulnerable code ahead of time and add security checkpoints/measures to tackle such irregularities. 

    DevSecOps Tools

    When we talk about DevSecOps, the software development cycle involves multiple checkpoints to identify bugs and any other vulnerabilities and resolve them so that the application works as designed. The aim is to deploy software with quality and confidence, and this is achieved by using various tools at different stages.

     

    Image credits: CatchPoint

    Organizations can choose to use the type of security tools that best work for their needs from their preferred vendor. While the security tools themselves are critical for detecting vulnerabilities in the application through the DevSecOps pipeline, monitoring is also considered an integral part of this process. It not only lets developers analyze and evaluate security vulnerabilities from the end user’s viewpoint, but it can help identify breaches and anomalies after an application is in production.

    DevSecOps Pipeline

    While DevOps focuses on speed and agility, DevSecOps adds a layer of security throughout the software development life cycle. A DevSecOps pipeline adds security activities to every stage of the development process, not simply tacking it on to the end as an afterthought. This is achieved by integrating security checkpoints, controls, policies, tools and techniques from the beginning of the DevOps pipeline until the end. These additions facilitate automated security controls at each stage of the software delivery pipeline.

    Image credits: JFrog

    In his swampUP conference keynote The Divine and Felonious Nature of Cyber Security, John Willis shared some valuable DevSecOps best practices, including:

    • Treat security issues as you would software issues.
    • Adopt a “security-as-code” approach to enable the automation of security.
    • Build security controls and vulnerability detection into CI/CD pipelines.
    • Automate security testing as part of the build process.
    • Proactively monitor the security of production deployments.

    Embedding security earlier in your software development life cycle will yield bug-free features and satisfied customers. Have security checkpoints at each stage and make sure developers understand that security is everyone’s responsibility, including theirs. Manage your software and build artifacts, scanning and securing them at every stage of the pipeline using appropriate tools and techniques. Happy DevSecOpsing!

  • Empowering Developers Through Lean Code

    Empowering Developers Through Lean Code

    Digital transformation is widely understood as a critical component for increasing user adoption, fueling workplace productivity and innovation, and driving business growth. Software has become increasingly important across countless industries, teams, and functionalities, and as a result, companies are beginning to understand that being an industry leader and gaining competitive advantages starts with software development.

    Easier said than done. Modern software apps are a complex orchestration of infrastructure, deployment pipelines, notifications, user experience, data filtering and processing, and error handling. A good engineering team will be able to account for each of these items and integrate them into a larger, comprehensive infrastructure.

    So what happens when you don’t have a solid, in-house team guiding your software maintenance and development? Consider a lean code development platform; a developer service that pares down the explicit requirements of standard functions, interactions, errors and software infrastructure into implicit scripts, templates, configurations and API requests. Using a lean code solution will give your team the power to build higher-quality features more quickly than they otherwise would on their own.

    There are several ways that lean code platforms can simplify processes for developers. Primarily, these platforms turn explicit requirements into implicit interactions, eliminating the need for developers to pull elaborate infrastructure to solve complex problems. Lean code also gives developers the freedom to customize everything to fit where it is needed, while limiting the amount of code necessary to do so. Lastly, lean code platforms provide tools that assist developers at all levels of the software stack, significantly increasing productivity while cutting down on time and workload.

    How Does it Differ from Low-Code?

    While many people might be unfamiliar with the term “low-code,” they’d probably be surprised to learn that they use it all the time. Low-code describes platforms that require minimal code in order to build applications and processes.

    Essentially, these programs enable users with little to no coding or software development experience to create and use applications. The idea of low-code is to introduce audiences to intermediate level software development concepts such as object-oriented programming, interface design, conditional logic and code syntax.

    However, no matter how helpful these low-code approaches might be, many users still run into the problem of wanting more customization than the low-code system is designed to give. In fact, many users become frustrated with the term “customization” because it’s viewed as the enemy, indicating a need for more advanced software development services than the consumer is able to provide.

    Lean code platforms, on the other hand, are designed to specifically meet developers’ needs at all stages of the software development life cycle. In essence, lean-code prioritizes simplicity, efficiency, flexibility and accessibility with the goal of making software development accessible on a larger scale.

    Instead of encouraging you to reduce your development team, lean code platforms allow users to expand development through exploring new avenues and abstracting away the explicit requirements of traditional software development.

    What Does Lean Code Have to do With Productivity Infrastructure?

    Many forward-thinking companies have gone beyond simply building technical solutions to specific development problems. Instead, they’ve taken the advancement of lean code platforms to a new level by providing fixes that meet developers at the central stages of the integration process.

    Because lean code offers such integrated solutions to developer challenges, the approach is increasingly becoming the norm across the bulk of the productivity infrastructure category — an emerging market category that combines elements of cloud communications services, API platforms, low-code development tools, business process automation and AI software development kits.

    If you’re looking to implement a lean-code platform, consider the following features:

    • Intelligence: By making it easy to pull information from unstructured data sources and flip it into more meaningful material, machine learning (ML) replaces the manual effort to build, implement and maintain models.
    • Automation: Workflows, triggers, templates and notifications can be abstracted away for common use cases.
    • Security: When handling user data, security is a top priority. From protecting health records to pet names, data security is crucial.
    • Connectivity: The ability to connect to a relevant data source can make or break an infrastructure.
    • User experience: User interfaces should follow expected conventions for whichever task the user is trying to accomplish. Maintaining a successful user experience requires constant evaluation.

    How will Lean Code Become Market-Leading Software?

    Lean code development platforms have emerged as market-leading software development solutions in three major areas:

    • Productivity Infrastructure: More companies have pivoted to providing full-stack development services that assist developers in building out native communications, search and indexing, scheduling and financial experiences.
    • Software development and deployment workflows: Continuous integration/ continuous deployment (CI/CD) pair down complex build and deployment pipelines to basic configuration files.
    • Infrastructure deployment: Infrastructure as a service (IaaS) products assist in deploying scalable web services via API requests and simple configurations.

    Where are lean code development platforms headed?

    It’s safe to assume that, given their ability to provide efficient and effective infrastructure to development teams, lean code has a bright future ahead. The number of developers in the space is only growing. A Slash Data survey from 2019 estimated there to be just under 19 million active developers across the globe and expected the number to grow to 45 million by 2030.

    As the market for software development rapidly expands, it would make sense to assume that the need for a platform that solves problems across the entire software ecosystem would, too. Lean code development platforms are emerging as a top choice for companies that want to build out a leading software platform into the future.

  • Using RPA for DevOps Process Automation

    Using RPA for DevOps Process Automation

    Modern operations teams today are at the cutting edge of automation. DevOps and SRE practices are entirely based on an automation-first philosophy. CI/CD is fully automated and the next generation of automated testing is now being adopted.

    However, these same teams are still running their human processes based on hastily thrown-together wiki pages. The increase in remote work over the last year has amplified the problem. Engineering teams that were used to quickly getting the right people physically into a war room now struggle to find the people with the right knowledge to troubleshoot or resolve a problem.

    Teams with ample resources have built tools and bots for DevOps process automation and orchestration. These tools can help automate things like getting communications channels set up quickly, with the right people, the moment an incident is identified. Or, they may automate actions across the DevOps toolchain that would otherwise live as static documentation for an on-call engineer to try to decipher and execute. But only a handful of organizations have the resources to build this kind of automation technology in-house.

    These higher-level DevOps processes are ripe for innovation similar to robotic process automation (RPA). Companies like UiPath and Automation Anywhere have shown that there’s an enormous market for this in every business unit and industry. Process automation for DevOps needs to be another layer of automation available to any engineering team. Automated remediation of incidents and resolution of service desk tickets has to be the future of this space. However, there are important differences in how automation is optimized for DevOps versus the application of RPA for business functions.

    Where RPA Fits on the Automation Spectrum

    RPA is great for solving deterministic problems – those that can be resolved in the same way every time you run into them – which are fully automatable. An example of this type of end-to-end automation is any back-office task, which are consistently good candidates for the application of automation. It’s the reason why fields like HR and finance have made some of the greatest progress in automating business processes to date. Every time you close the books, for example, the steps are pretty similar.

    Process automation for DevOps and SRE, however, is on the other end of the spectrum. Although teams may benefit from RPA for simplistic, repeatable tasks, many DevOps processes are dynamic and unpredictable, making full end-to-end automation too often fallible. The complexity of modern technology stacks and infrastructure requires processes to be agile and adaptable to change, rather than deterministic. This is where it’s important to enable humans to step into the spectrum of automation. Human-in-the-loop automation – the idea that lets humans be in the cockpit while letting machines provide guidance and support – is the future of DevOps automation.

    Applying Process Automation to DevOps

    To use automation in a meaningful way, DevOps teams need a mix of what RPA provides with the added element of human judgment. Human-in-the-loop automation can be applied to numerous DevOps processes including incident management, service requests and continuous deployment and delivery.

    Rather than automating from end-to-end right from the start, DevOps teams should use incremental automation to strengthen and build trust in automation. This starts by codifying free-form processes so teams can more easily identify predictable steps that can be automated. Then teams can mature to more complex workflows, automating pieces within processes and incorporating human checkpoints for areas that will need judgment or approvals.

    As teams incrementally automate, they build up sustainability – in their systems and their roles – in high-stress environments and are able to orchestrate processes across the continuum. The goal is to find the right balance between leveraging machines to make lives easier while creating room for human judgment and adaptability.

    Level-Up DevOps Automation

    Looking ahead to the next few years, what the RPA movement did for back office tasks will also happen for DevOps and SRE. Technology leaders can take notes from the RPA movement and apply them to eliminating the toil of modern production operations. The issue is that, in DevOps, many problems are unique.

    The next evolution of DevOps automation will force us to look at how we can model the real world in a more holistic way, which includes incorporating human judgement and manual execution of non-repeatable tasks while automating repetitive steps.