devopsy.kinsta.cloud

Category: Enterprise DevOps

  • Top Misconceptions About Low-Code Development

    Top Misconceptions About Low-Code Development

    Accelerating the pace of development has become one of the biggest goals in DevOps. After all, having teams speed the creation and modification of applications is supposed to be one of the primary advantages of DevOps pipelines. Yet, far too many DevOps teams run into speed bumps that can derail the process. For some, turning to alternative methods to generate and modify code can slay the beast of project deceleration, and that is where low-code platforms come into play.

    However, the waters are muddied by misconceptions about what low-code is and what it isn’t. Here, three industry experts share their own observations about these common misconceptions.

    Low-Code is Only for Citizen Developers

    “People often confuse low-code with no-code platforms,” said Richard Rabins, CEO, Alpha Software Corporation. “No-code platforms, which are driven by visual modeling and user interface builders, are aimed at citizen developers with little or no coding experience to quickly build applications.” Rabins makes some good points; no-code environments are designed to put the power of application development into the hands of a citizen developer, meaning those no-code platforms are optimized for simple workflows and not for teams. On the other hand, low-code platforms can integrate with deployment platforms, which helps expedite delivery while also enforcing consistency.

    Low-Code Makes Collaboration and Reusability Difficult

    “Over time, many developers have come to believe that low-code platforms were built for a single programmer working on a waterfall project,” said Deepak Anupalli, co-founder and VP of Engineering at WaveMaker, Inc. “Today, low-code platforms, like WaveMaker, offer development environments that integrate with code management tools, as well as APIs, to support the sharing of code and fostering team collaboration.” Anupalli’s point is that the majority of low-code platforms incorporate workflows that support team activities. What’s more, those platforms support scale-up, reuse of code and the ability to work within pipelines.

    Low-Code Prevents Custom Development

    “From the beginning, low-code was all about building applications with little or no code needed, creating the perception that low-code platforms did not lend themselves well to customization,” said Rabins. “However, nothing could be further from the truth. Developers are able to expand the functionality of low-code projects as the need arises. They can insert custom code, call APIs, integrate with other platforms, and so on.” Rabins points out that low-code doesn’t mean no-code, simply that low-code can generate code that can be modified and adapted as needed, supporting pipelines that deliver fully customizable applications.

    Low-Code Platforms are Only a Trend

    Many seem to think that low-code platforms are just a trend that will ultimately fade away as other technologies come to light. However, according to Business Wire, the low-code market will grow 44.4% by 2022 to reach $27.23 billion. Anupalli added that WaveMaker has experienced year-over-year growth since the company was founded.

    Leaders in the market today include Visual LANSA, GeneXus, Appian, Mendix, OutSystems, WaveMaker and Alpha Software. 

    Low-Code Solutions are Proprietary

    One of the leading assumptions about low code platforms is that they all use some type of proprietary programming language. That may be true for a few platforms, but many embrace open source ideologies and use common languages, such as Java, PHP, and C#. Take, for example, the RISE Editor, which enables users to output code based on models in a number of different languages. RISE is not alone in the industry; several other platforms embrace either open source or can create easily accessible code. VisionX and WaveMaker create in Java, allowing generated code to be edited with other Java IDEs.

    Low-code development platforms (LCDPs) have come a long way since the term was first coined by Forrester Research back in 2014. And yet, these numerous misconceptions remain, which may have slowed the adoption of LCDP in enterprises using DevOps pipelines. Dispelling these assumptions goes a long way towards helping application developers realize the benefits that LCDP can offer. 

      

     

  • SolarWinds Unveils Revamped APM Platform

    SolarWinds Unveils Revamped APM Platform

    SolarWinds is unifying its various application performance management (APM) tools within a single suite accessed via a software-as-a-service (SaaS) platform.

    Denny LeCompte, senior vice president for products at SolarWinds, said the APM Integrated Experience offering combines the company’s AppOptics, Loggly and Pingdom tools to collect metrics, traces and logs along with real user monitoring capabilities and tools to simulate application traffic.

    APM Integrated Experience also provides access to a landing page that identifies what has changed in an IT environment over the previous week and highlights which of those changes might become an issue. As most IT professionals well understand, most IT issues can be traced directly back to a recent change made to an IT environment that was previously running as intended.

    Other capabilities include advanced search tools and the ability to inspect lists and maps for hosts, containers and plugins for web servers and databases. In addition, APM Integrated Experience is integrated with both Amazon Web Services (AWS) and the Microsoft Azure cloud.

    LeCompte said the goal is to make it less expensive for the average IT organization to broadly instrument applications using an agent developed by SolarWinds that can automatically be installed.

    Ultimately, the goal is to enable organizations to better manage their software supply chains, LeCompte added. Of course, SolarWinds is at the heart of the software supply chain breach that involved its Orion network management software. That software runs on-premises. In comparison, APM Integrated Experience is a SaaS platform that is more secure. In fact, LeCompte said, IT professionals that manage software supply chains are generally sympathetic to the challenge the company currently faces. Most of those organizations have dealt with similar software supply chain issues, albeit on a smaller scale, LeCompte added.

    In general, LeCompte said it’s apparent most organizations need access to an APM platform that costs substantially less than rival platforms and still delivers 90% of the capabilities they require. In many cases, IT organizations are paying for extra capabilities that only very high-end enterprises require, LeCompte noted. In fact, one of the primary reasons APM platforms are not as widely employed is most organizations have found rival APM platforms simply too expensive to deploy, said LeCompte.

    LeCompte declined to speculate whether there is a meaningful difference between what has been historically known as monitoring and so-called observability platforms that have emerged in recent months. Regardless of approach, however, it’s clear as distributed IT environments become more diverse and complex, the number of organizations that need more visibility into their application environment is steadily increasing, said LeCompte.

    Of course, with the rise of microservices-based applications, that level of IT complexity is only going to increase. As applications grow and expand, the number of dependencies between microservices starts to increase. Understanding how an issue with one microservice is impacting a distributed computing environment will undoubtedly require more instrumentation. The challenge, going forward, will be determining how best to achieve that goal at a reasonable cost.

  • Why Every DevOps Team Needs a Spot Instance Strategy

    Why Every DevOps Team Needs a Spot Instance Strategy

    Most DevOps teams use the public cloud extensively and focus a lot of energy on reducing cloud costs. According to one estimate, U.S. businesses spent $14.1 billion in 2019 just on wasted, unused cloud resources. Spot instances are one of the most important ways to reduce the cost of public cloud services.

    Spot instances are not a new development. Amazon was the first to announce this pricing option in 2009, turning cloud computing into a market influenced by the dynamics of supply and demand. Microsoft Azure took more than a decade to follow suit, announcing its spot virtual machines offering in May 2020.

    The spot pricing model is simple on the surface but can be complex to implement. Spot instances are how cloud providers sell their unused capacity. When compute instances are not ordered by anyone via the regular, on-demand pricing model, the cloud provider puts them up for auction at very favorable prices, usually around 10% to 20% less than the on-demand cost.

    However, it’s not so simple to get this 80% to 90% discount. When another cloud customer requests the spot instance, the cloud provider sends a notification, and you need to immediately move your workloads before the instance is terminated. This makes it difficult to use spot instances for stateful, database-driven applications, or those that require high availability.

    This is not the only challenge. The market price fluctuates and is much more volatile than on-demand pricing. This means the level of discount that can be achieved over time is relatively unpredictable.

    However, there are at least three ways savvy DevOps teams can make great use of spot instances:

    1. Automation—DevOps teams are proficient with a range of automation tools, including configuration management, infrastructure as code (IaC) and cloud provider auto-scaling tools. These can be used to manage workloads in clusters and automatically fail over when a spot instance terminates.
    2. Dev/test environments—DevOps teams are responsible for setting up development and testing environments, which are extremely suitable for spot instances because they can usually tolerate brief interruptions, and, in many cases, are not stateful.
    3. CI/CD jobs—running jobs on Jenkins, GitLab and similar tools can be easy to scale on spot instances. These jobs are stateless, and if an instance drops, it’s easy to rerun the job on another.

    AWS Spot Instances

    AWS Spot Instances let you buy unused Amazon EC2 computing power at a significantly discounted price. You can specify a price, and when a spot instance is offered at that price, it is launched with the Amazon Machine Image (AMI) of your choice.

    How Spot Instances Work on AWS

    Spot instances are priced at a variable spot price, which is adjusted according to supply and demand conditions. To see current rates, use the AWS Spot Instance Advisor.

    You create a spot instance request, specifying what instance types you are interested in, and the availability zones (AZ) in which they should run. If capacity is available, and their current price is less than your maximum bid, instances are launched.

    Spot instances continue running until:

    • Capacity is no longer available (because the instances were requested by on-demand customers).
    • The price has risen over your maximum bid.
    • You request to terminate the instance, or it is automatically terminated by auto-scaling.

    You can also order spot instances with a predefined duration—you then pay a static hourly rate for that entire duration (even if market price changes in the interim).

    Automation Options

    You can automatically scale instances on Amazon EC2, including both on-demand instances and spot instances in a single auto-scaling group. If spot instances are not available when scaling up, the group can use regular on-demand instances.

    Amazon also supports mixing in reserved instances and savings plans (additional ways to save on on-demand instances by committing to a certain period of time or total capacity). So you can combine multiple saving methods in the same auto-scaling group.

    You can improve availability by deploying applications across multiple instance types running in multiple AZs. By allowing multiple instance types, you tap into multiple pools and increase the chances of obtaining a spot instance when you need it.

    Azure Spot Instances (Spot VMs)

    Azure offers Spot VMs that give you access to unused compute capacity. You can request a single spot VM, or launch multiple spot VMs using an Azure VM Scale Set (VMSS). Spot VMs replaced the previous Low Priority VMs feature, which let you purchase VMs that were in low demand on Azure for a reduced price.

    The spot price of VMs on Azure depends on the total capacity available for that specific instance size and SKU (instance type) in the Azure region. Azure commits to changing pricing slowly—avoiding sudden spikes—to maintain pricing stability and make it easier to manage budgets.

    Like on Amazon, discounts fluctuate significantly, and spot VMs can be up to 90% cheaper than the base price of the same VM.

    How Spot VMs Work on Azure

    The Azure Portal provides access to Azure spot VMs. When you create a spot VM, you can see the current price for the selected region, image and VM size. For consistency, prices are always in U.S. dollars, even if you use a different base currency for billing.

    There are two options for eviction of spot VMs—you can choose the condition on which spot VMs will be evicted:

    • Maximum price eviction—You set a maximum bidding price, and when the spot VM rises over that price, it is evicted.
    • Capacity eviction—You always pay the current price of the VM (without setting a maximum price), and when Azure does not have sufficient capacity of the requested VM type, your VM is evicted.

    When a VM is evicted, Azure applies an eviction policy called Stop / Deallocate. This means the instance is paused, but attached disks remain, and you are still charged for them. When the price goes down or capacity becomes available, the instance is restarted and continues working on the same disk data.

    Automation Options

    Azure provides virtual machine scale sets (VMSS), which can automatically increase or decrease the number of VMs running your application. You can create a scale set that includes spot VMs, and as your application scales, more spot VMs will be added as they become available. Spot scale sets operate in a single fault domain and do not guarantee high availability. Unlike AWS, Azure currently does not allow you to mix on-demand VMs and spot VMs.

    Both Amazon and Azure provide robust capabilities for cost savings using spot instances. Azure’s offering is newer and provides less-advanced bidding and auto-scaling capabilities, but these are expected to be added as the service matures.

    Whether DevOps teams choose to run in AWS, Azure, or both, they literally cannot afford to ignore spot instances, especially for low-criticality workloads like dev/test and CI/CD job execution.

  • Engineering Practices Can Overcome DevOps Challenges

    Engineering Practices Can Overcome DevOps Challenges

    When I think about the challenges of DevOps transformations, I love the viewpoint expressed by Ryan Harmon, strategic accounts manager at Trace3: “DevOps is a complex maze that has many leaders frustrated. In many large enterprises, ‘DevOps’ is simply a buzzword they are striving to achieve but struggle with what the end state looks like or even where to start. There are many layers across the organization that are instrumental to truly executing on DevOps. DevOps is not something you go get a quote for and simply buy. It’s an evolving journey.”

    In my previous article, The Time Has Come for an Engineering Approach to DevOps, I agree with Ryan.

    Too many organizations launch into a DevOps transformation initiative without considering the full scope of what it takes to engineer a successful DevOps solution. DevOps is more than “just” a CI/CD pipeline or toolchain, as complex and challenging as this alone may be to accomplish, for even one application. DevOps is more than simply a culture change, as difficult as that always is.

    There are 24 DevOps practice categories that need to be mastered to realize and evolve well-engineered, high-performance enterprise DevOps solutions. Not all 24 need to be mastered to begin the DevOps transformation, but, ultimately, all of them do need to be mastered before the solution will achieve the high performance levels to which many enterprises aspire. The good news is, successful enterprises have demonstrated that the investment in people and technologies required for the journey is worthwhile, and a strategic approach will get you there systematically if you follow a disciplined engineering implementation roadmap.

    As explained in my book, Engineering DevOps, the core of every successful DevOps practice includes nine core pillars. These are the pillars to focus on as you start on your DevOps journey. The following paragraphs illustrate requirements for each of the nine pillars.

    Leadership – Leaders set an inspiring DevOps directional vision for the organization, and proactively stimulate and sponsor team activities that help progress toward goals.
    Collaborative culture – The organization encourages cross-functional collaboration, shared responsibilities and works to reduce restrictive boundaries between organizational functions.
    Design for DevOps – Products are architected to support service-oriented, modular, independent packaging, testing and releases, in accordance with the principles of twelve-factor apps.
    Continuous integration – Changes to software code and artifacts needed for releases are frequently merged, built, tested and packaged into executable artifacts for application releases.
    Continuous testing – Software tests that are most important to verify the results of pipeline stages including pre-flight, integration, regression, performance and release acceptance tests are automated as much as possible.
    Elastic infrastructure – Infrastructure-as-a-service (IaaS) resources (i.e., software, computing machines, storage and networks) are integrated into a toolchain for automating builds and testing environments to efficiently support variable workload demands.
    Continuous monitoring – Applications, pipeline tools and infrastructure components are instrumented with metrics, and the collected data is analyzed to manage health and performance.
    Continuous security (DevSecOps) – Security practices, processes and technologies for applications, databases, pipelines and infrastructures are integrated into the culture and pipelines to assure security.
    Continuous delivery – Release artifacts are validated, prepared for deployment to production and may be automatically deployed to production once acceptable deployment measures are achieved.

    A Comprehensive Engineering Blueprint

    While the above nine practice categories are the core of every successful DevOps solution, 14 additional practice categories are essential parts of a comprehensive engineering blueprint for accomplishing high-performance DevOps.

    DevOps infrastructure practices are used continuously by the nine core practice categories as follows:

    Version management systems – Every stage in the end-to-end DevOps value stream depends on keeping track of versions of software source code, configuration data and executable images using source code management systems, artifact repositories and configuration management systems.

    Infrastructure-as-a-Service – Virtualized cloud and dedicated resources must be served up ephemerally to meet variable workload and storage demands, which requires mastering engineering practices for orchestrating dedicated and cloud, hybrid cloud and multicloud services, and deployment of applications to the infrastructure.

    DevOps pipeline practices, which constitute a layer just above the nine core practices, include two additional categories that make use of the nine core DevOps practices.

    Application release automation – Directing and monitoring software changes through the CI/CD pipeline requires process orchestration and data analysis working at a level above the CI/CD pipeline itself.

    Value stream management – A deployed service, composed of multiple application microservices and data services, requires orchestration and analysis of a federated set of services, end-to-end, across the value stream from planning through to operations.

    Management Practices

    DevOps management practices include four categories that make use of the DevOps Pipeline practices, as follows:

    Service catalogs – Organized collections of software stacks, tools and pre-configured pipelines, made available through portals, enable DevOps-as-a-Service operations and simplify access, configuration, control and evolution of DevOps resources needed by enterprise application teams.

    Disaster prevention and recovery – Proactive disaster monitoring coupled with high-availability configurations and disaster mitigation practices, such as automated processes, work together to minimize the blast radius of disastrous events, and orchestrate rapid restoration of essential DevOps services, when needed.

    Site reliability engineering (SRE) – A combination of practices which include service level objectives (SLOs), error budgets, monitoring, work-sharing policies, toil reduction, proactive anti-fragility practices and blameless post-mortems help to “shift left” production wisdom from operations into development, thus improving the efficiency, readiness, security and resiliency of applications and infrastructures in production.

    Governance – The scale and scope of enterprise DevOps solutions are controlled using governance policies-as-code implemented through service catalogs, management dashboards and error budget policies.

    Project Practices

    DevOps project practices include four categories that guide enterprise adoption and evolution of DevOps, as follows:

    Adoption – Enterprise-wide DevOps adoption strategies that engineer and build DevOps capabilities in logical phases. This begins with a small set of representative applications that serve as a model for other applications; this method has been shown to be more successful than “boil the ocean” strategies that attempt to transform all applications at the same time.

    Model applications – Applications – that can serve as a model for greater enterprise adoption – are selected and monitored according to criterion that spotlight the value of DevOps for other applications.

    Return on investment (ROI) – DevOps investments must compete with other IT projects for resources. An ROI model that demonstrates measurable cost savings is an approach that has proved to achieve greater numbers of DevOps project approvals compared to other methods.
    Continuous improvement – Competitive forces and industry trends require that DevOps solutions continue to evolve to keep up with enterprise requirements.

    The Human Element

    Human development practices include two categories, as follows:

    DevOps engineering – The development of DevOps engineer roles, skills and the structure of DevOps engineering within an organization is critical to the success of designing, building, maintaining and evolving DevOps solutions.

    Continuous learning – A DevOps continuous learning strategy is critical for the development and transfer of DevOps knowledge needed to develop and continuously improve DevOps for the enterprise.

    In summary, the importance of practice categories – and specific practices within each of the categories – will evolve as each application does, alongside the changing needs of the enterprise. DevOps is complex, but a comprehensive engineering strategy that considers the 24 DevOps practice categories described in this article can help enterprises achieve the high-performance DevOps solutions they need to justify the investment and to meet business goals.

  • 3 Common App Modernization Pitfalls to Avoid

    3 Common App Modernization Pitfalls to Avoid

    Application modernization has been a consistent theme for organizations year after year. Businesses today still rely on their legacy applications for the majority of their revenue generation, from mainframes to Java EE. New digital transformation initiatives and mandates have pushed application modernization back to the forefront, as a new generation of automation platforms and best practices are helping these organizations avoid the pitfalls of the past and accelerate their journey to cloud-native architectures.

    Here are the three most common application modernization pitfalls and how to avoid them.

    Lack of a Clear Organizational Destination

    Application modernization journeys need a north star – a destination point that development, infrastructure and DevOps teams can rally around. In order to establish a clear, unified direction, everyone must share a precise understanding of that destination. That journey starts by defining a set of business goals you are trying to accomplish (such as cost savings, business agility, resiliency and/or competitive advantage) that can guide and prioritize the technical decisions.

    Next, you need to decide which application modernization technologies, platforms, and infrastructure will help you deliver on those business goals (e.g., Red Hat OpenShift, Kubernetes, serverless, hybrid, private or public cloud, etc.). Setting common architectural standards will help avoid disparate future stacks and ensure a common shared platform for microservices architectures. For example, if an application starts with Java WebSphere, should the goal be to move to a modern container framework like Spring Boot, or a specific microservices Java framework like Quarkus? To avoid analysis paralysis and internal drag, set tangible objectives and targets that prescribe how teams are expected to prioritize, select and modernize these applications and how their progress should be measured, communicated and shared.

    Taking Shortcuts – Don’t Fall For the Old ‘Lift and Shift’

    The more one modernizes, the more value one gets from the cloud – but not all modernization efforts are created equal. For example, ‘lift and shift,’ putting a monolith into a container and application refactoring into microservices are all called “modernization.” But only refactoring allows enterprises to realize the true benefits of the cloud, including elasticity, regained engineering velocity and cost savings. Don’t fall into the quick-and-easy modernization temptation with lift and shift or brute force containerization. You will still have a monolith, in the end; you’ll simply end up paying more and trading one set of monolithic problems for another.

    Not Bringing Everyone Along – No Developer Left Behind

    Modernization is never just about technology. Modernization requires that your team’s processes (for example, agile practices), structure (vertical domain teams as opposed to horizontal teams – UI team, for instance) and skills must all adapt. The organizational culture needs to grow to adopt DevOps and cloud-native best practices to take advantage of the deep development, build, deployment and operational benefits that are available. Take the time to train and educate so everyone is brought along. Modernizing legacy applications can introduce a whole new set of technologists, responsible for those applications and the supporting infrastructure, into the cloud-native world. This is a win-win for everyone.

    In other words, the organization needs to have crystal clear answers to the where (destination), why (path) and how (team) to ensure success and avoid the three most common pitfalls of application modernization.

  • Languages and DevOps: Embedded

    Languages and DevOps: Embedded

    Every once in a while, it is a good idea to take stock of the environment and the market around us to determine if what we are doing is what we should be doing. I’m a developer, not a prognosticator, so these observations are simply based on my experience in the development world, along with a frank assessment of what’s changing, to try and offer some advice to my fellow developers. This week I’ll talk about embedded, but we’ll hit a bunch of application targets over the coming weeks.

    Embedded development is, generally speaking, more low-level than most other modern programming practices. Considering the goal of embedded is to interface with the hardware, this is no surprise.

    C/C++

    And that brings us to the king(s) of the embedded hill. C, and to a lesser extent, C++ rule this roost. You will see a lot of talk about Python, and we’ll get to that in a minute, but for historical and performance reasons, C is still your best bet. If your DevOps tools support your Make variant – Ant, Maven, or whatever – it can be plugged directly into the DevOps process. Testing will still take extra steps. Since deployment is most often handled directly, that does not generally need extra steps, but, in some cases, it might. A lot of general guides claim C/C++ is “easy” to learn. I disagree with this assessment; newer languages tend to be more intuitive while retaining some of C’s better features, but it is, at this time, necessary to embedded development.

    There’s a Reason (or Many) for That …

    There are a lot of conflicting factors going into where embedded is headed in the future, so we’ll try to address them here.

    Historically speaking, C/C++ have had better performance and smaller footprints than interpreted (even bytecode) languages. They are easier to code in than Assembly and supported by a ton of libraries to get work done; they struck the right balance for embedded. The weight of these developers and the legacy source code they created are part of the reason C/C++ are still on top.

    But three things have caused a shift that will open up room for other languages to make headway into this space. Embedded devices are getting faster, and storage is getting both smaller and cheaper. That means that a less-efficient means is more viable. Arduino and other hobbyist embedded platforms have also come of age. It is now viable to pick up an embedded board and start prototyping over the weekend. These embedded environments are well-supported by newer languages, and, historically speaking, today’s hobby builders are tomorrow’s startups. This is pulling other languages into the mix far more frequently, particularly our next entry.

    Python

    Python is popular in almost every category of programming. It has been around long enough to lose the mantle of “language du jour,” and should be considered based on its strengths and weaknesses. Given that the newer wave of prototype systems is well-supported by Python, it is easy to maintain, and has a broad selection of libraries (often, ironically, written in C), it has gained clout in the embedded space. It doesn’t perform as well as C/C++, but modern embedded devices are of a class that this may not be as important (early embedded devices were high-performance tools, while IoT often has devices that don’t require anything like near-real-time), and hardware improvements make this less of a concern. Python’s popularity makes it relatively easy to work in to your DevOps toolchain, with targeting of embedded devices making deploy and test require extra steps … but that’s true for all of these languages, as it is a targeting feature, not a language issue.

    Assembler/Assembly

    In the end, we need machine code. Even bytecode (or p-code, if you prefer) interpreted languages execute their instructions in hardware, eventually. Because performance and space have long been issues in embedded development, assembly language has been a preferred tool for pieces of the development pie. While few developers use assembly language for everything, most embedded developers use it, at some point, to modify libraries or create targeted subroutines. Certainly not a “popular” language among developers, most of us buckle down and just do what we need to. Like C, assembly is normally built using a Make variant. These are easily integrated into DevOps toolchains to get reporting and results up to those dashboards that run a company’s agile efforts.

    Others

    There are a wealth of other languages in use on embedded platforms, most notably C#, but also newer languages like Rust. The environment is changing, and as the ramifications of terabyte SD cards and multi-core embedded boards start to sink in, they are worth watching. We are not that far off from having a desktop-quality computer in a board only a little larger than a credit card. That changes a lot of things. More room for data storage and processing, and the need for tools that are easier to use in these contexts.

    Libraries

    All languages use libraries to make development easier. We are past the days when coders wrote all of the functionality of their project. The tools that bring libraries into a project need to be incorporated into the DevOps toolchain. For C/C++, the mechanism is largely manual, and can be automated with scripts or with the build tool. For newer languages, tools like NPM will need to be hooked into the toolchain. The ease of library dependency management is one of the appealing aspects of newer languages, and making it easier for a developer to hook in libraries also makes it easier for DevOps tools to make use of library management functionality.

    You. Rock.

    Whatever your language of choice, in the end it will likely be your build tool that links it to DevOps. Maybe we should do a series on those, too; comparing when to use Gradle versus Ant, for example. Until then, keep rocking it. All development depends upon embedded in the end, so you’re holding up the foundation.

  • The Digital Transformation of Decision-Making

    The Digital Transformation of Decision-Making

    The ability to make effective business decisions has historically relied on factors such as experience, qualifications, talent and, in many cases, luck. Strategic and tactical choices are also informed by ‘borrowing’ the insight and knowledge of others. In many situations, ‘follow the leader’ can be an effective approach, and one need only study any sector of industry or commerce to see organizations everywhere mimicking each other, with competitive advantage based on relatively small acts of incremental innovation.

    Today’s ambitious businesses have many more options at their disposal to inform decision-making than previous generations. The extent, for instance, to which businesses can now collect, store and analyze data has been transformed by improvements in technology. The result is many businesses are now sitting on enough data for advanced analytics to inform smarter, more efficient decisions.

    The Role of Advanced Analytics

    Stepping back for a moment, advanced analytics is a set of tools and techniques that use existing data to develop insights to inform business decision-making – in particular, improving the accuracy of predicting future events based on certain actions. In practical terms, it is particularly useful when organizations are planning to implement a new business activity, where it is employed ahead of legacy processes that simply can’t deliver the same levels of insight.

    Online shopping is a prime example. When a consumer clicks on various items on a website, they will often be given other suggestions – sites often will present this information with the words ‘Frequently bought together,’ for example. This is based on what has already been viewed and uses advanced analytics to predict what else the shopper might be interested in. It’s just one of many ways retailers now rely on advanced analytics to intelligently cross-sell, and as many people will know, it’s very effective.

    In a wider sense, advanced analytics can be used in a huge variety of scenarios, irrespective of industry, market or type of customer. In doing so, businesses stand to reap a number of potential benefits, including:

    • Smarter decision making. By generating data based on strong logic and reliable information, advanced analytics can draw on both past and present data to make more reliable, more confident choices about what to do in the future.
    • Focused knowledge. By understanding the factors that improve business outcomes alongside those that are negligible, leaders can focus their attention and investment on what is more likely to deliver the desired impact.
    • Risk management. In an era of heightened risk, and the need for greater risk management, advanced analytics can identify risk based on available data, helping to inform business models that minimize risk, or help businesses make decisions based on insight rather than hope. This can also help improve governance and mitigate regulatory and compliance risk.
    • Strategy automation. Because advanced analytics takes care of analyzing data, teams have greater flexibility and time to focus on wider strategic considerations, which can help make the business better.

    Advanced Analytics in Action

    Advanced analytics employs various techniques across an ever-increasing range of use cases. For example, data mining is applied to ensure organizations make use of the most relevant data, extracting what is useful from the raw data. This data can then be subjected to further analysis for a multitude of objectives, such as understanding customers better or building more effective strategies.

    In addition, ‘big data’ analytics can be applied to both structured and unstructured data, and can be particularly helpful when the data generated isn’t in a proper format, or for organizations with massive data volumes to process.

    Then there’s predictive analysis, where machine learning algorithms are applied to current and past data to identify relationships and patterns. The algorithms then build a prediction model that can inform decision making. In general, the more data that is available, the more accurate predictions will be. It can’t, however, deliver 100% accuracy, and random, unpredictable factors such as a mass healthcare crisis illustrate the obvious limitations on predictive analysis.

    Taking this a step further, prescriptive analysis helps determine the best way to implement a business decision. Even when a decision is close to being made, there can be more than one possible implementation path to follow. Prescriptive analysis uses current and past data to suggest possible outcomes before actions are set in stone, recommending the best implementation options.

    Advanced analytics is a rapidly growing niche of the technology industry, regularly described by industry experts and commentators as among the most important, transformational IT trends. Given the pivotal role of decision-making across every organization, this perhaps shouldn’t come as a surprise, and for businesses everywhere, it’s now only a question of when they apply it to improve their fortunes, not if.

  • Grafana Labs Adds On-Premises Observability Platform

    Grafana Labs Adds On-Premises Observability Platform

    Grafana Labs announced today the availability of a unified Grafana Enterprise Stack that enables IT teams to deploy a curated observability platform, based on open source tools such as Prometheus, in their own on-premises IT environment.

    Tom Wilkie, vice president of product for Grafana Labs, said Grafana Enterprise Stack, in addition to providing a version of Prometheus optimized for distributed computing environments, includes curated instances of open source Grafana dashboards, Loki log aggregation tools and the Cortex query engine.

    Grafana Enterprise Stack also includes plugins for visualizing data from other commercial tools in Grafana, such as Splunk, New Relic, MongoDB, ServiceNow, Oracle and Snowflake. In addition, it includes reporting tools and enhanced authentication and security controls.

    Grafana Labs

    Developed under the auspices of the Cloud Native Computing Foundation (CNCF), Prometheus has gained traction as a monitoring platform used primarily in Kubernetes environments. However, Wilkie said it’s only a matter of time before IT organizations expand their use of Prometheus to include a wider range of platforms. The Grafana Enterprise Stack facilitates that transition by making it easier for organizations to deploy a complete observability platform based on open source software, said Wilkie.

    While open source software has a key role to play in enabling observability, most IT organizations today rely on a range of proprietary tools to monitor IT environments. Many of those tools will be replaced, in the months ahead, by observability platforms that provide more context than individual monitoring tools currently do. However, it’s not clear yet if IT teams will prefer open source or proprietary platforms that, in most cases, incorporate open source components.

    As IT environments become more complex, the tools most organizations rely on today don’t surface enough actionable insights. Most IT teams today still waste a fair amount of time navigating amongst multiple tools in an effort to discover the root cause of a problem. It’s not uncommon for IT teams to take days to discover the source of a problem that is then fixed in a few minutes. The expectation is that observability platforms will sharply reduce the amount of time and effort required to investigate, for example, a performance issue.

    Of course, to achieve that goal, applications themselves will need to become more instrumented. A mix of open source and proprietary agent software will make it easier to collect traces, logs and other metrics at a reduced cost. In many cases, IT teams have been reluctant to fully instrument an IT environment, because of both the cost of proprietary agent software and the inherent challenges associated with deploying and managing it across an extended enterprise. However, as the deployment process for agent software becomes more automated, organizations are finding it easier to instrument their applications.

    It may be awhile before IT environments are fully instrumented. However, as it becomes easier to collect traces, logs and metrics, the level of observability that can be achieved across an enterprise will undoubtedly increase. The challenge and the opportunity then becomes employing the greater visibility provided by these platforms to manage the sprawl of applications and platforms across IT more proactively.

  • DevSecOps Implementation: Patch Management

    DevSecOps Implementation: Patch Management

    Patch management has been a thorn in IT’s side for decades. While it is undeniably necessary for security and stability purposes, the staggering array of patches that are available make managing them effectively nearly a full-time job – or a job better left to users. Few IT professionals want to be the “patch person,” and yet, nearly all IT professionals want their systems protected by the latest patches. No one wants to find out an intrusion was easily preventable via patching. It goes without saying that the SolarWinds attack has left quite a few IT departments scrambling for an alternate solution.

    The world of patch management has gone through several evolutions that we won’t cover in much detail here. Instead, we’ll focus on what’s available, what you should look for and where the market is headed. Suffice it to say that patch management as a practice has grown from an individual applying patches to servers and server apps while employees were encouraged to patch their equipment, to complex systems that manage the patching themselves, with IT oversight. Currently, patch management is often bundled with endpoint management, simply because many of the patches are going to end points (user machines and devices).

    Out of this growth have come several market differentiators that you should be aware of when looking into a way to manage the avalanche of updates the average organization is seeing.

    • Centralized or de-centralized? Increasingly, patch management is run on servers/cloud and pushes changes out to those machines/devices that need them, or agents on each machine manage updating themselves. Which is best for a given organization really varies. If a lot of machines are running at the edge of their capability, an agent running updates in the background might be too much to ask. In that scenario, a server-managed install environment is probably better. Conversely, if a centralized location to run updates from is considered an architectural weakness (because it requires access to the update pushing engine), agents might be the better plan.
    • OS support. We’re way past the days when single-OS solutions were considered useful. List the operating systems used in your environment, and choose a product accordingly. Most products on the market support Windows/Mac/Linux, but there are variances. First off, make certain your chosen product supports all three. Then ask, “Which flavors of Linux?” Make certain the flavors running in-house are supported by the software.
    • App support. There are a lot of applications out there. Which are supported by the tool? How much leeway does the customer have for adding applications by pointing the tool at update locations? Does it support FOSS updates? There are a lot of those too, and which ones matter. Make certain the tools the organization uses are supported or can be added.
    • Many tools will scan machines for update issues. Understand what your chosen tools scan for, and how the scans impact performance on the target machines.
    • Updates are not 100% perfect. There will inevitably be problems. Make certain the tool can roll back changes. Updating 50 core infrastructure machines, only to find, hours or days later, that the update breaks something demands roll-back capability.
    • Inter-relationships. Updating a piece of software can require that another piece of software or a driver be updated, which can demand that UEFI/BIOS be updated. Or the CMOS on a RAID controller. Understand how cascading updates are handled by the tool, and if the cascading updates can be packaged, so that a failure automatically rolls back the updates applied thus far.
    • Approvals. Speaking of driver and BIOS updates, what tools are available to allow a human to approve changes? No one wants willy-nilly updates to the drivers on core systems – or even to drivers on user systems.
    • Security of the tool itself. We’re in perilous times where IT security is concerned, and frankly, an update command-and-control server is an attacker’s dream machine – take it over, put your back-door update into the mix, and tell it to update every machine it manages. It cannot be understated: an update management tool must be as protected as possible – both by the customer locking down the installation and by the developer making sure the code is solid.

    Once the system is chosen and worked into your DevSecOps environment, updates will be more responsive, which leaves one more thing to consider – does it feed into your agile tools? If the update management tool notifies a project management tool like JIRA when it updates a server, then if code on the server breaks, the developers aren’t in the dark about changes, and will have breadcrumbs to help them track down the source of the problem.

    Imagine if manual updates became a rarity instead of the norm. There is a cost associated with this process, but most of the products on the market can be had for a reasonable price – if you waste a couple hours a month (or for some products, a year) on updating, they’re paying for themselves. Of course, your time is a sunk cost, and the cost of these is an operational cost, so there is an evaluation your organization will have to undertake. Given how busy everyone in DevOps and DevSecOps is, it is likely worth the investment for all but over-staffed organizations. What? There are a few!

    Find what suits your organization, and put it to use. Faster updates, less time investment. The idea is sound, you just need to figure out what suits your needs. And keep rocking it. You are managing updates already, this is just a way to make your life easier. And we could all use a bit of easier in a busy, complex IT environment.

  • Open Source Advocates Want Share of Wealth

    Open Source Advocates Want Share of Wealth

    The egalitarian spirit of the open source community might be starting to fray as individual developers begin to question who is really benefiting most from their efforts.

    A survey of 4,440 developers who actively participated in open source projects in 2020 finds more than half (54%) of respondents said they feel that individuals should be paid for their contributions to open source projects. The report, conducted by DigitalOcean, a cloud service provider, finds only 14% of respondents are currently paid for their contributions.

    Developer opinions vary, depending on age group, about whether individuals should be paid for their contributions to open source, with those between the ages of 18-25 being more likely to be in favor (60%), while only 34% of those aged 55 or above agreed.

    “A lot more people want to be compensated,” said Raman Sharma, vice president of product marketing for DigitalOcean. “That may mean there will be a need for more sponsorships by vendors.”

    The DigitalOcean report arrives just after Elastic announced it is changing the licensing terms under which it makes Elasticsearch available. Elastic is shifting away from an Apache license to a dual license strategy that includes a server-side public license, or SSPL, along with a proprietary Elastic license. That shift was made to help ensure more of the revenue generated by Elasticsearch winds up in the hands of Elastic rather than cloud service providers, such as Amazon Web Services (AWS), that routinely launch paid services based on open source code. AWS responded to the Elastic move by announcing plans to fork Elasticsearch code that it will now curate.

    Nearly half of respondents (47%) to the DigitalOcean survey said they think technology companies should fund the payment of open source contributions, while a quarter (25%) believe project owners or individuals should pay.

    Discontent among startup vendors has been building for months. Startups typically are backed by venture capitalists, and make their software available as open source code in an effort to build a customer base on the strength of their support services. Cloud service providers have discovered they can take the code these vendors create to launch their own service. Those cloud service providers may contribute code to the project, but the startup vendor doesn’t typically enjoy any of the economic benefits.

    In most cases, the open source code used by the cloud service provider comes from a small cadre of developers that work for the startup company. Most of those startup companies are hoping to build a community around their code that includes individual contributors, external to the company, to help defray those costs. In fact, that business model has contributed to open source projects’ role as one of the primary drivers of IT innovation in recent years. If more independent developers demand to be paid for those efforts, the innovation rate of open source projects is likely to slow. The labor required to support those projects simply won’t be as freely available.

    In contrast, massive open source projects, such as Linux, have already reached a level of sustainability that other projects can only dream of. The Linux community is still ruled by Linus Torvalds, who some say is something of a benevolent dictator. Many of the contributors to Linux, however, are full-time employees working for vendors like IBM and SAP, which have a vested interest in the underlying open source platform.

    In contrast to the Linux community’s approach, the Eclipse Foundation, The Open Infrastructure Foundation (formerly the OpenStack Foundation), the Apache Software Foundation and the various arms of the Linux Foundation have stepped in to provide some level of independent governance for a wide range of open source initiatives, with mixed success. The Cloud Native Computing Foundation (CNCF), for example, as an arm of the Linux Foundation, has played a major role in spurring Kubernetes adoption. However, Google recently spurned the CNCF when it decided to create its own steering committee to oversee development of the Istio service mesh.

    Despite all the internecine rivalries and the potential for more forking of open source projects, overall, these projects are still viewed favorably. A survey of 3,400 developers, conducted by O’Reilly Media on behalf of IBM, finds 94% of respondents said they rate open source software (OSS) as equal to or better than proprietary software. Nearly two-thirds of respondents (65%) also agreed that contributions to open source projects impress potential employers and result in better professional opportunities.

    Of course, one of the primary reasons so many developers prefer open source is that there is no purchasing process for them to navigate.

    “Developers can move faster,” said Todd Moore, vice president for open source at IBM. “Programmers can move at the speed of light.”

    Regardless of motivation, it’s clear open source software isn’t going away. However, how it is built and exactly what is permissible under an open source license may soon vary much more widely from one project to the next.