devopsy.kinsta.cloud

Tag: automation

No, You Don’t Want to be a Prompt Engineer

Prompt engineering became a thing in 2023 and will stop being a thing in 2024.

We’re already seeing signs of its decline, but let me set the stage for those only vaguely familiar.

A prompt engineer is someone who knows how to get an AI (Normally a chatbot, but honestly, any of the raft of generative AI solutions out there for art, literature, code generation, etc.,) to do what is necessary. While at first, no one was really thinking about this, it quickly became clear that getting a quality result required some massaging in the same sense that getting a quality result from a search engine requires a specific ordering of keywords and limiting of possible results. So, as time went on, those who were better at it started being referred to as ‘prompt engineers.’ I am not certain of the genesis of the phrase, but it honestly doesn’t matter for this discussion. The name became a title, and it got popular; there were some crazy claims about exorbitant salaries for a wordsmith that made it attractive to many.

AI Tools

For some of my work, we use subcontractors to do technical research. Normally, we use technical people and have them turn out results we know will be topically close, even if they didn’t understand the specific technology when they started researching for us. On occasion, we have tried people with less technical backgrounds, and the results are definitively lower quality. With the advent of decent AI tools, we have done some research on using them for this work and have discovered that they allowed less technical people to generate roughly the same quality of results as the more skilled folks.

What this approach doesn’t do is raise either of these researcher types to the level of being able to do something with the results or understand the market well enough to determine results. And we see this across the board. In our development/scripting work, we use generative AI to help, but it doesn’t write finished code complete with the context of our tools/environment. We do that afterward, along with validating the code it does produce. My other business is in publishing and uses a lot of art, so we have contacts with a fair number of artists. Our art director uses generative AI for inspiration but not creation. She asked the other artists we use to follow that example at least until the courts work out IP law about generative AI, and across the board they came to the same conclusion. Some artists are using it for personal/learning purposes, but most AI-generated art is not being generated by professional artists. That has the usual implications for professional artists, but they’re used to being under-appreciated and being underpaid for their work … whether it’s an AI depressing costs versus an amateur doesn’t matter much in context.

Learn it and Use It

Prompt engineering will steadily become less necessary due to simple forces. Just like people learned the ins and outs of search engine prompts, they will learn to do so with AI in their space. As the AI gets more training/specialization, it will get easier to use, and at the same time, it will get more accurate. So, my advice is the same as with all automation tools. Learn it, see how it fits into your organization and use it to your advantage. But it’s not a career; it’s an automation tool. As time goes on, it will be able to do more and more, and maybe one day we’ll be able to say, “Hey Google, write me a CRM that works with our billing system.” But for the foreseeable future, focus on coding and use it to do some of the grunt work.

The only caveat I have also offered before. Go out of your way to hire and train good entry-level people. The weakness of AI doing a lot of the grunt work is that the “learning period” is shorter, so you do need to consciously say, “What is necessary to debug and/or manage our systems?” and make certain your entry-level people get that – because the day-to-day that they have traditionally trained in is far smaller.

And use it to create and maintain even better systems in the new year! I am stoked about the possibilities it opens up, and you should be, too. If I don’t have to write and debug that for loop, just generate, review and test, I’m thrilled. We should all be happy to use that time on more productive things than entry-level code.

January 17, 2024
Broadcom Survey Surfaces Raft of IT Automation Challenges

A survey of 501 IT leaders published today by Broadcom suggested that islands of automation that operate independently of each other are major contributing factors to organizations failing to meet service level agreements (SLAs).

The survey found that 98% of respondents encountered at least one automation issue, with 61% noting that SLA issues are a monthly occurrence. More than a third (35%) said these issues occur weekly. A full 88% indicated SLA breaches directly impacted customers in a significant way.

The survey also found nearly three-quarters (74%) lacked visibility across their data center and cloud environments, with an equal number lacking end-to-end visibility of their automated business processes. Only 28% possessed tools that could predict automation issues, and more than two-thirds (68%) were unable to model or predict whether a new or updated automation job would actually meet SLAs. A full 69% said they are unable to properly size automation workload resources, with 61% having no way to simulate automation workloads before implementation, the survey found.

Only slightly more than half (55%) of respondents, however, could provide SLAs and key metrics in a format that is appropriate for business stakeholders.

Aline Gerew, head of automation for the Agile Operations Division at Broadcom, said that while it’s routine for IT teams to leverage a range of automation tools and platforms to manage IT, it’s clear they are challenged to manage those platforms in a comprehensive fashion. As a result, whenever there is an issue with one tool or platform, it creates a bottleneck that impacts customer experience in a way that ultimately adversely impacts revenue, she noted.

In fact, many IT teams are now starting to identify automation architects within their IT teams to enable DevOps teams in a more holistic manner, added Gerew.

Overall, the survey found that 80% of respondents are using three or more automation platforms, with just over a quarter (27%) using six or more.

Nearly three quarters (74%) are also relying on three or more observability solutions, with just under a quarter (24%) using six or more. Just under three-quarters of respondents noted they are consequently subjected to alert storms that ultimately serve to make it more challenging to resolve issues. More than half (56%) of respondents are blind to the downstream effects of an initial alert and, as a result, do not inform response teams of which systems, jobs, applications and business processes are affected.

In some ways, IT automation has become too much of a good thing. Every provider of an IT platform makes automation tools available, but as they are invoked, each one creates yet another island that IT teams are expected to manage in isolation. However, all it takes for an SLA not to be met is for an issue to arise that IT teams can’t easily determine the root cause of because one or more automation platforms obscure the underlying issue in the absence of any standardization, noted Gerew.

Given the complexity of modern IT environments, automation is now a requirement. The issue, as always, is finding a way to apply it all in the most transparent way possible.

January 9, 2024
5 Time-Saving and Problem-Solving Technologies for DevOps Teams

DevOps has transformed almost every aspect of software development, taking us from the dark ages of big-bang deployments every 12 months to the reality of releasing software multiple times per day. However, DevOps professionals have to overcome several hurdles to successfully implement and sustain the DevOps culture within the organization. Let’s review some of these challenges—and five technologies that can help.

Cultural and Organizational Differences

One persistent challenge for DevOps teams in a mature setting is navigating cultural and organizational differences. These differences often manifest in varied working styles, communication methods and priority management across teams. For instance, development teams might prioritize rapid innovation, while operations teams focus on stability and security, leading to conflicts. Managing these diverging priorities requires constant negotiation and alignment, which can be time-consuming and complex.

Furthermore, as organizations grow and incorporate diverse teams, including remote and international members, ensuring cohesive communication and understanding becomes even more critical. This diversity, while beneficial, brings challenges in terms of aligning on common goals and working practices, often leading to delays and misunderstandings that impede efficient workflow.

Complexity of Automation

In mature DevOps environments, the complexity of automation can become a significant day-to-day challenge. As the infrastructure and application landscape grows, the automation scripts and tools that were once efficient can become unwieldy and difficult to manage. Keeping these automation processes up-to-date with the constantly evolving technology stack is a never-ending task, requiring significant time and resources.

Additionally, the integration of new tools or updates to existing ones often introduces complexity, leading to potential errors and downtime. This complexity not only slows down the deployment process but also requires DevOps teams to invest time continuously in learning and adapting to new tools and methodologies. Ensuring that automation is both robust and flexible enough to handle a dynamic environment is a delicate balance that DevOps teams must constantly maintain.

Cross-Team Collaboration

Effective cross-team collaboration is vital for DevOps teams but can be challenging in day-to-day operations. The interdependence between various teams–such as development, operations, QA and security–requires a high level of coordination, often hindered by differing goals, tools and terminologies. Communication barriers can lead to misalignment on project objectives, causing delays and inefficiencies.

Moreover, in a fast-paced DevOps environment, keeping everyone on the same page regarding updates, changes and issues requires meticulous attention. This is compounded when teams are distributed geographically, as time zone differences and cultural nuances add another layer of complexity. Regular synchronization meetings, clear communication channels and shared tools are necessary, but managing these effectively without causing meeting fatigue or information overload is a continuous challenge.

Consistency and Standardization

Maintaining consistency and standardization across various projects and teams is a significant challenge for mature DevOps teams. As the number of services and microservices increases, ensuring that each adheres to the same set of standards for coding, security and deployment becomes increasingly difficult. Inconsistencies can lead to significant problems in integration, testing, and deployment, slowing down the entire delivery process.

Furthermore, as different teams may prefer different tools and practices, creating a standardized environment that accommodates everyone’s needs without compromising on efficiency or quality is a complex task. Regular audits, effective governance and a balance between flexibility and standardization are essential to manage this challenge, but they require constant effort and vigilance.

5 Lesser-Known Technologies That Can Help

While there is no one technology that can completely alleviate the above challenges, there are several tools that can help. Let’s review five technologies that can make the life of a DevOps team just a little bit easier.

Chaos Engineering Platforms

If you’re part of a DevOps team, you’re probably all too familiar with the panic that ensues when a system fails unexpectedly. Chaos engineering platforms aim to save you from such unforeseen surprises by embracing the philosophy of ‘breaking things on purpose.’

Chaos engineering involves intentionally introducing failures into systems to understand how they respond and to identify potential weaknesses. By doing so, DevOps teams can proactively find and fix issues before they manifest in production, thereby saving significant time and preventing potential downtime.

Platforms like Gremlin and Chaos Monkey offer controlled environments for running these chaos experiments, providing teams with valuable insights and recommendations. In the long term, this can not only save time but also instill a sense of confidence and resilience in your systems.

Application Dependency Mapping

Application dependency mapping (ADM) is a process that helps DevOps teams visualize and understand their applications and their dependencies. It is a crucial technology that aids in quickly diagnosing and rectifying application performance issues, thereby saving time.

ADM provides a holistic view of the application architecture, showing how different components interact with each other. This comprehensive visibility is invaluable during troubleshooting as teams can easily pinpoint problematic areas and rectify them without having to sift through mountains of code.

Furthermore, ADM is crucial when planning for application changes or upgrades. Understanding how changes in one component may affect others helps in preventing potential disruptions, leading to smoother and faster software delivery.

Policy-as-Code Solutions

Policy-as-code is a practice where policy definitions are codified and managed as code files within version control systems. This approach offers a host of benefits that can save time for DevOps teams.

Policy-as-code solutions enable automatic enforcement of policies, eliminating the need for manual checks. This significantly reduces the time taken to ensure compliance and allows teams to focus more on developing and delivering quality software.

Moreover, since policies are managed just like any other code, they can be versioned, tested and reviewed before being deployed. This allows for a more streamlined and error-free policy management process, saving valuable time and effort. Tools like Open Policy Agent and HashiCorp Sentinel are commonly used to implement policy-as-code in DevOps environments.

Service Mesh Technologies

Next on the list is service mesh, a dedicated infrastructure layer for managing service-to-service communication in a microservices architecture. Service mesh technologies offer a range of features that can save time for DevOps teams.

One of the primary benefits of a service mesh is its ability to offload common communication concerns from individual services to the mesh itself. This includes functionalities like load balancing, service discovery, and traffic management. By doing so, it saves DevOps teams the time and effort required to implement these functionalities within each service.

Additionally, service mesh technologies provide detailed observability of the service communication, making it easier for teams to monitor, troubleshoot and optimize their applications. Istio and Linkerd are two common open source options for implementing service mesh.

Codeless Test Automation Tools

Codeless test automation tools represent a significant shift in the way DevOps teams approach software testing, offering a more efficient and less time-consuming method. These tools allow the creation of automated tests without the need to write extensive code, making test automation accessible to team members who may not have deep coding expertise.

Using a graphical user interface, codeless test automation tools let users design tests by simply dragging and dropping elements into a workflow. This approach not only accelerates the test creation process but also simplifies the maintenance and updating of tests as applications evolve. Additionally, these tools often integrate seamlessly with various CI/CD pipelines, ensuring that automated tests can be easily incorporated into existing DevOps workflows.

With the speed at which technology is advancing, it’s crucial for DevOps teams to stay updated on the latest tools and technologies that can help them optimize their processes. The technologies listed in this article can significantly save time for DevOps team and make complex DevOps workflows a little smoother.

January 8, 2024
All Your IT Team Wants This Holiday Season is a Break!

The holiday season is all about giving. As organizations increasingly look to IT as they move toward new digital tools and processes, now is the perfect time to give back to IT teams tirelessly working to keep the modern enterprise online. Whether your system performance has been naughty or nice this year, there’s no denying that tech professionals have earned our appreciation, respect—and the tools to set them up for success in 2024.

For IT teams limited in both time and resources, simply maintaining systems can feel as impossible as squeezing themselves down a chimney or delivering gifts to millions of homes in a single night. On top of that, instead of being greeted with milk and cookies, they’re inundated with endless performance issues, support requests and alerts—leaving little time left over for the important work of innovating.

They say the best gifts are the ones you can’t wrap. That holds true for IT teams, too. This year, bring your organization the gift of a simpler, speedier, more rewarding workload. If your team is dreaming of a tech-savvy future, here are some enterprise software solutions to make their lives easier that they won’t want to re-gift:

Enjoy the View With Observability

Everyone loves to cozy up at home during a winter snowstorm, but with the widespread migration to combined remote, on-premises and distributed hybrid environments, the daily monitoring journey for today’s IT teams is more akin to trekking blindly through a blizzard.

Observability tools are metaphorical snowshoes and goggles that can help them not only weather the storm but see clearly from the mountaintop. Observability is the answer to the modern enterprise’s struggle to gain full visibility into their organization’s apps, networks, databases and infrastructure—something nearly half of IT professionals lack, according to SolarWinds research. IT teams will be able to rest easier at night with visions of sugarplums, rather than outages or anomalies, dancing in their heads.

Even better, integrating artificial intelligence (AI) capabilities into observability solutions to collect and provide data on what’s not performing as expected and why will help your teams take a proactive approach to solving issues.

Lend a Helping Hand With AIOps

AI isn’t just the shiny new toy of the tech world. Organizations using AI for IT operations (AIOps) can give the gift of support to their overworked IT teams by automating some of the time-consuming and mundane tasks that stand between them and a focus on innovation.

Adding AIOps to observability can provide IT teams with maximum visibility into the state of their digital ecosystems through automated discovery and dependency mapping. Additionally, your teams can gain the ability to easily track inbound connections linked across the organization’s application stack and storage volumes with auto-instrumented views.

Today, it simply isn’t feasible for humans alone to manage modern IT environments without intelligent automation. Think of AIOps as a workshop of elves operating in the background to ensure workloads and processes are streamlined and moving as efficiently as possible. With AIOps in place to analyze data and streamline workloads and processes, IT teams are relieved of some pressure—and can focus on accelerating your digital transformation rather than just maintaining it.

Give the Gift of Time

Finally, although you can’t outright give the gift of time to your IT team, you can still arm them with tools that save time year-round. In 2024, more organizations will embrace IT service management (ITSM) to improve their IT outcomes, service delivery and employee satisfaction.

SolarWinds research found that enterprises using AI-powered ITSM tools were able to reduce system downtime by 21% and decrease time spent resolving incident and service requests by 23%. In fact, the average SolarWinds customer reported savings of 23 hours weekly due to reduced ticket volume—nearly the equivalent of adding an employee to the team.

This holiday season, the best gifts you can give your IT teams are the actionable, predictive intelligence and tools they need to optimize performance, cut down on complications and quickly resolve issues. (As a bonus, it will certainly make you more popular than the guy from The 12 Days of Christmas who thought it was a good idea to gift his partner geese, milkmaids and leaping lords.)

December 22, 2023
The Future of Jenkins in 2024

As software development practices evolve, the tools that support them must also adapt to stay relevant. Jenkins, the venerable automation server used by many DevOps teams, is at such a crossroads as it faces the changing landscape of cloud-native applications and Kubernetes adoption.

This article examines Jenkins’ current utility, particularly its features that have made it a staple in CI/CD practices, and addresses the question of whether it will maintain its relevance in 2024 and beyond.

How Jenkins is Used by DevOps Teams

CI and CD

Jenkins has become an integral part of many DevOps teams due to its CI/CD capabilities. Continuous integration (CI) is a software development practice where developers regularly merge their code changes into a central repository. Continuous delivery (CD) is the practice of delivering the software in short cycles to ensure that it can be reliably released at any time.

Jenkins supports the CI/CD approach in a number of ways. First, it allows developers to automate code integration. This means that whenever code changes are pushed to the repository, Jenkins can automatically build and test the project. Second, Jenkins allows for the continuous delivery of software by automating simple deployment processes (although it lacks full deployment orchestration capabilities).

Build Pipeline Visualization

Another key feature of Jenkins is its build pipeline visualization. This feature allows DevOps teams to visually represent the process of software delivery from code integration to deployment. The pipeline visualization provides a clear and intuitive representation of the software delivery process, making it easier for teams to understand and manage the process.

Build pipeline visualization in Jenkins is customizable, allowing teams to tailor it to their specific needs. For example, it can be customized to show the status of each stage of the delivery process, from code integration to testing to deployment. This allows teams to quickly identify any issues or bottlenecks in the process and address them.

Automated Testing

Automated testing is another area where Jenkins shines. With Jenkins, DevOps teams can automate the process of testing their code by running scripts and integrating with automated testing tools as part of the build process.

Jenkins supports a wide range of testing frameworks and tools, allowing teams to choose the ones that best fit their needs. Moreover, Jenkins can be configured to automatically run tests whenever changes are made to the code. This ensures that any issues or defects in the code are detected early and can be addressed promptly.

Workflow Scripting

Workflow scripting is another key feature of Jenkins in DevOps teams. With Jenkins, teams can script their workflows, automating the process of software delivery. This allows teams to standardize their workflows, ensuring that each delivery process is consistent and predictable.

Jenkins supports the Groovy scripting language for workflow scripting. This means that teams can use the power and flexibility of Groovy to script their workflows. Moreover, Jenkins provides a user-friendly interface for scripting workflows, making the process easy even for those who are not familiar with Groovy.

Current Usage and Adoption Statistics

As of 2023, Jenkins remains a leader in the CI/CD space. With its extensive plugin architecture and large user base, it is used by thousands of organizations worldwide. According to the 2022 DevOps Survey by the Cloud Native Computing Foundation (CNCF), Jenkins was used by over 50% of respondents for automation and CI/CD, maintaining a significant presence in the market.

According to the Jenkins website, there are over 1 million active Jenkins users and more than 200,000 active installations worldwide, with the platform handling 1 million+ jobs per day. It had about 44% of the global market share for CI/CD in 2023.

The growth trajectory of Jenkins, however, has shown signs of plateauing when compared to newer, more cloud-native-focused CI/CD tools. This observation is backed by Google Trends data, reflecting a steady but not sharply increasing interest in Jenkins over recent years.

Despite facing competition from emerging tools designed for cloud-native development, such as GitHub Actions, GitLab CI and CircleCI, Jenkins has responded by evolving its ecosystem. The creation of Jenkins X is a strategic move to cater to Kubernetes users and address some of the criticisms related to cloud-native development. However, as we’ll discuss, Jenkins X has not been readily adopted by the industry as of yet.

Limitations of Jenkins in Cloud-Native Environments

While Jenkins is a powerful tool for DevOps teams, it has serious limitations in cloud-native environments. These limitations stem from the fact that Jenkins was originally designed for traditional, monolithic applications, not for the dynamic and distributed nature of cloud-native applications.

Statefulness

One of the critical limitations of Jenkins in cloud-native environments is its inherent statefulness. Cloud-native architectures typically favor stateless applications, which are easier to scale and manage in dynamic environments like Kubernetes.

Jenkins, however, relies on maintaining state—it stores configuration, build history and user data, making it challenging to scale and manage efficiently in a cloud-native setup. This statefulness requires persistent storage, which complicates deployments in distributed systems where resiliency and horizontal scaling are key. Moreover, the risk of data loss or inconsistency increases in cloud-native environments, particularly during scaling operations or when recovering from failures.

Configuration Complexity

Jenkins is often criticized for its complex configuration requirements. Setting up Jenkins in a cloud environment involves intricate configurations that are not only time-consuming but also prone to errors. This complexity is exacerbated when integrating with modern cloud-native tools and services, requiring extensive customization and maintenance of plugins.

In addition, Jenkins’ traditional master-slave architecture was not designed for dynamic cloud environments, where resources need to be provisioned and deprovisioned rapidly. This leads to a significant operational overhead for DevOps teams.

Immutable Infrastructure

Adopting an immutable infrastructure approach is a cornerstone of cloud-native environments, promoting the idea of replacing or rebuilding servers rather than modifying them.

However, Jenkins’ traditional model does not align well with this principle. It typically requires modifications to its runtime environment for updates or changes, contrasting with the cloud-native approach of immutable deployments. This can make it challenging to maintain consistency and reliability in cloud-native systems where automation and minimal manual intervention are paramount.

Microservices Limitations

Jenkins was originally designed for monolithic architectures and doesn’t naturally fit into microservice-based applications, which are prevalent in cloud-native environments. In microservices architectures, applications are broken down into smaller, independently deployable services.

Jenkins, however, tends to treat applications as monolithic units, making it less efficient for handling multiple, independent microservices. This limitation affects its ability to provide fine-grained control over individual services’ build, test and deployment processes. Consequently, Jenkins may struggle with the high degree of automation, flexibility and scalability required in microservices environments.

What is Jenkins X?

Jenkins X is an open source project that provides pipeline automation, built-in GitOps, and preview environments to help teams collaborate and accelerate their software delivery at any scale. It is an improved version of the original project, designed specifically for Kubernetes, a popular container orchestration platform.

Jenkins X streamlines the technical aspects of building, testing, and deploying software. It automates the installation of popular tools like Helm and Draft, making it easier for developers to get started. It also provides pre-configured pipelines so teams can start delivering software right out of the box.

Jenkins X also introduces a new Jenkinsfile format, Jenkinsfile.yaml, which provides a more straightforward and human-readable way to describe pipelines. This new format, combined with the automation provided by Jenkins X, helps reduce the complexity of pipeline configuration and management in Kubernetes.

Limitations and Criticisms

Jenkins X has not been widely adopted by the software industry and has not achieved the respect and prominence of its predecessor, Jenkins. Let’s review the main limitations and criticisms of the tool.

Lack of Maturity

Jenkins X’s relatively recent entry into the market means it has not had the time to mature like its predecessor. This lack of maturity can manifest in several ways, including fewer features compared to established tools, fewer community-tested plugins, and potentially more bugs or unresolved issues. Due to the fast-paced evolution of cloud-native technologies, tools need time to adapt and stabilize; Jenkins X is still in this phase of its lifecycle.

Moreover, a mature tool typically has extensive documentation and a large community of users who can offer support and share best practices. While Jenkins X has seen growth in its user base, it still lags behind more mature CI/CD solutions in these areas. This can lead to challenges when teams are looking to solve complex problems or when they require advanced support.

Limited to Git Projects

Another criticism of Jenkins X is that it’s limited to Git projects. While Git is the most popular version control system, it’s not the only one. Some teams may use other systems like Mercurial or SVN, and these teams would be unable to use Jenkins X.

Even within the world of Git, Jenkins X has limitations. For example, it doesn’t support all Git hosting platforms. While it works with popular platforms like GitHub and Bitbucket, it doesn’t support others like GitLab. This can be a significant limitation for teams that use unsupported platforms.

Community Support

Finally, there are concerns about community support for Jenkins X. While the project is open source and theoretically has the potential for wide community involvement, some critics argue that it’s primary source of support is CloudBees, the company behind Jenkins.

These critics worry that if CloudBees were to withdraw its support, the project could falter. They also argue that the project’s focus on certain tools and platforms (like Kubernetes and Git) reflects CloudBees’ business interests more than the needs of the broader DevOps community.

Conclusion

In conclusion, Jenkins’ role in the CI/CD landscape has been pivotal, and its continued evolution with projects like Jenkins X demonstrates an effort to adapt to the needs of modern, cloud-native environments. While Jenkins X has yet to reach the widespread adoption and maturity of its predecessor, its specialized focus on Kubernetes and GitOps reflects the shifting priorities in software development.

Jenkins, despite showing signs of plateauing in growth, still possesses a strong user base and an established presence in the market, supported by a vast plugin ecosystem and a robust community. Its limitations in cloud-native contexts—such as managing statefulness, configuration complexity, and supporting immutable infrastructure—pose challenges that Jenkins X seeks to address.

As we approach 2024, the relevance of Jenkins will likely hinge on its ability to integrate more seamlessly with cloud-native technologies and the willingness of the community to evolve its ecosystem. The breadth of its adoption and the depth of its integration in existing development pipelines mean Jenkins will continue to be a significant player in the DevOps toolchain. However, its long-term relevance will depend on the project’s ability to innovate and remain adaptable in a landscape that is rapidly embracing cloud-native solutions.

December 15, 2023
Where to Start With Secure Design – Tips for Developers

Designing secure software is an increasingly hot topic, and for good reason. In recent weeks, we’ve seen a global effort from governments and cybersecurity agencies to encourage security by design through new guidance and changes to procurement rules. During the G7 in May, the QUAD nations (Australia, India, Japan and the United States) released a set of Joint Principles for Secure Software. In the U.S., there are even plans for legislation to make software makers liable for the security of their products. The need for software to be made secure at the design phase has been spotlighted further by recent cybersecurity incidents such as the hack by ransomware group CLOP that exploited flaws in MOVEit software.

But secure design is not a part of every developer’s skill set, and it’s not on the syllabus of every coding course. If you’re brilliant at writing code and keen to know more about how you make that code secure, where do you start?

Here are five tips from former DevSecOps and threat modeling engineer Fraser Scott, now IriusRisk’s VP of product:

1. Get to Know the Threats

To be able to design secure software effectively, you need to understand the threats. A great starting point is the OWASP Top 10. OWASP aims to identify the most critical security risks to web applications. Top of that list is broken access control, and insecure design itself sits at number four. Understanding the biggest risks is the first step towards mitigating them in software development.

2. Try Threat Modeling

The foundation of secure design is threat modeling, which is the process of identifying and mitigating plausible security threats. In its simplest form, this is about looking at your software design and asking Adam Shostack’s four questions:

1. What are we working on?
2. What can go wrong?
3. What are we going to do about it?
4. Did we do a good enough job?

The OWASP Top 10 gives you a sense of what the risks might be, and threat modeling is the process of identifying them in your design. See if any of those top ten risks might impact your design, consider what could go wrong and work out what you can do to mitigate it. Before you start writing any code, consider whether you’ve done a good enough job to make the software secure. The sooner you find design issues, the cheaper and easier they are to fix.

3. Connect With a Community

There are plenty of communities of people out there grappling with the same challenges that you are. Experts in security and threat modeling and developers are coming together to discuss and try to solve some of the key challenges in secure design. One such community is Threat Modeling Connect, where people use the forum to share ideas and discuss issues. Experts also host webinars and write blogs. The community also held its first-ever conference in October. There is also a community dedicated to threat modeling on Reddit, and the OWASP Slack has a #threat-modeling channel.

4. Experiment With Automation

Companies who are developing software on a large scale use tools to automate the threat modeling process. IriusRisk makes a community edition of its automation tool available for free, for life, to people who want to give threat modeling a try. As someone who is starting out in secure design, using a tool like this, which generates many of the relevant threats and countermeasures for you, is a great way to experiment with your first threat models. And we’re not the only ones; you can also use the OWASP Threat Dragon and Microsoft’s Threat Modeling for free.

5. Build a Library of Secure Code

One shortcut to secure design is using existing secure design patterns and road-tested components. As you develop software, keep a library of secure code that has been threat-modeled and scanned using application security testing tools that you can return to in the future. The Threat Model Cookbook on Git Hub has some sample threat models you can use as a starting point.

November 17, 2023
Grafana Labs Acquires Asserts.ai to Bring AI to Observability

At its ObservabilityCON event, Grafana Labs today announced it has acquired Asserts.ai to automate configurations and customization of dashboards.

In addition, the company is previewing an ability to apply artificial intelligence (AI) to incident management to make it simpler to surface the root cause of an issue. Sift is a diagnostic assistant in Grafana Cloud that automatically analyzes metrics, logs and tracing data, while Grafana Incident is a generative AI tool that summarizes incident timelines with a single click, creates metadata for dashboards and simplifies the writing of PromQL queries.

Grafana Labs is also making generally available an Application Observability module for Grafana Cloud to provide a more holistic view of IT environments.

Finally, Grafana Beyla, an open source auto-instrumentation project that makes use of extended Berkeley Packet Filtering (eBPF), is now also generally available. That tool enables DevOps teams to collect telemetry data for an IT environment from a sandbox environment running in the microkernel of an operating system. That approach makes it simpler to automatically instrument an IT environment, but there are instances where DevOps teams will be managing complex applications that will still require them to collect telemetry data via the user space of an application.

Richi Hartmann, director of community for Grafana Labs, said collectively, these additional capabilities will make it simpler to apply observability across increasingly complex IT environments. For example, the AI technologies developed by Assert.ai will make it possible for DevOps teams to start sending data to Grafana Labs that will enable the cloud service to identify the applications and infrastructure being used. AI models will then be able to automatically generate a custom dashboard for that environment that DevOps teams can extend as they see fit, said Hartmann.

In general, machine learning algorithms and generative AI are starting to be more widely applied to observability. The ultimate goal is to automatically identify issues in ways that reduce the cognitive load required to manage complex IT environments while also making it easier to launch queries that identify bottlenecks that could adversely impact application performance and availability.

It’s not clear to what degree observability tools might eliminate the need for monitoring tools that track pre-defined metrics, but most DevOps teams will likely be using a mix of both for the foreseeable future.

In the meantime, IT environments are only becoming more complex as various types of cloud-native applications are deployed alongside existing monolithic applications that are continuously being updated. The challenge is the overall size of DevOps teams is not expanding, so there is a greater need for tools to streamline the management of DevOps workflows.

AI will naturally play a larger role in enabling organizations to achieve that goal, but it’s not likely to replace the need for DevOps engineers, said Hartmann.

Conversely, many DevOps teams will also naturally gravitate toward organizations that make the tools they need to succeed available. Today, far too many manual tasks are increasing turnover as DevOps teams burn out. Organizations that want to hire and retain the best DevOps engineers will need to invest in AI.

Of course, DevOps, at its core, has always been about ruthlessly automating as many manual tasks as possible. AI is only the latest in a series of advances that, over time, continue to make DevOps more accessible to IT professionals of all skill levels.

November 14, 2023
Improvement Cycles and AI

It’s that time of year when pundits start to crank out their predictions for the next twelve months! I’ve been asked to contribute to a private group (and occasionally, Techstrong Group asks me to contribute to their annual list), so I won’t thrill you with my predictions just yet; but I will discuss the cycles and trends that previous predictions have shown.

Back when I was testing physical servers, we thought of throughput bottlenecks in terms of where the problem was. The circle would go disk->memory-> CPU, and in the case of things like NAS, insert network in there somewhere, too. When one bottleneck was resolved, another would crop up. And users will always seem to be pushed to the next bottleneck.

We see similar cycles in DevOps. First, Agile made development faster, and that caused us issues throughout the rest of the process. Then, DevOps helped with the build and a little with the testing and deployment steps, but in the early days it was more standardization than speeding up. And since then, we have been in that cycle. We’re seeing GitOps speed up and automate deployment and upgrades—based upon the automation that came before—we’re seeing test and security tools feeding directly into the IDE to speed resolution, we’re seeing container automation cause “cloud-first development” (I hate that name; Kubernetes has cloud-like interfaces, but is not precisely cloud) to be fully automated, and thus faster.

And we’re looking at that cycle continuing. No doubt you will hear a lot of predictions that start with “AI …” For the most part, AI is a tool and/or technique. Listen for what it is improving because it is doing very little that’s actually new. (Delete list of things that sounded alarmingly like those predictions I said I wouldn’t make.)

All in all, we are still living in a very cool technological time. We can have a tool generate code snippets of the dull work for us so we can focus on the important bits. Some shy away from this use of “AI,” but honestly, we already use libraries to do the grunt work—I haven’t written to a screen buffer in forever—because we don’t have to. Same story—use the tools that make you productive. Some of the more obscure bits of Linux command lines are far better dealt with by asking an AI to give me examples than searching through tons of docs across several platforms. Never was much of a fan of man output for complex commands, this makes it easier. The nice bit is that bash and UNIX utilities don’t change much over time, so answers can be a bit dated and perfectly accurate. That’s not true for much of IT.

The environment has consistently gotten more complex, while the building blocks have gotten easier to use. This allows things to keep moving and changing while it offers us better ways to serve the organization’s needs. This does change at some point—infinite change is indeed a thing, but any technology does have its limits—but I honestly can’t see the point at which we are content with our IT tools and stop looking for more. Oh, most organizations go through those cycles—organizationally, once an infrastructure and process works correctly, it is likely to hang around until better is found—but the industry will keep changing.

And you’ll keep learning and changing. That’s one prediction I will make for next year. Teams and IT staff will grow and learn new things, because at its heart, that is what we do. We learn new things and keep the lights on. In good years, we offer sweet new tools and functionality to the company or our customers. But either way, we keep things online. Year after year. You all rock, so that’s an easy prediction.

November 1, 2023
DevOps Halloween: Tricks and Treats

The world of DevOps is like a labyrinth—filled with choices at every turn. Some paths lead to efficiency and success, while others may lead to unexpected challenges and delays. In the spirit of Halloween, let’s explore the tricks and treats of DevOps choices to ensure your team ends up with a bag full of treats rather than some nasty tricks.

Version Control: Treats of Consistency, Tricks of Complexity

Treat: Implementing a robust version control system is crucial for any DevOps team. Tools like Git provide a reliable way to track changes, collaborate on code and maintain a history of your project’s evolution.

Trick: However, the complexity of these systems can lead to confusion and errors if not properly understood. Branching strategies, for instance, need to be clearly defined to avoid chaotic merges and lost work.

Continuous Integration & Continuous Deployment (CI/CD): Speedy Treats, Tricky Configurations

Treat: CI/CD pipelines automate the process of code integration, testing and deployment, speeding up release cycles and ensuring more reliable software.

Trick: Setting up these pipelines can be complex and error-prone. Misconfigurations can lead to failed builds, delayed releases or even the deployment of buggy code to production.

Containerization: The Sweetness of Isolation, Beware of the Overhead

Treat: Containers provide isolated environments for applications, ensuring consistency across development, testing and production. Tools like Docker and Kubernetes have revolutionized application development and deployment.

Trick: Containerization adds an additional layer of complexity to your infrastructure. Mismanagement of containers can lead to resource inefficiencies, and Kubernetes itself has a steep learning curve.

Monitoring and Logging: The Treat of Visibility, The Trick of Overload

Treat: Comprehensive monitoring and logging give teams visibility into system performance and behavior, enabling proactive issue resolution and performance optimization.

Trick: The sheer volume of logs and metrics can be overwhelming. Without proper tools and strategies for filtering and analysis, important information can be lost in the noise.

Infrastructure-as-Code (IaC): Sweet Automation, Sour Complexity

Treat: IaC tools like Terraform and AWS CloudFormation allow teams to automate and version infrastructure setup, ensuring consistency and reducing manual errors.

Trick: IaC scripts can become complex and difficult to maintain. Errors in these scripts can lead to misconfigured infrastructure, potential security issues and resource waste.

Collaboration and Communication: Treats of Teamwork, Tricks of Misunderstanding

Treat: DevOps emphasizes the importance of collaboration between development and operations teams, fostering a culture of shared responsibility and continuous improvement.

Trick: Miscommunication and lack of alignment between teams can lead to inefficiencies, mistakes and a breakdown in the collaborative process.

Security: The Unseen Specter

Treat: DevSecOps enables a “shift left” on security, integrating security checks and practices early in the development life cycle, ensuring safer, more secure applications.

Trick: But this integration requires continuous attention and maintenance. Outdated dependencies, misconfigured settings and inadequate security practices can leave your applications vulnerable, turning the unseen specter of security issues into a ghastly reality.

What about you? What areas of DevOps are ripe for trick or treat this Halloween season?

In the grand scheme, DevOps offers a treasure trove of benefits, from faster releases and improved collaboration to higher-quality software. However, it’s not without its challenges. Navigating the DevOps landscape requires a careful balance between embracing automation and maintaining control. By being aware of the potential tricks and focusing on the treats, teams can build efficient, reliable and scalable systems. Like a group of trick-or-treaters navigating the neighborhood on Halloween night, success in DevOps comes from preparation, collaboration and the ability to adapt to unexpected surprises. So, put on your costumes, grab your bags and get ready to collect those DevOps treats!

Image Source: Photo by Antonio Gabola on Unsplash

October 31, 2023
AI

October 30, 2023