devopsy.kinsta.cloud

Tag: scripts

Generative AI: Some Parts of the Future Are Cool

I’ve been around long enough that I normally roll my eyes when ‘the new shiny thing’ shows up. If I start gushing about something, I truly see it as game-changing in a market space that is full of things that simply are not. Perhaps an example is in order. For years after college, my hobby was compilers and linkers. Until my Master’s program had a class in compilers and the newer variants of Lex and YACC showed me that those skills are archaic. Lots of fun, and I’ve kept my source to a 100% original MS-DOS linker, thinking one day I’ll update it to generate modern executables. But I won’t because the other thing that reduced the appeal of compiler and linker skills is open source. I can download code that, by virtue of being exercised by like a billion people, is inarguably better than my efforts were. Better to put my attention elsewhere. Open source was indeed a game changer. Today, when I see there is a new language du jour, my very first question is, “Okay, but why?” the answer is rarely, “To make developers’ lives easier.”

The same is true with generative AI. Just as open source has spawned a never-ending collection of failed or misdirected OSS projects, AI is starting to create buzz about dozens of “that’s not really useful” use cases. But amongst all those are the “This changes everything” use cases, and for IT–be they Devops or devOps, generative AI for code creation is huge. This will almost immediately become a mandatory job skill, and you should start learning how to get generative AI to spit out the code you need today. It will increase your productivity, help your company, and it will increase your career prospects, helping you.

Generative AI Goes Further

Two examples. In completely different projects that aren’t even for companies in the same space, Lori and I both used generative AI to develop systems faster. I know very little about Lori’s work other than that it was to generate Python that did the grunt work and took care of the repetitive bits she hates. My work was to generate bash scripts for things I don’t generally do in Linux. Any honest assessment agrees that the power of Linux is masked by the insane level of obscure commands. While researching online has helped us get over the “What the heck does that do?” for decades, generative AI goes one step further. It allows you to tell it what you want to do, then give it whatever additional information it might need to tweak the results, and the script is done.

We all use APIs, libraries, SOs and snippets pretty much daily; this is literally just an extension of that usage. Most of us would search online for examples of what we needed to do, then copy and modify them. This really is literally, “Give me an example of that. Good, now customize it to my needs.”

There are risks with any significant change, and generative AI for script development is no different, but the largest risk identified – that entry-level people will not learn and grow – is blown out of proportion. Given an example to read, the curious and those who are going to be responsible for supporting the system will go over it with a fine-toothed comb. The learning will be more reading than doing at that point, but in the end, the script still has to execute. Troubleshooting won’t perform itself, and people will continue to strive to understand — both because they need to when supporting systems, and because curiosity is indeed the hallmark of IT staff.

You all are the backbone that keeps the organization online and employees productive. We are way past the time when 90% of companies would fail without IT. Keep rocking it, but give your org that competitive edge. Take any tool that will increase your productivity and help your career in the process.

November 8, 2023
Scripts in the Development Pipeline: Best Practices for Avoiding the ‘Scriptocalypse’

The use of scripts as a key tool in the average developer’s bag of tricks has altered the development pipeline in significant ways. It has even led some organizations to outsource their entire team to manage the glut of scripts that have been developed over the course of time.

To help companies move beyond the “scriptocalypse,” we recently participated in a webinar with our friends at Tricentis. Sunil Mavadia, global head of customer journey at Electric Cloud, and Thomas Stocker, director of product management at Tricentis, walked through best practices to follow to avoid the scriptocalypse. Here are some takeaways from their discussion.

Improve Process Robustness and Scalability

Of course, everyone wants to improve their process robustness and scalability, but how? One of the most effective ways of doing this is to employ a tool that helps orchestrate your releases and provides reusable objects. This reduces risk because you no longer have to write scripts for every single application, which creates a cascading effect—not only do you write fewer scripts, but also you don’t have to maintain these scripts across the board. As a result, you are assuming much less risk across many levels of your technology and pipeline. Additionally, system-level visibility across all of your processes will increase—which has many benefits for your entire development life cycle.

Don’t Just Automate, Orchestrate

Many developers make the mistake of thinking that because they have scripts, they’re automated. But to be truly automated, every action needs to be automated and driven by your pipeline. Your approvals, your handoffs … everything. Oftentimes throughout the release process, team members find themselves having to stop in the middle of their tasks to ask for approval. This only creates a process that interrupts the entire pipeline. But what if you could automate all of that?

A great quote from the “2018 Accelerate State of DevOps Report” that speaks to this says, “High performers automate significantly more of their configuration management, testing, deployments and change processes than other teams.” If you look at it purely from a team perspective, the high performers are those that have fully automated all their processes. While this can certainly sound like a daunting task, it’s best to look at it with an agile mindset—this is an iterative process. You have to start somewhere, while keeping the goal of 100 percent automation in view.

Not only should you be automating, you also should orchestrate your pipeline. This is critical. By automating and creating reusable objects, you can get to the point of using your pipeline as a service. Imagine reusing this particular pipeline for every application within your process, so you could have hundreds, even thousands, all using this pipeline and all these objects underneath that are all reusable objects. It’s a pretty picture, isn’t it?

Create a Value Stream Map

When you create a value stream map for a pipeline, you want to be very inclusive about who you invite in the room to create the value stream. Include representatives from release management, development, deployment engineers, team leads, executives and infrastructure engineers. Basically, anyone who has “skin in the game” for your release should be present.

Once you feel you have a solid representation of who is impacted by your releases, sit down and draw out the entire process. This includes figuring out what each script does. While that may be a big task depending on how your pipeline is currently structured, there’s actually a lot of opportunity there. For example, you might find there is one application for which you use one script to do that deployment. Typically, that script is duplicated for another application and so on. Imagine doing that 100 times. You’ll have 100 scripts or variations of that one script for your 100 applications. That’s an obvious and beneficial place to deploy a bit of automation.

This is also a great time to understand what tools you are using for each step. When you’re jumping from toolset to toolset, it can be difficult to get a holistic view of how this is impacting your overall performance without a complete picture of the value stream. When you have that picture, you can start to ask yourself questions such as, “What are all these tools that we interact with that we can integrate into the pipeline? How are we handling approvals?” By including detailed sequencing timelines and identifying bottlenecks, you can realize some impactful efficiencies.

Finally, creating a value stream map gives you an opportunity to organize your scripts. Laying out the entire value stream can give you a picture of the variety and quantity of scripts you’re running—whether they are CI scripts, deployment scripts or test automation scripts—and bucket them in a logical way.

Identify Redundancy Dependencies and Bottlenecks

Of course, identifying (and hopefully rectifying) dependencies and bottlenecks is a good best practice for any aspect of your development pipeline. But it becomes even more important in the context of using scripts. If you have one application dependent on another application, it’s almost impossible to script. For example, imagine you’re trying to do some testing with two dependent apps. If you don’t have the right versions across both, you can’t test one because the other one is not up to speed or it hasn’t been updated in an environment. Something even as simple as that can be really disruptive and make scripting very difficult. By understanding where your dependencies and redundancies exist, you can take the first step toward solving the issues they present.

Scripts are one of the most effective ways of streamlining your pipeline, but they need to be used in a way that will allow your organization’s applications to be scalable and sustainable. By following some of the best practices outlined above, your organization can use scripts like pros—and avoid the dreaded “scriptocalypse.”

For more insights watch the full webinar here.

— Anders Wallgren

October 11, 2018
The Emergence of UI and UX in DevOps Tools
Most of the popular DevOps tools on the market such as Puppet, Chef and Jenkins, are open-source and require a high level of proficiency. With the complexity comes the challenge of learning new frameworks and paradigms, a benefit to those who want to challenge themselves and learn new skills.

But while these tools have become widely adopted, the widespread adoption of DevOps is creating a need for interfaces that cater to all users, from the most technical engineers to those who don’t want to learn, don’t see the value in learning or don’t need some of the top-end power that a complex scripting interface provides.

It is for this reason that CloudBees developed Blue Ocean, a new interface for its continuous delivery tooling, and Red Hat released a user interface (UI) for Ansible.

The Right Time for UI?

Why have there not been UIs before now?

I would say two reasons:
1. Users did not demand nor care for one until now. A UI in many circles is seen to been more presciptive/restrictive of what can be achieved, and that only scripting/programming interfaces expose the real power to an engineer. Also, many of the most progressive engineers like scripting; they derive satisfaction from learning new skills, especially those that are the most highly desirable/in demand. And, I think many engineers like things because they are difficult to do; it’s a challenge, they have bragging rights and means they likely are getting paid more/are in demand.
2. The open-source community is not as good at or geared toward building UIs.
Why has a UI become important now?

At MidVision, like many of the other commercial automation tool vendors, we have always believed that user experience and accessibility is critical, so we have provided command-line, scripted, API and GUI interfaces to our tooling. But this has not been the common approach.

What makes this moment different is the wider adoption of DevOps automation tooling by groups that want at least graphical user interfaces (GUIs) and improvements to user experience such that you don’t need to be super-technical to be able to use them.

The main groups we see are:
- Business users within an organization. Continuous delivery/release automation tooling very much promotes collaboration, interaction and self-service, allowing release managers to see the status or pipelines, approve releases or initiate deployments. Managers may want to see dashboards and metric, produce reports or usage data. Other less technical staff, offshore teams, second level support, etc., may want to be able to provision, clone environments or deploy changes.
  There is a gap between the nerds in the basement speaking one language – and the business users who speak another which accessibility, UI and UX hopes to bridge.
- Operations. There are certain parts of IT Operations that have welcomed the technical script-based tooling with open arms—it was a natural evolution for a UNIX system admin to become a DevOps/automation engineer. They can still script in shell (or are they meant to be doing it all in Python? ?). Again, some of the security tooling, storage, etc., are generally more comfortable down in the weeds. As you move up the stack however, toward the business application deployments, users are much more familiar and comfortable with UIs. Also, at this level dashboard, metrics, access control and reporting are much more important—and all of which are very hard to achieve without a proper UI.
- Wider industry adoption. Adoption of the tools and techniques that are often advocated in the DevOps world started with the same poster child companies—Netflix, Twitter, Facebook, Google—and other newer companies that are cloud-native, have an open-source-first policy and are unencumbered by heritage technology stacks.
We have then seen parts of enterprises adopt DevOps practices and tooling. Obviously, it is important to start small, establish value and then scale—running an initiative like you would a startup within an enterprise works well.

Now that there is a compelling body of evidence to suggest that DevOps practices provide competitive advantage, cost savings, improved time to market, etc., other industries that are heavily dependent on technology have not had yet adopted these practices for varying reasons. Some parts of the finance and healthcare industries are two such areas. These industries have concerns around security and stability that may have slowed their adoption to date.

Overall, however, this wider adoption means we need to cater not just for the new breed of DevOps engineers that are both system administrators and programmers and can script in Shell, Rudy and Python. We also need to cater to a wider group that either don’t have the skills to use these tools or don’t care for them.

What change, then, are we likely to see?

I believe DevOps tooling will evolve to provide new interfaces to cater to business users and self-service, and to make it easier to achieve the same thing without needing years of experience using a tool before being considered a competent user.

I would compare many of the current tools today to a map and compass. Lovers of orienteering or mountaineering may suggest using a GPS or map on a mobile phone is cheating, cannot be relied upon or won’t provide the same same information as the contours on a 1 to 10,000 map. Or maybe you prefer vinyl to Spotify.

Neither of these things are wrong; they are just not the only right. If you want to nerd out with your map and compass, vinyl LPs and Python scripts, I say fill your boots. All of which, BTW, I have some passion for! However, if you also want to get in your car, enter a zip code and know that it will get you there (accepting there is maybe a route that is slightly quicker if only you had done your research more thoroughly), this makes perfect sense to me.

User experience is at the heart of the most of the other products we choose to consume, and I think many new and existing users will be pleased with UX becoming more important as tools like ours and others put this at the forefront of their design.

About the Author / Dave Sayers

Dave Sayers is a co-founder and technical services director of MidVision, a provider of Release Automation and Continuous Delivery Software. He works closely with customers and partners on DevOps and Automation initiaitves throughout Europe and North America and is responsible for devising and managing the partner and channel strategy. Connect with him on LinkedIn.
May 30, 2017