devopsy.kinsta.cloud

Tag: bots

Creating Automated GitHub Bots in Go

Events management requires quick responses, which can be challenging since there is typically an incredibly large volume of events that take place every day. Therefore, relying on humans to respond to these events quickly is error-prone, messy and often frustrating. These events have easily reproducible steps that can be codified and are perfect reasons for a bot to exist.

Creating a bot reduces the daily noise of events so that important humans can focus on important human activities—such as programming, innovating or, most importantly, sleeping. There are already many bots that exist and that can solve numerous problems. For example, Dependabot helps keep your GitHub dependencies updated with automatic pull requests. Google Calendar has a Slack bot that reminds you when a meeting is about to start. Of course, there are also situations where there isn’t a pre-made bot yet, which provides the perfect opportunity for developers to create their own!

My preferred programming language is Go, and I’ve used it to create multiple bots. A favorite bot I created helps to manage a large open source project called Telegraf, which I discuss in more detail at the end of this article. Bots are not limited to open source projects; another bot I created proved valuable when I worked for a larger enterprise company. It was a Slack bot that could run commands on Amazon EC2 Windows instances and help teams quickly perform actions for customers, such as resetting passwords in seconds as opposed to days.

Creating Your Own Bot for GitHub

Let‘s look at how you can get started creating a bot written in Go that can interact with GitHub. The package google/go-github lets you interact with the GitHub API and makes creating a bot for GitHub easy. Tools like these are your first step in developing your own bot. Using this package, you could create something similar to what Dependabot does. In order to begin receiving GitHub events for you to react to (such as someone creating a pull request), you need to create a GitHub App.

The GitHub documentation does a great job describing how you can do this, but the important piece to be aware of is a callback URL. The callback URL will be a publicly available address associated with your Go program so that GitHub can route events to it. I use AWS Lambda functions to host the bots I’ve created and managed by using Serverless Framework, but you could use anything you want as long as you get a public URL. If you are interested in using a Lambda function as well, your “main.go” can be as simple as just calling “lambda.Start” and passing in the function that will hold the business logic.

Because you are using a public URL, security is important. Once you have created the GitHub App, you will be given a secret token that you can use to help validate incoming requests. The google/go-github package makes this easy; all you have to do is call github.ValidatePayload and pass it your secret token and incoming HTTP request to ensure the payload is valid. If successfully validated, the function will return the JSON payload that defines the event you just received. You can parse the payload by using the function github.ParseWebhook, which will return to you an empty interface and can hold any value. Therefore, you will need to use type assertion to access the underlying type to see what event you have received.

Depending on the event type, you might want to take different actions. Perhaps if someone made a new issue you would receive the type github.IssuesEvent, which would be good to perform initial triage such as adding the appropriate labels. It is also possible to receive the event type github.CommitcommentEvent, which would indicate someone had just made a comment that you could check for special commands such as automatically restarting the build system. You could, for example, also create a welcome comment whenever someone creates a new issue, as seen in the code snippet below.

Creating a Slack Bot

These examples have focused on interacting with GitHub, but can be replicated to create a Slack bot similar to the Google Calendar bot. The package slack-go/slack allows you to interact with the Slack API with which you can post messages or create “modals” that allow users to interact with custom buttons and lists.

Creating a Slack bot is a similar process to creating a GitHub App. Slack provides a user interface to register a bot, and similar to GitHub, the main thing it needs from you is a public URL so it can send your bot events. You can then also verify incoming events by using the function slack.NewSecretsVerifier using the secret token provided by Slack. As an example, once these steps are completed, your bot can then begin sending messages to channels to help you answer questions.

Bringing it to Life

An example of a bot can be found in the Telegraf project, which is InfluxData’s open source data collection agent. We have a bot written in Go helping us maintain the repository named “telegraf-tiger.” One of the first interactions users have with the bot is when they submit a new pull request. When the request is made, the bot will chime in with a friendly reminder to sign our contributor’s license agreement (CLA).

Once signed, the bot can be asked to verify if the CLA has been signed by adding a comment “!signed-cla” and give the pull request a shiny green checkmark to let the maintainers know everything is ready.

Another area our Tiger bot supports is sharing the artifacts that our continuous integration (CI) system builds for each pull request with a helpful comment that links to each type. This helps spread awareness of the artifacts so that anyone can easily help test the new feature or bug fix without having to set up a local developer environment themselves.

The Big Takeaway

Bots are an exciting developer tool that can automate mundane assignments and allow engineers to focus on bigger, more engaging tasks. Resources such as GitHub also make it easy to develop bots using Go in various ways. Bots are commonly found across interfaces, with some simply reminding us of upcoming meetings, and others ensuring that important license agreements are signed.

The use cases for bots are endless, and with the right tools, bots can help engineers focus on more significant tasks at hand. Developers can rest easy knowing their bots are automating tasks and responding to the endless alerts they don’t want to deal with. Instead, they’re able to focus on important human activities, like sleep, so they can dream up the next big thing.

May 18, 2022
The Role of Bots

April 22, 2020
Bots: ‘Like a CLI on Steroids’

Overheard in a Palo Alto coffee shop:

“… so, we tokenize the utterances, lemmatize the tokens, vectorize with StarSpace embedding, and run the featurized input through the neural net. The softmax activation layer tells us what the user intent was.”

“Sorry, you what the what and then what?”

“Or, you can say the user types commands, the bot interprets those and takes the appropriate action.”

“Oh, now I get it; it’s like a CLI on steroids!”

Calling a chatbot “a CLI” is like referring to a mobile phone as “a portable wireless teletype”: not even close to a complete representation, yet not entirely wrong. Like a smartphone, a well-trained bot will do as much or as little as the user requires of it.

Historical Perspective

The command-line interface (CLI) dominated human-computer interaction for a few decades, until the growing number of options software offered made GUI the new favorite way of talking to computers. Both of these interaction models are based on human to human communication: the operator communicates “an intent” complete with some input “entities,” each inserted in its “slot.” This is exactly how bots operate, so how is a bot not just another UI?

Fig. 1: User Interface Elements

Bots as UI

While a user interface is merely a human-readable catalog of machine’s capabilities and a set of input slots tightly coupled to a specific application, an NLI-enabled bot acts as our agent: it infers the intent (even multiple intents) from a natural language utterance, identifies any entities in that utterance, such as food: pizza, toppings: (olives, pepperoni) and fulfills that intent, be it by talking to Pizza Hut’s API or by calling the local pizza shop on the phone. Or even by making the pizza itself.

NLI bots have already made quite an impact on the sales and support side, but how suitable are bot agents for more technical work? How would a chatbot perform in the DevOps world, replete with esoteric terms and complicated interfaces?

Although there is a lot less training material available (after all, more people eat pizza than reconfigure Kubernetes clusters), there are also a lot fewer ways to express a technical intent, so in the end the technical bots are easier to train and make fewer mistakes than their sales brethren. But is a bot’s ability to classify an intent to deploy a new EC2 instance of specific flavor enough to make hardcore DevOps techies give up their favorite CLI?

Beyond Interface: Advantages of Bot-assisted System Interaction

Context memory: Whether a bot remembers your favorite pizza toppings, or you have to say “pepperoni and mushrooms” a couple of times a week is of little consequence. However, if your input looks like this:

I’m sure you’d rather not have to type that ever again. The machine learning models of the leading bot platforms not only remember your slot values, they also remember the context in which those values were entered, so a well-trained bot is always able to suggest an appropriate value when looking to fill a slot.

Natural language understanding: Just like a human would not be fooled by variations in the word order or use of synonyms, neither would a well-designed bot. Not having to remember the exact syntax or even spelling translates into higher efficiency.

Fig. 2: Intent classification and slot tagging

Continuous training and improvement: The more time a bot spends with a team, the better it gets at its task of helping members of that team, not only due to filling its context memory with details about team members and tasks, but also because of reinforcement learning. In addition to the initial training corpus of data, most contemporary bot platforms feature interactive training mode, wherein a human user can provide a real-time feedback on the bot’s performance.

Seamless automation: Since it’s an agent rather than a mere translation layer, the bot remembers successfully fulfilled intents and chaining those into a sequence, with or without slot elicitation, with or without branching and decision making is significantly simpler and easier than even the simplest of scripts.

Collaboration platform integration: Teams typically meet in a collaboration platform such as Slack, WebEx or MS Teams, decide what needs to be done, then go and do it. With a bot on their team, they can get right to the “doing” part without leaving the collab platform; avoiding context-switching improves productivity.

Built-in analytics: A bot’s main job is to classify intents, fill slots and invoke fulfillment. However, in its spare time it could listen in and take notes. At the end of an event (release, incident, outage, etc.) it could use its NLP savviness to provide a linguistic breakdown of the event: How many questions were asked? How many answers? How many statements? Or, even, What was the level of profanity use as compared to the average? This kind of summary provides benefits for parts of the company beyond bot’s original team.

Conclusion

Bots can and do increase productivity in a software development or operations environment. Automation is important, but when done through traditional means such as scripting, its high cost often causes it to be prioritized behind other tasks. The use of an NLI-enabled bot as an agent for interfacing with various back-end systems not only ensures the very best in user interface (“CLI on steroids”), but also minimizes the need to switch contexts and significantly lowers the cost of automation.

More importantly, the emerging agent-facilitated process control prepares us for the next phase in the human-machine interaction: the post-app world.

Bots will help us navigate the new world of commoditized services that will replace the familiar apps. They will track our experience and base future selections on our satisfaction with their past choices. What more can we ask for? Aren’t most of humanity’s dreams centered on intent (wish) fulfillment? What are the wish-fulfilling creatures from the fairy tales if not expressions of human desire for the perfect bot?

— Mike Dimitroff

December 5, 2018
Automation: Addressing Bot Security with BotChain

Bot technology is all the rage these days, and with good reason. Bots help you shop and provide customer service. In fact, one bot keeps contacting me on Facebook, telling me how I can optimize my marketing spend. They’re everywhere and they’re getting smarter all the time. Any one bot can access the entire intelligence of the internet to answer your most arcane questions or give you insightful suggests tailored to the details based on your online profile.

Still, the full potential of bot technology has yet to be realized. Right now, most bot interactions take place between a bot and a human. But, what would it be like if bots could bring their powers of recognition, recommendation and bargaining to bear interacting with other bots? Such a time is not that far away. The implications will be profound, particularly for those in DevOps.

The Emergence of Bot-to-Bot Communication

Machine-to-machine communication is nothing new. Unassisted stock trading has been going on for years. A machine monitors stock exchanges throughout the world looking to buy or sell stock according to its programmed logic. Once an opportunity is identified, a trade takes place without any human intervention. While it is true that each machine is acting with some degree of autonomy, the scope of interaction is limited. A machine designed to buy and sell stock is not going to go awry and buy a car. Also, the details of the interaction are well-known. When a trade takes place, the details of the transaction are recorded and an audit trail is created on both ends of the transaction. However, when it comes to bot technology, things start to look different.

Bots can act autonomously. For example, imagine a bot that is on a mission to find the best price for red sneakers in size 9. Instead of using a predefined list of sneaker vendors, the bot can use an internet search service such as Google to find online merchants that sell sneakers. The bot will use the result of the search to continue forward, looking for the best sneaker price. (See Figure 1.) As you can see, there is a good deal of machine autonomy in play already.

Figure 1: Bots use the internet and can act autonomously

Once the best price for red sneakers in size 9 is identified, typically, the bot will defer back to a human to make a purchase decision. However, deferring to a human is not mandatory. As we saw with the stock trading scenario, a machine-to-machine purchase transaction is entirely possible. Thus, given the proper programming, there is nothing to prevent a “buyer bot” from interacting with a “seller bot.” Furthermore, it also is entirely possible for the “buyer bot” to interact with a variety of “bank bots” to determine the best credit card to use to make the sneaker purchase. Who knows? Credit card “A” might be offering better air miles than Credit card “B” on that purchase date.

The important thing to understand is that, in the past, machine-to-machine interaction was pretty much a two-party interaction that is well-known. Today, modern bot technology makes it possible for a bot to engage in any number of transactions with any number of other bots with a high degree of autonomy. Along with such autonomy comes a good deal of risk: Any one of those bots could be an impersonator or could be using fraudulent information.

The Perils of Bot Impersonation

Any transaction is subject to fraud—a person writes a check without the backing funds, another makes a purchase using a stolen credit card, somebody else submits an invoice to a company by impersonating a vendor. These types of misdeeds happen all the time within the scope of human activity. So, too, will such crimes happen with bots. Just as a cybercriminal can impersonate a bank website to lure unsuspecting customers into giving away money and sensitive information, bots will be able to fool other bots into fraudulent transactions.

As bots become increasingly autonomous, the degree of harm that one bot can perpetrate upon another will grow, too. So, what’s to be done?

Ensuring Bot-to-Bot Interaction Using Blockchain

If you want to know that bots are playing by the rules, you have to be able to observe their behavior. For those in DevOps, the usual way to observe the behavior of the systems within an enterprise is by analyzing logs.

Logging is a way of life in DevOps. We put logging statements into the software we write to report the details of the commands executed; we put entry dates in the records we write to our databases; we log the requests being made to websites, and administrators use logs to provide the audit trail necessary to determine the integrity of the transaction being made in systems and between systems. Coupling logging with comprehensive system security and data encryption practices makes it so that IT operations knows who is in the systems and what those entities are doing.

However, given the autonomy that bots can have and the wide scope of interactions that they can engage in, the usual mechanisms used to determine transactional integrity degrade. Going back to our red sneaker example described above, imagine the “buyer bot” contacts an unknown number of “seller bots” to negotiate the best price for a sneaker. Part of the negotiation process is in a bidding war among all the “seller bots.” Then imagine that once the sneakers have been identified, the “buyer bot” negotiates with two “bank bots” to determine the best credit card to use. That’s a lot of transactional activity, all of which is dynamic and most of which is unknown. How do we know that each of those “seller bots” is authentic? What do we know about the details of the negotiations done by all the bots in the bidding war for the sneakers? How do we know that the credit card used to make the purchase is not stolen? The fact is, without a common source of truth that describes all the transactions in the sneaker scenario, we don’t.

While it is true that each bot might keep records of its part in a given conversation or transaction, there is no common ledger that records all interactions made by all the bots in the given scenario. Without a common ledger by which to audit the activity, the security, integrity and quality of bot behavior is compromised.

This is not a new problem. Those who work with cryptocurrencies have had this problem for a long time—making sure that the information provided in a transaction is authentic, true and auditable. The way that cryptocurrency systems solve this problem is to use blockchain technology. Given that cryptocurrencies and bots share many of the same security issues, blockchain technology can be applied to bots. In fact, there is an emerging technology that applies the principles of the blockchain to bots. It’s called botchain.

Botchain makes it so that all activities conducted among bots are reported in a common, secure ledger. The botchain ledger is a distributed resource on the internet. Botchain provides the mechanisms and audit trail required to ensure that bot-based transactions on the internet are conducted by authentic bots, acting within the scope of their rights.

Botchain technology is projected to be an important part of the modern internet, particularly as IoT technology grows. As bots become more a part of general DevOps landscape—particularly in the e-commerce-enabled IoT space (think: refrigerators that can buy their own inventory)—systems and engineering staff will need to become more adept at using botchain technology to protect the enterprise.

Bots will bring added benefit to the enterprise, no doubt, but they will also create new types of threat vectors. Agent impersonation is but one one of many security risks that are sure to emerge. The wise DevOps organization will do well to prepare. At the least, having a working knowledge of botchain will be a good insurance policy to battle the threats on the horizon. As those of us in IT have learned, the best insurance you can buy is the insurance you never use.

Mastering the details of botchain technology is an excellent way for DevOps to address the security issues that will arise with the increased use of bots in the modern internet. There is little downside in preparing for a future that is sure to come.

— Bob Reselman

October 20, 2017