devopsy.kinsta.cloud

Tag: Heavybit

  • Don’t Make Me Code: Environment Protection

    Don’t Make Me Code: Environment Protection

    Thanks for listening to “Don’t Make Me Code,” a Heavybit podcast that discusses developer experience design and the unique challenges of building developer-facing products. “Don’t Make Me Code” is hosted by Steve Boak, co-founder and CPO at Opsee, and David Dollar, co-founder and CEO at Convox.

    In this episode, Steve and David are joined by Sean Li, lead product designer at Docker. Sean, Steve and David discuss Docker’s acquisition of Kitematic, the friction of attempting to replace a developer’s existing tool, and the increasingly prominent role of design in developer tools.

    This podcast is brought to you by Heavybit, a program that helps developer-focused companies take their product to market.

    [soundcloud url=”https://api.soundcloud.com/tracks/274576622?secret_token=s-QHiyf” params=”auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&visual=false” width=”100%” height=”166″ iframe=”true” /]

    Steve: We’re calling this episode of “Don’t Make Me Code” “Environment Protection,” and we’re here with Sean Li, lead product designer at Docker and one of the founders of Kitematic.

    Sean: Thank you.

    Steve: Thanks for being here with us. This is a really exciting conversation for me because we get to talk to a product designer for a company like Docker.

    Sean: It’s good to be here.

    Steve: Kitematic is a real interesting project for a bunch of reasons. I mean, it caught the interest of Docker. It’s also proven to be a really novel bit of development environment technology. And so we’re curious how it all got started.

    Sean: It got started in a university dorm room. I was a student from University of Waterloo in Canada, and the school actually offers a pitch competition. So we had this idea around making development environments easier, and so we pitched the idea and won. That’s how we got enough money to get started.

    Steve: Wow, and was it at all tied to Docker at that point?

    Sean: Docker was actually what we used for the back end. At the time, Docker was version 0.3. And then we’re on Hacker News, and we’re like, “Oh, this is interesting. It could be super useful technology to do what it wanted to do.” Our idea was essentially like Dropbox for your code.

    You get a Dropbox-like icon on your task menu, and then you can put code in it. For example, Django, it would just run it for you. And you can see it on the browser.

    Steve: Oh, nice. And so you have your production environment there, compiling for you and ready to go.

    Sean: Yeah. Then, when you added the code, it will sync to the cloud environment and then update right away.

    Steve: Nice. That’s awesome. It’s like a step beyond GitHub. It’s not just committing the code; all the CI and everything is built into this one workflow.

    Sean: That’s like the future plan right at the time, but at first, our MVP was super simple—just syncing code to a remote server and then seeing the changes.

    Steve: From the beginning it was implemented as a desktop application, you said like a Dropbox tray application.

    Sean: Yeah, because Dropbox was obviously the inspiration.

    Steve: And so why a desktop application?

    Sean: At the time, we actually didn’t realize the power of desktop application yet. At one point we used to pivot a lot, as startups usually do. And then we’re talking to customers, and they start realizing what we’re using underneath.

    We talk to Canadian Tire, the biggest retailer in Canada. They were looking for something like this, and they’re like, “Are you using Docker underneath? We want to use that.”

    At the time, the product we have is in the cloud, like an on-prem web browser setup. And then we started thinking of, instead of abstracting away Docker, why don’t we embrace it and then make it easier for the customers? Then we start brainstorming ideas, and it came across that having a desktop application is actually super beneficial.

    It’s like the best platform for developers, because they can import their code super easily. They can open a directory instead of having to upload files in a browser. You can also get a lot of access to their desktop, so once your tool gets more powerful, there’s more things that you can do.

    It’s also super cool because we use Chrome in an electron shell. And we wrote the UI in React, so that means we only have to worry about how the UI runs in that one browser, instead of having to worry about browser compatibilities across the board.

    Steve: So it’s not something React needed, it’s just running in a web view in the electron browser.

    Sean: And we make it feel as native as possible.

    Steve: All right, nice. So it’s a true desktop app in many ways, but specifically the UI, it’s a web UI.

    Sean: A lot of inspiration comes from the Atom Editor by GitHub. They’re using the same technology.

    Steve: And so this whole bit was open-sourced from the beginning?

    Sean: We were working on it, and then we open-sourced it.

    David: I’m curious what made you decide as you’re releasing this tool. Was open-source service a strategic decision, or you figured you could get more users by doing that? Or is it just wanting contributions from the community? Could you talk through a little bit of your inspiration behind that?

    Sean: I think the biggest part is if it’s something that touches their code, it’s really important that the developer trust it by seeing what the code is like.

    We’re really open about what we do, and also the part on gathering the feedback from the community is super, super helpful. I feel like, in general, open-source projects get more feedback from the community.

    Steve: With developer tools it seems like a great way to encourage participation. If you’ve got an open-source project, you’re inviting people to contribute. Obviously, our Opsee product for example, it’s closed, a SaaS application. So we don’t have the level of community support from developers that we would really like in some ways.

    Sean: We hired an engineer from Docker because he was a maintainer and he was super passionate about the project. It really helps on that front as well.

    Steve: Following from that, you launched this to an open-source community, did you have an idea of the kinds of users you were targeting at that point?

    Sean: We sort of did from the original product that we had, the Dropbox-for-code product. We had a signup list. A lot of these users are more leaning towards willing to use UIs and prefer something simpler. The super active ones are actually consultants that work on multiple projects.

    Steve: That’s an interesting segment. People there, maintaining a lot of projects, and they’re looking for an easy way to keep track of all that stuff. And Kitematic fit that mold.

    Sean: Yeah, and Docker being isolated also provides that value.

    Steve: That too. Now, if I’m maintaining multiple projects, I can have a Docker environment for each one. The app makes it very easy to move between those. And so specifically the React UI and the GUI, what was it like having that be part of the open-source project? Did you get contributions for that as well, or was that entirely internal?

    Sean: The React UI helped us in a way, because React is, according to a Google trend, almost on the same trajectory as Docker. So there are a lot of people willing to work on something that’s built in React. It was in a way accidental, because when we started, it was a really good technology to use.

    Steve: Did you have any designers contributing to the project?

    Sean: We do have design-minded people but not professional designers.

    Steve: For example, you’re a lead product designer at Docker and you, I assume, identify mostly as an engineer, or your background in coding. But there’s definitely this hybrid world in the developer tool space.

    Sean: Definitely. In the beginning we had hackthons. A lot of participants were trying to add features to Kitematic, then make a huge effort on figuring out the right UI right experience to build there. By default, I think what I wanted is sort of like designing an OS.

    A UI is very extensible, and you want to figure out the right points where you can extend on it. You actually focus on per component and really set an example of how you extend the current UI. When people contribute, it can follow that pattern. And you only had to make slight adjustments to it.

    Steve: That seems true of open-source projects, generally, too. You want build a broad base that people can build on top of and customize.

    Sean: Yeah, but the initial effort has to be really strong. Because otherwise you will get swayed by different opinions. And design by committee isn’t always a nice thing.

    David: That’s pretty interesting because you oftentimes see open-source projects that are sort of notorious, especially visually, for bad design or design by committee that ends up being a little too weird. I can think of some Linux desktop managers that I won’t name, but Kitematic is actually a beautiful application.

    Sean: Thank you.

    David: I’m curious how you managed to maintain that, even when you’re doing community-driven design.

    Sean: I think a big part of it is keeping focus and actually be willing to cut things. In the beginning, Kitematic actually had build, you can actually build an image and then push it. But to keep focus and make things simple before we figure out the next steps is actually making it run content only.

    Essentially the focus of Kitematic was helping users install Docker on their computer. You know, in a visual way, it feels very reliable. We solve a lot of underlying issues with VirtualBox in the beginning. And then after that choose a beautiful catalog where you can click one button and run a container. And then that’s the experience we were aiming for and focused on. So it’s super obvious what to do.

    Steve: Did you ever get push-back on those decisions from the community where you had to assert your position strongly there?

    Sean: No, I think the community is mainly supportive. What I had to manage was people trying to add too many things that are not focusing on that road map currently.

    Steve: I think I remember you saying, on the Software Engineering Daily podcast, that you did end up cutting quite a few things from the product after a bit of time. Can you talk more about that? You were trying to clean it up?

    Sean: The one thing that was cut was building images from it. Even though it’s a super important thing, we decided to delay it because we want to do it right. Instead of releasing something super early, and then get sidetracked into different things, we want to keep focus until we figure out this is it.

    And then release that as a whole. Talking about cutting more, the next version of Kitematic is sort of Docker for Mac. It really solved the core issues that Kitematic was trying to solve in the beginning, to install Docker reliably on people’s desktops in a way that it almost feels like it’s seamless.

    You don’t feel like it is running inside a VM at all, and that’s the goal in the beginning. You install Docker for Mac. When you open the terminal you can type “Docker run,” and it just works.

    Steve: Had the Docker for Mac project begun before the Kitematic acquisition?

    Sean: It was after the acquisition.

    Steve: So you were involved with that from the beginning.

    Sean: Yeah, I was the founding member of Docker for Mac.

    David: So it’s interesting to think about this this sort of internal debate. I guess with most design questions it’s customizability and configuration versus limited options, and this curated a beautiful experience. I feel like that happens even more with developer tools.

    It seems like Kitematic and Docker for Mac, some of these tools, are definitely trending towards the latter there, where you’re removing some of the optionality of Docker to make a better user experience. I think that’s really interesting.

    Sean: I think the two tools are, in a way, more complimentary. People usually use Kitematic by having the Kitematic UI side by side with the terminal. So they like the transparency that they’re getting from the Kitematic side, and the sense of it feeling reliable.

    “I see things are running. I can check the logs super easy.” On the other end, the terminal is how they’re used to working before, and that was sort of expected in the beginning. And then you go from there. You never want to displace a tool that they’re already using, one they’re so familiar with.

    So instead we embraced that but made Kitematic more of a complimentary to the Docker CLI.

    Steve: That’s really interesting. I’ve seen similar comparisons made with GitHub and some of their tooling. GitHub has their Mac app and their web UI and stuff like that. And I found myself going to them for things like diffs and comparison, merge states, something where I want to get a lot of text or get some status update.

    But for actions, if I want to do merges, everything I’m actually doing with it I’m still doing through the command line. But that’s a really interesting comparison, because it sounds like you’ve made the same distinction. To view things, maybe you go to the UI. But to actually take action?

    Sean: GitHub desktop is like a huge inspiration for all these decisions as well.

    Steve: I never really heard that distinction made this clearly, that there are certain things that developers are going to continue to want to do through the command line and through terminal. But that we can make the experience better in some ways, like just figuring out whether a container is running or not. That’s really nice to have a UI to see that all the time.

    This podcast is brought to you by Heavybit, a program that helps developer-focused companies take their product to market. Throughout the program, founders work on developer traction, product market fit and customer development. If you like what you hear, be sure to subscribe to Heavybit’s Podcast Network on iTunes.

  • To Be Continuous Podcast: Have We Forgotten How To Code?

    To Be Continuous Podcast: Have We Forgotten How To Code?

    Welcome to To Be Continuous, a show about continuous delivery and software development, hosted by Heavybit members Paul Biggar, founder of CircleCI, and Edith Harbaugh, CEO of LaunchDarkly.

    In this episode, Paul and Edith discuss the fallout after the widely covered “left-pad” incident, and while they agree that these sorts of moments move the entire open source community forward, they wonder: Have we forgotten how to code?

    [soundcloud url=”https://api.soundcloud.com/tracks/264605904?secret_token=s-Kv9Ep” params=”auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&visual=false” width=”100%” height=”166″ iframe=”true” /]

    Edith: Hey, Paul. So, somebody unpublished a module called left-pad and the internet basically exploded. And now it’s been a couple weeks since then, so there’s a lot of different topics to cover in terms of what this means for open source, whether it means that people have forgotten how to program.

    Paul: So LaunchDarkly. Or, I’m sorry, the left-pad. I’m obviously looking at my own t-shirt.

    Edith: You’re looking at your LaunchDarkly t-shirt?

    Paul: The left-pad incident was pretty interesting and one of the things that we did when coming up to this was just, like, looking at all the Hacker News discussions and comments on it.

    And there’s wildly varying positions on what this means for open source, what it means for whether people can program, what it means for, like, about the Node ecosystem. So to kick this off, why don’t you give us a recap of the entire incident?

    Edith: Yeah. So, in short, a programmer had a module called Kik, K-I-K. Which is also [the name of] a very popular messaging app. I mean, even I’ve heard of it and I’m not a teenager.

    Paul: The Kik lawyers contacted the dude. Said, “Please take down your module, change it.” And the programmer said, “No. I basically don’t believe in copyright laws.” So then, the Kik people contacted NPM. The NPM people took down the module called Kik. And the guy, in I’d say, kind of a fit of pique, said, “Okay, you took down my one module, I’m gonna take down all my modules.” Basically, I’m taking my marbles and going home.

    So, he took down all of his modules. One of which turned out to be an incredibly popular module called left-pad, which did basically what the name implied. It sorted left pads. And without this in NPM, the quote I read was, “Builds all over the world started to fail very quickly.”

    Paul: So as I understand it, left-pad was used by another module, which was used by another module, which was used by something core, or like, this incredibly popular module—

    Edith: Yeah, it was basically turtles on top of turtles on top of turtles, and at the bottom turtle was this guy who—

    Paul: Took his marbles and went home.

    Edith: Yeah, or yank the bottom Jenga, basically.

    Paul: Gotcha. And so all the builds started to fail. People’s deploy started to fail. And people got pissed.

    Edith: People got pissed because they didn’t even realize that they had this dependency. So what happened was NPM basically republished it. They said that, even though you’re the original publisher—

    Paul: So there was a lot of outcry on the internet and then they republished it.

    Edith: Just basically said, “You don’t have the right to un-publish this.”

    Paul: Gotcha. So, um, but they had an Unpublish button, as I understand it. So he did something that was built into NPM to remove a module.

    Edith: Yeah, and since then they have tightened their rules a lot. They basically said that this should be something that’s used extremely rarely, if at all, and not something you can just do capriciously.

    Paul: So one thing that’s interesting about the NPM ecosystem, and the Node ecosystem in general, is they’ve had a bunch of things like this. Things that are security-related, or that are drama-related.

    And people have repeatedly come out and say, “Oh, no, JS is a tar pit, or ghetto,” or whatever it is that people use to describe poor ecosystems, that normal programmers don’t know how to code, and so on, so on.

    And what it looks to me like is, you know, each of these incidences brings the Node ecosystem quite a bit forward. To the point that, I mean Node is relatively new, it hasn’t had 15 years before prime time in the way that Ruby or Python did, or 20 years before it got big, and it’s a real, kind of bottom-up thing.

    I mean, if someone built it and there were some steroids and then someone else took over, and then NPM took over, and then. It’s this sort of weird language from the ecosystem perspective. It didn’t have the same advantages that, like, Java had, or that C or C++ had, in order to get to maturity. And so each of these incidences propels it closer to maturity.

    Edith: Yeah, I compare it somewhat to when electricity first started coming out. And then you were just working through all the issues that could happen. Like, for example, when a power line went down or a transformer blew.

    Like, you work through, Okay, what are our points of failure? Let’s not make that a point of failure again.

    Paul: Hmm. So, NPM changed their policies on this basis, right?

    Edith: Yep, so now you can’t have an individual contributor be able to unpublish, even if they originally published. Now you have to contact them, and it’s much more stringent.

    Paul: Gotcha. The response, despite people being … Well, people were annoyed at their response.

    Edith: It seems perfectly rational to me.

    Paul: Oh, I think it seems perfectly rational. This is the thing about the in trend. There’s always some set of people who are annoyed at everything you do.

    Edith: Well you know, the internet is full of billions of people. I mean, there’s gotta be some sort of standard deviation, or otherwise it would be a very boring internet.

    Paul: Right, right, and this is why there’s always haters on Hacker News. No matter what you do. If you do it one way, there’ll be one group of haters, if you do it another way, there’ll be a different group of haters.

    And so, with left-pad, there were people who were annoyed that NPM retroactively decided that they had ownership over their namespace, which previously, they had sort of gifted to people in the community, the first people to take the namespace.

    Edith: You mean the original thing about Kik?

    Paul: So there’s that, but there’s also the left-pad. They took away the sort of autonomy of the left-pad offer.

    Edith: I read a good article that basically says part of open source is, if it truly is open source, do you maintain the rights to it? Or is it a community project?

    Paul: Hmm. The traditional way of deciding that sort of ownership was the fork. That you could go, and you could fork, and when you forked, you weren’t necessarily entitled to the same name, you weren’t entitled to the same trademarks. When Debian forked Firefox, they called it Iceweasel.

    When Jenkins was forked, or Hudson was forked, it became Jenkins. GCC got forked. They all got new names. But what was the fork here, it wasn’t even a fork. It was a republish of the same thing, the same software, in a namespace that the original author didn’t control.

    So typically you’d own the URL, you know, if you have a software library, you’d own the URL. And they did own, in some sense, the URL within NPM, or the name within NPM’s space, and NPM, the sort of owners of that namespace, overrode it.

    Edith: I think they did what was necessary for good order.

    Paul: Sure. I completely agree, I think that was a good decision.

    Edith: I mean, so to take this to the logical conclusion, what if you had a module named left-pad, and you decided that, like, as a prank you were going rewrite it to be right-pad? Like, sure, that’s … Is that within your rights? Kind of, but it’s a real jerk thing to do that you probably want to prevent.

    Paul: This is kind of one of the dangers of the whole ecosystem around centralized package managers, of sorta trusted package managers, where there’s no actual trust involved. So, what I mean by that is what is published does not have any gatekeepers.

    Edith: Yeah, exactly.

    Paul: I mean, it’s great, it allows an ecosystem to grow much faster, but it has safety concerns because people generally don’t pin their packages to an exact version.

    Edith: You’re basically pinning it to a floating value.

    Paul: You’re pinning it to a floating value. And that floating value could change. There’s no guarantee of what a minor number means or what a major number means or what a patch number means or what a patch even means. There’s some sort of, like, general community guidelines in semantic versioning, but there’s no enforcement of that.

    Edith: Yeah, and I mean this basically goes back to the whole Cathedral versus Bazaar thing.

    Paul: Right, right. It’s very Bazaar. And intentionally so. It’s the thing that allows a community to grow, and is there a point at which open source communities need to, like, batten down the hatches to prevent that sort of behavior? Or prevent that sort of risk?

    Edith: Yeah, and it’s really interesting. I think this goes back to something we’ve talked about with Nadia Eghbal and Sean Byrnes a couple episodes ago about when you buy from a corporation, you have some sort of guarantee of standards that they publish somewhere. You know, this is our guarantees versus with open source, you don’t normally have those same sort of guarantees.

    Paul: Even when you do, it tends to be—

    Edith: There’s no enforcement mechanism.

    Paul: Right. So there’s no enforcement mechanism in the Bazaar.

    Edith: Well, there’s no, um … Like, so with a company, if a company has said, “Okay, we’re gonna release twice a year and it’s gonna have this level of quality,” and they break that, you can walk with your feet or your money. If an open source project stops or does something that you don’t like, there’s very little enforcement end.

    Paul: Theoretically, you could walk. You know, you could fork the thing, but when you end up in an ecosystem that has massive network effects, and in the way that NPM does, you can’t walk with it. Because the pends is all transitive. The left-pad was, you know, a turtle way at the bottom of the stack.

    Edith: It was almost like a turtle hatchling.

    Paul: So, to fork left-pad, you would have to fork the entire NPM ecosystem.

    Edith: Yeah, and it’s not even. The fork is not the issue, though. The issue was that people were depending on it being in a certain place in a certain way. It wasn’t that it wasn’t forked, it was just like this entire structure at the bottom was gone.

    Paul: I guess what I’m saying is that if you get pissed off with a certain set of developers or a certain set of practices within the NPM community, because they’re so interlinked, you don’t have the traditional open source option of forking.

    Edith: Absolutely. Absolutely.

    Paul: You could rename it. You could pick a different library, but because of the way—I’m not sure if they still do this, ’cause I read something about it, but in NPM, it used to be that the version of the software that your library took was the version that library got. Do you know what I mean by that, okay. So let’s say there’s some software, let’s call it left-pad.

    Edith: Let’s call it right-pad.

    Paul: Right-pad. So, right-pad version one is being used by library A. And right-pad version two is being used by library B. In the Java ecosystem, and many, many other ecosystems, only one version of right-pad would be used.

    And I believe in NPM, both versions of right-pad would be used. So library A will use version one, and library B will use version two. Now that’s definitely the way that it used to be, I’m not sure if they’ve actually changed that.

    Edith: Yeah, I don’t know, but that sounds very messy, very quickly.

    Paul: So both options are terrible. And there isn’t a right way to do it.

    So, really, what it depends on. So, the NPM way of doing it, every library get its own version of the dependencies, is fantastic. Because it lets that library do what it actually intended to do under the covers.

    If, however, you start having an object that gets sent around, or that two modules are, let’s say it receives a, I think, an HTTP packet or something like that would be a good example. Let’s say you take an HTTP packet from one module and just hand it to another module, and they have different understandings of the shape of that object, or what functions you can run on that object, then that’s messy.

    But, on the other hand, if you override the version of right-pad that some other library’s using, you end up with all sorts of problems. This is a major problem in the, sort of the Maven ecosystem. And at CircleCI, we used Clojure, and Clojure is built on top of the Maven ecosystem, and we would have bugs where the library that we used.

    One library that we used had a dependency, and that was in the package file for another thing. And so, that’s the version of the packets that we used. So we would have to pick. And as a result of that, we have to pick the library that works. And, often, there isn’t a library that works for both, or there isn’t a, sort of subdependency that worked for both. So those are close to forked no matter which way you do it.

    Edith: Yeah, I mean, we ran into the same issues with our STKs. I mean, ’cause we have dependencies on other libraries that sometimes conflict.

    Paul: Right, disaster.

    Edith: On the other hand, I mean, the bigger point, I think, was. So there was an interesting backlash, I think, that everybody kinda projected what they wanted on the NPM thing.

    People said, Oh, people don’t know how to program anymore. You know, why do you even use what they called left-pad? You know, why don’t you just grind your own wheat and make your own bread from scratch and eat it at lunch also?

    Paul: I have no sympathy at all for this view of the world.

    Edith: Uh, that you should make everything from scratch? Or—

    Paul: That, “Oh, we’ve forgotten how to program because we need left-pad.”

    Edith: Yeah. The fact that we have modules available that make it easier and easier and easier to program is a good thing.

    Paul: I think it’s kinda different to that. I think it’s more of a curmudgeon view of the world. It’s a get-off-my-lawn—

    Edith: Wait, that you’re the curmudgeon?

    Paul: No, the people who complain, “Have we forgotten to program anymore?” I think it’s a no true Scotsman kind of argument. Real programmers know how to write their own left-pad, real programmers write in statically typed languages, real programmers write in C, real programmers, and I’m going back in time here, real programmers write in Assembly, real programmers hand-code their. Tweak the needles on the.

    Edith: They do their own garbage collection.

    Paul: Oh, and it’s garbage collection, what is this? Garbage collection is for amateur coders, right? I mean, you could just as easily have written, have we forgotten how to manually allocate memory?

    Edith: Or, you might as well say, like, nobody knows how to make their own transistors anymore. I mean, like, the entire computing revolution has been that you no longer have to assemble a computer by hand.

    Paul: Right. And each layer of the ecosystem allows a vastly larger number of people to take part in it. So I think the thing that we’re seeing now. This has been happening for five or 10 years, but what’s getting really, really big now is people who only know how to front-type program.

    People who only know Javascript and HTML/CSS, and they go to boot camps and they come out, they maybe have some, like, vague real understanding.

    And they call themselves coders, and we, the real programmers, who know how to write a quick sort, and who know how to write left-pad, we don’t welcome them to our community. And frankly, I mean, I have felt this in the past, I have since changed my mind, but it’s a very, very common perspective.

    This podcast is brought to you by Heavybit, a program that helps startups turn products loved by developers into successful businesses. Throughout the program, founders work on developer traction, product market fit and customer development. If you like what you hear, be sure to subscribe to Heavybit’s Podcast Network on iTunes.

  • To Be Continuous Podcast: Staging Servers Must Die

    To Be Continuous Podcast: Staging Servers Must Die

    Welcome to To Be Continuous,  a show about continuous delivery and software development hosted by Paul Biggar, founder of CircleCI, and Edith Harbaugh, CEO of LaunchDarkly. In this episode, Edith and Paul discuss a blog post by Edith, in which she asserts that you should kill your staging servers so that continuous delivery can live.

    This podcast is brought to you by Heavybit, a program that helps developer-focused companies take their product to market.

    [soundcloud url=”https://api.soundcloud.com/tracks/248544923?secret_token=s-fUn5T” params=”auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&visual=false” width=”100%” height=”166″ iframe=”true” /]

     

    Paul: So today we’re gonna talk about a post that Edith wrote. The post is about the death of staging servers. Can you give us a little bit of an introduction to what the post is, because I think we’re gonna talk about it quite a bit.

    Edith: Sure Paul, happy to. By the way, did you read it yourself?

    Paul: I confess, I may have skimmed it, a little more than read it.

    Edith: Yeah, so I’ll give you the summary. And then we’ll talk about it. And then you should go read it, and then we’ll talk about it again.

    So the article’s an outgrowth from what I was seeing with LaunchDarkly, my company. So already one of the comments I saw on it is, and I like this comment because they talked about me in the third person. They said, “Harbaugh is biased.”

    Paul: Right, I would say so.

    Edith: But by the way, she does have a good point. So I’ll state upfront, yes, I’m very biased. My company makes a platform to manage feature flags. So my bias, though, is I’ve seen what our customers are doing.

    And it’s the reason I wrote the article, is that this was a national extension to the way I saw our customers using feature flagging.

    Paul: So were your customers using your product wrong, or was there a thing that was happening, that you’re talking about?

    Edith: So feature flagging at it’s most basic is very simple. It’s just an if-then statement. You don’t need LaunchDarkly or any sort of system to manage that, you can just put a conditional in your code. What happens after that is you get more sophisticated. And you wanna have some sort of dashboard where you could see the different feature flags that are visible and uplevel them. Not even just to the business side, but even amongst the various developers.

    You want to have a central place where you can manage. That was kind of our first version, is just you can manage feature flags.

    The next thing that our customers were asking for in our road map was to support for environments. They wanted to be able to have feature flags on dev, QA, staging production, have visibility all in one place.

    Paul: I’m a little confused when you say this, because surely an environment is just a flag?

    Edith: Yes, go on.

    Paul: It seems like you could say, have another flag that is a string, that is like staging or production or Dev or whatever. That you flag features on.

    Edith: And that’s basically what LaunchDarkly is doing, so for us, we just have another flag.

    So each environment gets an API key.

    Paul: So again, forgive ignorance on this, why not use the existing flag?

    What was missing from the existing feature flag infrastructure that meant environments had to be a separate top level, or first class feature?

    Edith: So now we’re getting a little meta. The reason why people liked using LaunchDarkly is because they can get a consolidated view of all their feature flags.

    So that they could not just have these floating around fig file for each machine, but have a roll up of what flags are turned on and off for different environments. And then on top of that the ability to manage the flags in different environments.

    Paul: Got you, so they wanted a view that applied to just their environments?

    Edith: Well, to see per environment what was turned on and off. So basically they were following a software life cycle, where you had dev people working on the developer boxes, pushing a QA, pushing a staging, and then pushing to productions.

    Paul: When features moved from one to the other, was there a human promoting these, or was there an API call promoting these?

    Edith: At this point it’s mainly human. I could see down the line that it would just be an API. And even beyond that … The next thing people ask for, once they got more people on board LaunchDarkly, we’re using more feature flags, is they wanted the ability to have lockdown of different flags in different environments.

    Paul: Lockdown like people can’t change the flags?

    Edith: So they wanted QA to have rights, and QA to change some flags, but not in production.

    Paul: I see, okay yeah.

    Edith: Then after that the natural step then is okay, if you have a feature flag in system where you can control any step, who sees permissions?

    Who gets to see what? Why do you really have a separate QA staging and production box? Why not just collapse all this, and manage visibility with a feature flag itself?

    Paul: Sure. That makes perfect sense.

    All the feature flags provide the primitives on which you can build the things that they want.

    Edith: The point I made in the article is that

    people use the substraction of a QA and a staging environment to basically try to encapsulate a change. So that was the original intent, and at the time it was very good, because the alternative was just to push everything to production to have everything break.

    But if you’re actually doing—

    Paul: If you’re doing feature flags properly, then the concept of really having a staging server doesn’t make that much sense.

    Because you’re not pushing a feature to the staging server, you’re pushing a feature to production and then slow rolling it to people and really having it in the staging server doesn’t make that much sense. Is that kind of the point?

    Edith: Yeah, so that was the title of the article, Kill the Staging Server.

    Paul: This might be a good point to actually stop and read the article.

    Edith: Yeah, I’d appreciate that. Thanks, Paul.

    Paul: We’re gonna take a, what’s gonna appear to be a 10-second break, but it’s actually going to be a five-minute break, while I read the article. I recommend you press pause and read the article yourselves right now.

    Edith:  So welcome back, Paul has been furiously whiteboarding. I’d love to hear his thoughts now.

    Paul: Yeah, so it’s a good article, I really liked it. The first thing that came to mind was the concept of names.

    So the idea of having multiple environments is a weird one in particular. So imagine that you have … Service rented architecture, and it’s got like six machines or something like that, or it’s got 20 machines or whatever. The idea that you can’t spin up a new instance of it is kind of a little bit odd.

    Edith: Yeah.

    Paul: So what people have done historically, and this goes back to when things ran on machines, or particular ports, or whatever, was that everything had a name. It had DNS name or had a staging name.

    And if you look in rails, configuration files, there’s a production name, there’s a staging name, there’s a dev name, there’s different environment variables for all of these. But it doesn’t actually make sense to have a name. Because a name implies there is one of them.

    Edith: Well this goes back, are you going down the whole pets vs. cattle thing?

    Paul: I’m going down the pets vs. cattle thing, yeah. If you’re naming your pets, then you can’t just suddenly get six of them, you can’t suddenly kill them. But what you really want is cattle.

    So a staging server is not, I mean a staging server is a pet. And a very very important pet that everybody loves, and everyone plays with, and it gets a little bit confused as a result of it. And that analogy went—

    Edith: I’m not sure if you’re agreeing with me, or disagreeing with me, so. We’ll continue, it’d be great if you disagreed, but I would also enjoy it if you agreed.

    Paul: If that analogy holds, then what you’re suggesting is that we killed the cherished family pet.

    Edith: Well, you know, sometimes … aw, I can’t even go there.

    Paul: Well the analogy breaks down, but let’s say it’s for the best if that pet spends some time on the farm.

    Edith: Went for a long walk.

    Paul: Right. So obviously production is an actual thing that needs a name, it is a unique environment. But nothing else is really unique environment.

    Edith: Yeah, and to fast forward, I think, I got a lot of feedback on the article, which I loved. That’s kind of why I wrote it, for feedback. I think there are cases where you do want to not go directly from a developer to production.

    I think there are many cases where you want to have other places to test them.

    Paul: I do agree with this, yes.

    Edith: I think a lot of the cases though, of a forced march of we go from this step to this step to this step to production is actually very harmful, when if you just pushed off much quicker and perhaps skipped some of these steps, you’d get the feedback you want directly.

    Paul: So what we did at CircleCI, and initially we had a staging environment. And we would occasionally use the staging environment if we want to test something that wasn’t that easy to write unitests for, or something along those lines. And usually for us that was stuff around LXC, or stuff around starting Amazon boxes.

    Things that you didn’t really do that often, and that either had an expense, or had a weird architectural thing, where you couldn’t just do it in software, and practice it in software. So I think there is that need. Occasionally you’ll have things where it’s not tested well enough, so you need to put it up somewhere where a human validates to the best of their knowledge that it actually works.

    And you can put that in your, I was gonna say you could run it on, but the whole point of the staging server in that situation is that it’s something which isn’t really that easy to put into VMs, or whatever. The other kind of use case that you see is where you don’t want to be running stuff on live production databases, and you can’t get a copy too quickly. So you see people like Haroku trying to build products that avoid the need for that.

    So with Haroku … You can take a copy of a database, you can have a view or a read only view on a database, that’s a copy on the right in some way or whatever. So that you don’t need a separate staging environment, or separate copy or separate staging database or whatever.

    But if everything is well-tested, if everything works in software, then there’s really no need for staging environments. So that says to me that staging environments is a sort of a yellow flag somewhere. It shouldn’t really exist, but sometimes you might need it.

    Edith: I actually want to write a follow up article now, of the staging server is dead, long live the staging server. Because I do think that there are used cases where you don’t want to push to prod. I do however think people—

    Paul: What are these cases where you don’t want to push to prod?

    Edith: So what I heard from Sean Burns, our advisor, is if you’re testing a really deep infrastructure change, for example switching some batch processing.

    Paul: Mm, no, I don’t believe that for a second.

    Edith: Alright.

    Paul: So if you’re testing a really deep infrastructural change, there’s two ways to do that. That deep infrastructural change. One of them is to say, “We’re gonna have these machines over here, we’re gonna have these machines over here.” And that’s the quarterly release cycle thing. We’re taking a big big risk, all hands on deck, whatever.

    The way that you want to be releasing that sort of thing is that you wanna have it in the code base. You wanna have an if statement, a feature flag, that controls how much of the data goes one way or the other, and you duplicate the data, or you put 1 percent of it through.

    Edith: I think the key there is to duplicate or have some fail safe. I mean the biggest risk you run with doing this is data loss, which is awful. If you’re cavalier about how you do this—

    Paul: Well, I don’t think data loss is the worst thing, the worst thing is your whole damn service goes down.

    Edith: Well, the worst thing is your whole service goes down, and you lose a case worth of somebody’s data.

    I think people are actually more forgiving of a five-minute flip than you losing like a lot of their analytics. So his point was he had been at Flurry, was that we cannot afford to lose people’s data.

    Paul: So it would be ludicrous then, in my mind, to create a whole brand new infrastructural thing where you’re gonna do some kind of overnight or immediate change, or wait for downtime change or something. If the data is all live, you can’t afford to have, we’re gonna switch over and see if it works. Regardless of how well it’s tested, like it’s ludicrous.

    Edith: Yeah, so that was, now I’m coming back around to my article. I mean that was the point I was making.

    People think that they are reducing risks by doing all this tested staging.

    Paul: Yeah, in fact they’re increasing risk.

    Edith: Yeah, you would think you’re—

    Paul: Well, the whole point of continuous delivery is that by having a harsh cross over between one thing and another thing, you increase risk even though you think you’re decreasing risk.

    Edith: Yeah, so Ket Beck wrote a really good article, about reversibility. So he’s at Facebook now, but he said everything at Facebook is reversible.

    Paul: Interesting.

    Edith: That this actually makes you much less risky, ’cause you’re like okay we make these risky changes—

    Paul: But they’re reversible, so it’s, right.

    Edith: Yeah, vs. the more cutover. As I called in the article waterfall deployments.

    Paul: I mean I think that’s a really good way of thinking about it. It ties it to a name that everybody knows is bad.

    Edith: Well it was a deliberate—

    Paul: Yeah, good choice. So the agile deployments then, are the ones where it happens seamlessly, and you can go back and forth and change the requirements and whatever else.

    Edith: So that was Sean’s example No. 1, was what if it’s a really risky back and change? He had some good examples from his own career.

    Paul: Did he have any examples that I would agree with?

    Edith: Well a priori without hearing them, I don’t know if you’re going to agree or disagree. Another thing people brought up, was it just seems very risky to people. It seems to increase risk because I think they’re used to thinking of staging server as a safe harbor.

    Paul: I think there’s some semantics around the use of staging server. You very often want to have a complete copy of your environment that you can test against. So is that a staging server, or is that where you type docker up, you know new environment.

    And then you run your testing on this whole brand new infrastructure that has never been touched by anything before?

    Edith: Yeah, and the critique I made in the article, is I think people do that a lot. And they spend a lot of energy testing there, but these are artificial test pieces.

    Thanks for listening to this episode of To Be Continuous, brought to you by Heavybit.

    You can find the full episode of this podcast here. For more on Continuous Delivery and to hear other Heavybit podcasts, copy this link into your favorite podcast player. We’ll have show-specific RSS and iTunes subscription options soon. You can also follow To Be Continuous on Twitter at @continuouscast.

  • To Be Continuous Podcast: The Clash in Open Source

    To Be Continuous Podcast: The Clash in Open Source

    Welcome to To Be Continuous, a show about Continuous Delivery and software development hosted by Paul Biggar, Founder of CircleCI, and Edith Harbaugh, CEO of LaunchDarkly. In this episode, Edith and Paul talk about the clash in OSS between developers looking to get paid and software companies looking for things for free.

    This podcast is brought to you by Heavybit, a program that helps developer-focused companies take their product to market.

    [soundcloud url=”https://api.soundcloud.com/tracks/244279037?secret_token=s-oeBCP” params=”auto_play=false&hide_related=true&show_comments=true&show_user=true&show_reposts=false&visual=false” width=”100%” height=’166′ iframe=”true” /]

    Edith: So Paul, what do you like about open source?

    Paul: So I guess that depends on whether we’re talking about writing open source or using open sources as part of a company. Writing open source, I like just having all my code out there.

    And I’ll do that as often as possible. Anytime I start a little bit of code, a little library, I’m looking to open-source it and have a readme and have it out there for other people to use. I had a very long discussion, which I eventually won, about the …

    Edith: I like the way you think of all discussions as win or loss.

    Paul: Well, yeah, yeah. It was more on the argumentative side of discussion. But with the other co-authors of the PHP compiler I wrote, which was originally GPL. And I argued very strongly that I actually don’t care if people give back, I just want people using my software. So to me, open source is a way of having people using my software.

    Edith: That’s really interesting. So, I liked what you said at the beginning about, like, you liked open source ’cause it meant getting stuff out sooner. ‘Cause I’m a big fan of that.

    Like, I always say you should just publish stuff. Like, my co-founder John said something really good at our team meeting last week. He’s like, if it’s 90 percent done and you don’t publish it, it’s actually 0 percent done.

    Paul: It’s 0 percent done, yeah.

    Edith: Yeah, and that’s a hard lesson, ’cause you always try for perfection lately.

    Paul: Right, right, right. Well, I think there’s also some really hard downsides about that. So when you publish something before it’s ready, before you really —

    Edith: Well, ready.

    Paul: Well, there’s ready ready, but let’s say ready is, let’s say you publish it before you clarify what the software is to do, or the values or the mission of the software.

    And then you get contributors who are saying, ‘Oh, this is great, but I really want it to do this other thing,’ and you’re like, ‘Well, that’s not exactly what I was going for here.’

    Edith: Well, I’d say most open source projects are so lucky as to get contributors at all.

    Paul: True, true, true. So I’ll give you an example. I wrote the V8 bindings for Closure. So a binding that allows you to write JavaScript and run it in Closure. And I did that for a purely selfish thing, for another, for a nascent pipeline that I was building called Stefan.

    And then people came along and they wanted to do other things with it. They’re like, ‘Okay, let’s add a binding for this other thing.’ I was like—

    Edith: Well, that’s kind of like a pivot.

    Paul: It is, it is, but I really was not interested in it.

    I’d rather someone took it over, took over the project and actually ran with it where they wanted to go, and I can stick on the old version, or I can see if the new version solves it for me, but I wasn’t interested in becoming a maintainer in this thing, I was interested in solving my itch.

    And I just happened to open-source it, because I felt it might be useful for other people.

    Edith: So Paul, I hate to say this, but I agree with you.

    Paul: No, not again.

    Edith: Not again.  No, I mean, I think what you said is exactly right, that you said, I did this for this reason, and I want somebody else to take it over now.

    I think where open source runs into issues where people are like, it’s only this purpose and you can’t do anything with it besides my original thing.

    Paul: So you think that’s a good thing, or you think that’s a problem? Or it causes problems in the community?

    Edith: I think open source is so misused for so many different purposes that it’s hard to even say. Like, so, in Europe, people say open source is good because it means free software.

    Paul: Libre or gratuit?

    Edith: I’m sorry?

    Paul: Free as in beer or free as in, what’s the other side of that?

    Edith: There’s no such thing as a free lunch, is that what you’re talking about?

    Paul: No no no, like people say open source software, it’s like free-as-in-beer software, as in software that has no cost vs. software that has no price tag vs. software that is, they use the French word libre for—

    Edith: Oh yeah, that’s what I was talking about. So I remember I went to Europe in, I think it was like 2004 or 2005, and they just had this big push for open source, ’cause they thought it meant free.

    Paul: Oh, this is like the German government and that sort of thing.

    Edith:  So literally like, so I went to this customer and they’re like, ‘So it doesn’t cost you any more to give us an extra copy.’ So why shouldn’t just everything be free?

    Paul: Oh, right, yeah, well.

    Edith: And so I think open source has been misused by many businesses who think of it as just like, hey, we get all these developers working for free.

    Paul: So I think that the main advantage to business of open source is not that it’s free, but it’s that it’s low-risk.

    Edith: Low-risk?

    Paul: Lower risk. So it removes—

    Edith: Ah, I would argue—

    Paul: Well, so, what I mean is that it removes a certain element of risk from it. So there’s, Joss Belsky wrote a blog post about how you shouldn’t use, I think it was a database that you didn’t have the source code to.

    Edith: Well, so I’ll argue the reverse, and I think open source is sometimes more risky. ‘Cause I talk to people now who are like, nobody’s maintained this open source project that I depend on.

    Paul: Right, right, right. But that vs. a closed-source version.

    Edith: Well, so the opposite risk is that you end up that you are responsible for this open source project, and you’re like, well, actually, I just wanted to use this thing. I didn’t want to do the maintaining.

    Paul: So absolutely, you don’t wanna end up the only person that, well, it’s actually fine to end up the only person which is using this thing.

    Edith: Well, no, it’s not—

    Paul: It’s not ideal.

    Edith: It’s that many people are using it, but nobody is paying money, and then you’re stuck.

    Paul: Because no one is maintaining it.

    Edith: No one is maintaining it, and you depend on it.

    Paul: Right, so, but imagine that no one maintained it and all that you could get was like a binary block. That’s the real problem. The company goes out of business. You have random database version seven, the binary, that only runs on 32-bit, that you’re maintaining, like, companies are literally maintaining COBOL—

    Fortran things from 30 years ago. It sits in a machine, you can’t touch it, you can’t fix it, you can’t apply security updates to it.

    Edith: Yeah, that’s fair. So Tim Chou, who is, he’s an awesome guy, he was early Oracle, he said that people don’t buy, when they buy software, they’re not buying it what it is today.

    Paul: Right.

    Edith: ‘Cause basically, anything that you’re buying today, you could just hire 20 engineers that could build it. What they’re buying—

    Paul: Yeah, you could just copy it.

    Edith: Yeah, what they’re buying is the promise of stability, that it will continue to be maintained. That there is a vision. And that’s actually, I went over to a big customer who won’t let us use their name yet, but they said they love LaunchDarkly because we thought every day about feature flags.

    Paul: Right, right, exactly. This is why people buy SAS, because, because people think all day about CI, about QA as a service, about queues, or whatever it is from various Heavybit companies that I just shilled for, they’re, you can probably figure out yourself which ones they are by looking at the homepage.

    Edith: Well, and Paul, by the way, thank you very much for wearing your LaunchDarkly t-shirt today. It looks very handsome.

    Paul: The, I definitely agree that the benefit of the thing is that you use it every day. But you can do that with open source as well. You can have open source software that, so, for example, our front end is open-sourced.

    Edith: Oh, I’m absolutely a fan of open source. Like, we use open source every day. Our client libraries are open source. I’m not saying that it’s, I don’t think it’s a religious thing vs. open source vs. not, I just think that many times people assume open source is something that it’s not.

    Paul: Yeah, fair enough, fair enough.

    Edith: I mean that if you have open source, that just means that developers spring out of the ground willing and happy to update your software every day.

    Paul: A mixed blessing, definitely. If it even happens.

    Edith: Like, I remember, so I remember when Sun, so I used to work on a, I used to be at Epicentric, and we had a portal server, and we were killing Sun. Absolutely killing Sun. And this was in 2003, so what they said is, ‘Oh, we’re gonna make our portal server open source.’

    Paul: Right.

    Edith: And so somehow this whole thing, we’re like, everything will be open source, and like, developers will spring out of the ground, and it’s like, well, no, developers, they don’t wake up every day like, ‘I’m gonna work on your thing.’

    Paul: Right, right. They will if it’s sexy or exciting. There’s lots of developers who are showing up at Mozilla just to contribute. And in fact, they’ve fought through incredibly large hurdles to contribute to Mozilla’s software.

    Edith: Well, that’s fascinating, ’cause I see so often the reverse, where people, so what do you think about Mozilla made people want to contribute?

    Paul: I think the fact that it was an open-source-first company, and that it was a nonprofit, and it was all about the free and the open web and that sort of thing. Also, people had had very good experiences with Firefox for, like, a decade at this point by the time I was there. Well, maybe seven years, but, you know.

    Edith: It’s fascinating, ’cause I went to dinner last night with the CEO of MySQL, and I don’t know if I should say this, but he said—

    Paul: This is Marten Mickos?

    Edith: Yep. So he gave a talk—

    Paul: He has a new startup?

    Edith: Called HackerOne?

    Paul: Oh, he joined HackerOne.

    Edith: Yeah, he’s the new CEO. So the dinner was kind of like a debutante ball for him.

    Paul: Nice, nice.

    Edith: So what he said was that somebody had asked him flat out how much of MySQL was actually from contributors. He said 1 percent.

    Paul: Seems about right.

    Edith: He said the open source was more just a branding thing. Like it wasn’t actually like a productivity thing. It was more just a branding thing.

    Paul: So you have this in Mozilla as well, that any time that there’s a really avid contributor, that Mozilla would hire them.

    Edith: Yeah, that’s what he said.

    Paul: Right, and if Mozilla didn’t hire them, very often they stopped being a contributor. Like, it was a real sort of a double-edged sword.

    So someone who did it for six months, they were doing it for the love of the game. They weren’t doing it to be hired at, at least I think for the most part, they weren’t doing it to be hired at Mozilla. And then six months in, someone’s like, well, you know, you’re doing a really great job, wouldn’t you love to work on this full-time? They’re like, yeah, that’s so exciting. They’re flown out for an interview. And then it’s like, you know, actually, I don’t think so.

    Edith: Wait, they didn’t wanna work for Mozilla, or Mozilla didn’t wanna hire them?

    Paul: Mozilla didn’t wanna hire them. I mean, this isn’t a particular case. But you know, some subset of people, Mozilla didn’t wanna hire. And then what happens? Well, you kind of don’t wanna keep contributing to the company that just told you to f*ck off.

    Edith: Yeah, I mean, it’s so true. I mean, I think there’s a very human thing about volunteering.

    Paul: Right, right.

    Edith: I volunteer, I was the moderator for a long time for the Lean Startup Circle, and somebody new came on, and they’re like, yeah, I’d love to raise some money so that we can get paid, and I’m like, they cannot afford to pay me.

    Like, I don’t do this to get paid, ’cause—

    Paul: Yeah, it’s like, it’s an altruism that people are trying to do.

    Edith: Yeah, I was contributing like 10 to 15 hours a week, and at my going rate, it’s not, it’s like, I’m not doing this to get paid, I’m doing it ’cause I like giving back.

    Paul: So there’s this thing that we were talking about earlier, about the current discussion around people getting paid in open source.

    Edith: Yeah, and I think a lot of what has fueled open source is the whole 20 percent time. Like, that Google will give their engineers 20 percent for open source, and I know that other big companies, like Twitter and so on, do that too. So these people are, in essence, getting paid.

    Paul: Right, right, there’s a, in most companies, in most startups at least, there’s an unspoken rule that you’re expected to contribute to the libraries that are in your application, and that sort of thing. If you’re spending time upstreaming paths that you’ve deployed, that’s work.

    Edith: Yeah, so I think we’re agreeing, which, I thought we agreed that we were not gonna agree. It’s funny, ’cause I’m Twitter friends with a guy I’ve never met, who’s, he started Jango, and right now what he tweets a lot about is—

    Paul: This is Jacob Kaplan-Moss?

    Edith: Yeah.

    Paul: Okay.

    Edith: That people are working their nights and weekends, and contributing to open source, and then just, people are kind of, for lack of a better word, jerks. People are like—

    Paul: People are jerks.

    Edith: Yeah, like—

    Paul: Even when you’re paid, people are jerks.

    Edith: But it’s more aggravating.

    Paul: Right, right, because you’re doing it for free, or whatever.

    Edith: Yeah, and so people will, like, they’ll be like, ‘I have this pull request and it’s been open for two hours. Why haven’t you reviewed it?’

    Paul: Right, right. I mean, we get the exact same thing. We get customers who email us, or more often it’s the people who get, who are using the free version, who send us a support request and then, 30 minutes later they’re on Twitter going, I can’t believe Circle hasn’t solved my problem yet. It’s been 30 minutes.

    Edith: What do you think drives that?

    Paul: There’s a pattern across the population in general, that people are jerks.

    Thanks for listening to this episode of To Be Continuous, brought to you by Heavybit and hosted by me, Paul Biggar of Circle CI, and Edith Harvaugh of LaunchDarkly.

    You can find the full episode of this podcast here. For more on Continuous Delivery and to hear other Heavybit podcasts, copy this link into your favorite podcast player. We’ll have show-specific RSS and iTunes subscription options soon. You can also follow To Be Continuous on Twitter at @continuouscast.

  • Don’t Make Me Code Podcast: A New Heavybit Series on Developer Experience

    Don’t Make Me Code Podcast: A New Heavybit Series on Developer Experience

    Welcome to Don’t Make Me Code, a new podcast that discusses developer experience design and the unique challenges you face when building developer facing products. Don’t Make Me Code is hosted by Steve Boak, co-founder and CPO at Opsee, and David Dollar, co-founder and CEO at Convox.

    This podcast is brought to you by Heavybit, a program that helps developer-focused companies take their product to market.

    In this first episode, Steve and David discuss the origins of the term “DX” (developer experience), they share their thoughts on what makes a great developer experience, and they talk about some of the challenges they face when designing developer-facing products.

    [soundcloud url=”https://api.soundcloud.com/tracks/242787724?secret_token=s-09pht” params=”auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&visual=false” width=”100%” height=”166″ iframe=”true” /]

    David: Hi, I’m David Dollar. I’m CEO and co-founder of Convox, a Heavybit company. I’ve been working in developer experience for about the last three or four years now.

    I’ve founded the DX team at Heroku, worked on a whole bunch of open source tools including Foreman, and now working on an open source platform as a service.

    Steve: I’m Steve Boak, the co-founder and head of product at Opsee. We are a monitoring company, and I’ve been working at several monitoring companies over the past six years, helping to help developers improve the experience of their products as well as find bugs in their code and improve the quality of their products.

    And though I didn’t hear the term DX for the first time until a few months ago, it’s been a passion of mine for several years, and it’s nice to finally attach a name to it. And I’m excited about this first episode because we get to unpack the term a little bit and figure out what it means.

    I’ve thought one of the most interesting parts of your background, David, was that you were part of this founding team at Heroku, that sort of created DX, and I’m interested to hear more about that.

    David: Yeah, so I guess this was maybe late 2012, maybe early 2013, right as at Heroku we started to expand, out of being just a Ruby host into being a host for really any language. That meant we also started to see more, you know our standard user before that had been sort of like, and I was ten user building rails and was very, there was sort of one happy path for that.

    As we expanded languages we had Windows users and Linux users and sort of, everybody coming together to try and build really any thing. And that started to get out of hand really fast. So, really, we formed the DX team, brought together basically the people that were in charge of the various languages at Heroku. Somebody in charge of the Ruby, and somebody in charge of Python and Node. Brought us all together into one team and really decided to try and make a consistent experience, sort of the Heroku way, across all of these languages and …

    Steve: And was it called the Developer Experience team from the beginning?

    David: It was, yeah.

    So the first time I heard the term was from Adam Wiggins. He sort of like came to me and said, “Hey, I think we should build this DX team. What do you think?” And yeah, so I mean it sort of exactly described what it is we were trying to do, which was make this nice experience for people building things on Heroku.

    Steve: And when was that? What year was that?

    David: I think it was the end of 2012, but it’s kinda fuzzy at this point. But yeah, it was a while ago.

    Steve: And it does seem to continue to be an important thing for Heroku. I, just in researching the term of it, saw that there’s a Heroku.com/dx, this is a big part of how they market themselves.

    And clearly it’s been important to Heroku for a long time. Like I said, it’s interesting to now talk about this with a name because a lot of us are doing this and have been doing it for a while but, giving it a name gives it so much more weight and it kind of marks how important it is for all of us. Mmm-hmm.

    David: It’s interest … you think of DX and what does that actually even mean? And its, there’s a certain type of user that you’re talking about now, right? When you’re talking about designing an experience for this user that likes to kind of get under the hood and break things apart and do things their own way and sort of knows what’s going on and maybe you could even consider a power user.

    Yeah, you want to make sure that they can sort of do whatever it is they’re trying to do but that the options aren’t overwhelming and that it’s not complete, just utter chaos.

    Steve: Yeah, it’s almost like this partnership that we as the makers of tools that developers then use to build tools for their customers, if they don’t like the product that they’re using that may show in the product that they’re ultimately trying to build. And so it’s this partnership where we all try to write good code and create great products together.

    And I think that’s part of what makes this different from user experience design as a whole, and why it’s interesting as a topic all unto itself.

    David: I definitely agree with that.

    The most rewarding thing about working, I guess in developer experience that I found is when people do things that you never expected them to do.

    You put all the pieces in place, you thought like, “Oh, maybe somebody will use these pieces to build X,” and it turns out that somebody comes and builds something you never even thought of. I think that that’s just really rewarding to see that.

    Steve: And it’s interesting that, I think a lot of product builders in general would probably share that sentiment in terms of the unexpected ways that people find to use their products, and the hacks that they create, and I’ve heard, even with consumer products you know, people being surprised by how the users are actually using the product in ways they never expected, but with developers that really is to another level because they may try to break your APIs.

    They’re much better at exploring the depths of your code and then creating tools that you never expected with it, which is both surprising and wonderful for us.

    David: It’s still largely about aesthetics, right? Like, you’re a good user experience, a good developer experience.

    At the end of the day is this thing easy to use? Is it pleasant to use? Are you able to figure out what’s going on? Does it make me happy to use the thing at the end of the day? So yeah, it takes a different set of things when somebody’s trying to come in and actually build something on top of your product.

    Steve: I remember in the early days of the Google Maps API just how many experiments and apps were being written with that.

    I think, partly, because of how easy and accessible they made it for developers. And I heard a talk from someone on that team talking about that the product that we all knew and loved then was not the first iteration of it and they actually had a much harder to use initial version. Do you know more about the history of that?

    David: I don’t really know.

    Steve: Yeah, but that was one of my first exposures to using another company’s API and it was a really rewarding experience. And they had all this boilerplate code that you could very easily lay down to plot pins on a map and see that.

    Especially for me as a designer it was so rewarding to get that gratifying output. The actual map on the screen with the pins that I had laid out. I think that is a really great way to measure how effective and how elegant your developer experience design is, in how quickly a developer can make something themselves.

    David: One of the barometers I use for the things that I’m building is you look for the blog-posts people are writing around your tools.

    Are they writing a blog post of like this thing was so insane I couldn’t figure out how to use it? Like here’s my giant blog post teaching you actually how to use it because I had to figure all this out. Or are they talking about how easy it made their life? Like, look how awesome this is, you should try it, too.

    Steve: Yeah, and that makes me think of the JavaScript world today and just how many frameworks are being launched, and the competing arguments for this or that. And in some ways this has really gotten good for the development community as a whole because there are so many choices now, and so many good frameworks to choose from. Almost too many, actually.

    And I think we’re starting to see some of that in the JavaScript world. Just competing ideas of how things should be done. And that almost raises a counter-argument to the value of developer experience design that when we open up too many possibilities and don’t create a set of tools, I think you described it in your Heavybit talks as like, sharply …

    David: I like to compose sharp tools. I mean it’s sort of like the Unix philosophy, right? Like, you want to make tiny tools that do one thing really well. You generally want to have some good examples or defaults, like, this is how to use these tools in combinations to make interesting things. But having those tools be sort of separate and composable is where you start to get really interesting things happening that you didn’t expect.

    Steve:

    Yeah, that each piece does its thing really well but also that when combined they can form something greater and that the whole is greater than the sum of the parts.

    David: Right. Like, a good example of that, it’s not really developer experience, but I have TTT, it’s sort of like an infinite variation of all these little components that you can put together, but they also have some default pipelines that come out of the box, you say, like, “Okay I really just want to make my email go to Google drive or something like, just like these nice defaults.”

    It’s really important, I think, when you have all of these choices to really also have the guides through them.

    Steve: Yeah. We use the Amazon APIs a lot now, and they’re an ecosystem all unto themselves. For every product Amazon offers inside of AWS there’s a unique API and many of those APIs seem to have differing views of the world and so, they in some sense they’re prolific in the amount of tooling they create for developers and they clearly have won a huge share of the market with just how much they offer, but it still feels to me like they’re a counter example.

    They seem to have APIs that a lot of us use but not APIs and tools that a lot of developers enjoy using.

    David: Even despite that, like pretty much everybody is still using their Sphinx, right?

    Steve: Yeah.

    David: So there’s something to that.

    Steve: You know, Facebook’s API or Twitter’s API I can’t speak to how good or bad those are but man, a lot of people use them. And there’s something to that, too, that the product as a whole has so much value and impact on the world that you are sort of forced into a place where you need to use it.

    And it doesn’t absolve you from the responsibility to make your tools great for your developers and your partners in this, but yeah, it’s sort of mind blowing to see how much can be done when you are hitting on a big share of the market.

    David: The AWS database is another great example of that. There’s so many options, so many ways to do things that you can, like, it’s great. You can really do anything. Like the sky’s the limit. But there is, like, finding the happy path can often times be, it’s just a lot of trial and error at the beginning.

    You end up just having to build up some expertise around that stuff and eventually you can, okay now I can understand how to design things using these tiny permanents, but yeah there’s not a really great, like I just have some code and I want to put it on the Internet.

    Steve: Yeah, and part of Convox’s business is really helping developers do this, right? That there are in fact so many possibilities to create an infrastructure in AWS, that giving a little guidance is probably a good way of helping developers through that process.

    David: Yeah it’s basically just trying to create a happy path. I think there’s a lot of choices that you don’t necessarily have to make.

    Steve: Yeah, we too, at Opsee, use AWS heavily. It’s a monitoring product designed just for Amazon web hosting. And when we were creating this product we were assuming we would see a pretty high level of consistency in how our customers are configuring their environments, but that was shot down almost immediately.

    There have been no two environments that are the same and so now we’re dealing with a wide variety of customer environments and it’s an example of just how much people can do with these tools, but how differently they’re choosing to do them.

    That though does get to this other interesting question of how we measure the quality of developer experience and how we know that we’re doing well. And we talked already a bit about time to value and how quickly developers can make tools with our products, but then I think it’s also interesting to talk about how that becomes part of how we price these tools. Like, we try to look at how much development time we can save our customers and then try to price the product based on that. Do you try to do something similar?

    David: We’re trying to think about this as a peer developer tool, so rather than, you know a lot of people try to charge for resources, things like that, we’re doing more of like charging per developer that’s using the thing. So we actually, actually creating value for developers I think is sort of the core of what we’re trying to do, so we’re approaching it from that angle.

    Steve: Yeah, and monitoring being a little bit different, we’re mostly trying to measure how much code our developers don’t have to write when they’re using this tool, and what they would have to do in a competing world, but

    I think the one thing that all Heavybit companies, certainly in all developer tools companies share is that at it’s core we’re trying to measure our value that way: How much code can we save developers from writing, and how much time can we save them?

    And that feels in many ways different from normal user experience design and consumer user experience design. That we have to look at that, the value that we’re creating for our users in order to assess our own value. So as designers of developer experience products I think it’s interesting to talk about the inspirations and the other companies that are doing this well.

    The first time I had ever really thought about developer experience as something to be measured was looking at Stripe and how … You said earlier how blog posts and recognition within the community can mean a lot. And I remember hearing lots of developers talk about the quality of Stripe’s documentation, to the point that it became synonymous with a great developer product.

    Like do you have Stripe-level docs? And so they were always a barometer for, you know, are we doing well and who do we measure ourselves against?

    And I feel like they’ve done a great job not just in creating great documentation and great APIs but in great products. I mean even their web design. They’ve taken all aspects of their design to a level that I think was not common in our world when they were doing it.

    David: Yeah definitely. I mean, we use Stripe and I just remember when I was first clicking through their web I was just like, “Wow, this is really nice.” You just have that thought to yourself while you’re using it, so yeah it’s very pleasant to use.

    Steve: And I don’t know, you know in the early days of Heroku I don’t know if you saw much of this but certainly five, six years ago if we designed a UI that was polished and really nice looking there was almost a bit of hesitation among our developer customers that we didn’t have the street cred like, “No, you can’t be good, you can’t be real because no one in our world would do this, would make a product like that.”

    And I think that’s changed a lot over the last few years, but we were almost fighting this uphill battle to try to create an elegant and beautiful user experience because people didn’t expect that.

    David: I certainly think that sentiment has changed. I think having a good design aesthetic, a sense of that, is actually like seen as a good thing in most developer circles now, as opposed to being like, maybe you don’t know what you’re talking about if you have a sense of style or something.

    Yeah, it seems to be important. People really look for it when they’re looking for tools or products and that’s also something that we tend to see celebrated.

    Steve: That’s about all we have time for today.

    If you’re interested in being a guest on the show, or if you have a DX topic you’d like us to dive into you can reach us at dmmc@heavybit.com or on Twitter @dontmakemecode. To learn more about Heavybit visit heavybit.com. And while you’re there, check out the library. It’s packed with amazing talks from founders of developer tools companies and other industry leaders.

  • To Be Continuous: Empathy in Software Development

    To Be Continuous: Empathy in Software Development

    Thanks for listening to To Be Continuous, a show about Continuous Delivery and software development hosted by Paul Biggar, Founder of CircleCI, and Edith Harbaugh, CEO of LaunchDarkly. In this segment of To Be Continuous, Edith Harbaugh and Paul Biggar talk with Sam Stokes about empathy in software development, how mini-services are the new microservices, and constantly updating software in a constantly updating world. Sam is currently a freelance software consultant and previously was the co-founder of Rapportive.

    This podcast is brought to you by Heavybit, a program that helps developer-focused companies take their product to market.

    [soundcloud url=”https://api.soundcloud.com/tracks/237727552?secret_token=s-k6OKe” params=”auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&visual=flase” width=”100%” height=’166′ iframe=”true” /]

    Paul: Alright, Sam, so what’s your favorite thing about continuous delivery?

    Sam: I like to think about it terms of shortening feedback loops. A lot of things that startups do can basically be looked at as doing something and then waiting to get some feedback on it, and then deciding whether to do more of that thing or to do a different thing.

    And continuous delivery is, people talk about sort of reducing the batch size and also about shipping stuff faster and clearly having some kind of automation. And continuous delivery is going to help you with both of those things.

    Paul: So is this specifically on the, sort of on the journey of a small startup towards product market fit? Is that kind of the sense you’re thinking of it as?

    Sam: I see it as applying to teams of different sizes. So for sure in a small startup you’ve got you need to iterate fast, you’ve got all these product ideas. But I think even if you’re a small team in a big company as I was recently working on, and still have a lot of the same pressures you have, product managers who want to try new ideas, you have customers who are sending in feedback all the time—some of which you want to listen to, some of which you want to tune out.

    It’s sort of, in some ways, a lot like being in a startup. At least, ideally it would be. And a lot of the problems you can run into are basically about not being able to close those loops quickly enough.

    Edith: Sam you’re someone who knows a lot about loops, so now might be a good time to introduce yourself.

    Sam: Sure, my name is Sam Stokes. Until recently I was working as a staff software engineer at LinkedIn. And right now I am, I guess a freelance software consultant, thereabout.

    Paul: I feel there’s a little bit of history missing there.

    Sam: Alright. Going back in time a little bit. So I was a co-founder of a startup called Rapportive. That was a plugin for Gmail that when somebody emailed you, would show you a social profile of that person in the right-hand side next to their email.

    So that you could get some insight into who you were talking to and put some more humanity into your communications with people—that’s sort of the way we saw it.

    Paul: That’s still the only plugin to Gmail that I use.

    Edith: I use it, too, and it’s for exactly that reason, ’cause I do a lot of customer development, and it helps so much to do, to empathize. To see that this—

    Paul: Right, right, yeah is a real person.

    Edith: You know, to see their face and you feel a connection.

    Paul: It’s interesting just with kind of customer communication. One of the tools that we use is Intercom. And once I started using Intercom it was the same sort of feeling I got the first time I used Rapportive, that like I feel that there’s a human at the other end of this and I feel I know who that human is and it just changes how you communicate with the people.

    Sam: It’s really motivating to hear and to see other examples as well. We really felt like email and sort of a lot of business communication in general can be sort of cold and formulaic.

    Yeah, just bringing that human element, even just seeing a face. I haven’t used Intercom but I’m guessing they have some similar ideas.

    Paul: Similar, when you send out messages or reply to someone it does the person’s face there.

    Sam: Right, we had a sort of plugin API in the, it was a sort of private beta version of our product and one of the things people used it for, so this would let you plug into the sidebar, people would plug it into their customer database and so they could see something like, they’d get an email from a customer it shows them how long they’ve been a customer, how much they’ve spent in their store—

    Paul: Right, right.

    Sam: What version of the app they’re running, that sort of thing.

    Edith: Was your original idea to do that sort of empathy or was that something that just you iterated into?

    Sam: The empathy was where we started. There were lots of ideas about taking all the information from all these different sources, you know public sources like, truly public sources like Google, quasi public sources like a Facebook profile if you’re friends with them on Facebook, and then private sources like stuff in your company database.

    All stuff that you would be able to search for if you could be bothered to go and do the searching but doesn’t necessarily bring itself to you while you’re talking to that person. And yeah, it was very much, what should you know when you’re talking to this person in order to make them feel like they’re interacting with a real person and an empathetic one.

    Paul: So I remember when you launched Rapportive it was like, or it felt to me like it was, an instant hit and maybe that was just the overnight success that comes after years of trying. I had a doco and what was the kind of software delivery process of that?

    Sam: Hmm, that’s a great question. I guess there’s two parts to that really. So how did it get there and what was our process that allowed us to survive it? So I guess how did we get there, it was a certain amount of luck in that process as there probably always is. We built our prototype and it was a pretty minimal prototype.

    You know, it had the functionality that was like this awful demo website up. The only reason there was a website up was we were trying to raise a seed round at the time, we were applying to OIC. Y Combinator requires that you have a demo site that’s public. They don’t like it to be password-protected. And so we had this thing up where you could install it and we had like 20 users. One of our friends decided that he was going to email a friend of his at one of the blogs, I think it was ReadWriteWeb and say, you know, hey you should check this out.

    And his friend checked it out and it turned out that one of the demographics that really liked the product that we built was journalists because they get a lot of emails from people they don’t know and they want to sort of know an extra nugget. I guess that was one of our takeaways was like, if your startup is really useful for journalists, that’s a great way to get some early press.

    Edith: That’s what worked for TripIt because journalists travel a lot. So TripIt just got all this free press basically ’cause journalists used it, loved it, and blogged about it. Unfortunately, journalists do not do continuous delivery so much.

    Sam: So how do we survive that? ‘Cause what happened in that 24-hour segment was that we went from 20 users up to 20,000.

    Paul: Wow.

    Sam: So I guess one part of it was our CEO had made the original platform choice. His name’s Rahul Vohra, he has a computer science degree, he’s an awesome technologist, but his focus is not generally on writing the software. He’d been writing the prototype, and he was like, ‘I need the quickest thing that I can get to get this market, so I’ll put it on Heroku.’

    Which turned out to be a life-saving decision because when the 1,000 X traffic spike arrived we just added some more dynos. So that was how we didn’t burn down. What did we do after that? I guess in terms of software process we were pretty young as a company, so it was pretty much hack away, read all the tweets. But there was a lot of, what you might call customer development more than software development at the time. So we pretty quickly decided that we were going to try and respond to every single tweet and every single email and so on to like try and live this idea of a good customer experience.

    And I think that helped to sort of amplify the viral effect of that press spike because people were sort of retweeting the things we were saying and getting a generally good feeling about us.

    And so a lot of those tweets were like, ‘Oh I wish I did this,’ or ‘I wish I did that,’ or ‘I wish you could see Twitter profiles in there.’ So it was a little bit of sort of customer support-driven development. Which obviously doesn’t scale but it can be a pretty good way to build some goodwill early on.

    And I think a lot of our users were sort of quite passionate about using us, ’cause they felt like they got that personalized attention.

    Paul: So did the, when you scaled the dynos did it just work?

    Sam: It worked perfectly until we hit the rate limit for the API that we were getting all our data from.

    Paul: The LinkedIn API?

    Sam: At the time we were using a service called RapLeaf.

    Paul: Oh yeah, yeah, I remember them.

    Sam: It wasn’t their main product, but they provided an API which … So basically the thing that Rapportive does, which is take an email address and produce social profiles about that email address. At the time that was entirely just UI wrapped around RapLeaf. We ended up building sort of our own version of that.

    And so we’d been using their API on a sort of demo basis. Because again, 20 users. Yeah, we immediately had to learn about things like request queues, so that our app wasn’t just falling over waiting for their API, and then frantically Biz Dev-ing to try and increase our rate limit, while at the same time figuring out the best ways to get around the rate limit without upsetting the Biz Dev deal. Yeah, we didn’t really have much of a sort of continuous delivery process at the time.

    But I guess not as many people were talking about it—this was 2010. I think the first blog post had come out from Iron View and so on, but I think it wasn’t as, you know, there weren’t full startups supporting it. What we did have was the Heroku deployment model, which was basically like within five seconds of writing the code you can deploy it to production.

    Which if you squint at it, kind of is continuous delivery because you’re saying, all of the bits of your deployment that you have are automated. We didn’t really have many tests at the time, so the tests we had were automated ’cause there weren’t any. Our deployment was absolutely, you do it get push, you don’t have to like move any servers around or anything like that.

    Paul: Seems like the testing model around, or the continuous delivery model around a Gmail plugin is probably going to be a fairly difficult process anyway.

    Sam: We never really had a good answer to, you know, what happens when Gmail changes their UI. The surprising part was it didn’t happen that often. We thought it would be breaking every week and we’d need all kinds of canary accounts and end-to-end testing to monitor that kind of thing. In the end, it happened rarely enough and we noticed quickly enough that we could usually just fix it on the spot. So even when we got better at testing and that sort of cycle of things.

    Paul: We had a problem or I guess solution with GitHub. So GitHub used to be super flaky around two, two and a half years ago, when they were still on Rackspace. And whenever something would go wrong with GitHub we would find out immediately. All of our, all the developments would start failing and so before they knew that there was a problem, we knew there was a problem.

    And we would often like, declare on Twitter, oh we think GitHub’s having a bit of an issue and two minutes later their status would tweet us and then we would retweet their tweet.

    Edith: So they were using you for their free testing.

    Paul: Well, our customers at least.

    Edith: No I mean you were acting as their tester and telling them that—

    Paul: Well, I mean it was very, you know, automated on our but we didn’t have necessarily automated fixes at the start. It kind of happened every, you know, two or three months and they would swear this is the last time and—

    Edith: It’s always the last time.

    Paul: And in the end it happened like six or eight times and by that time we had built a thing that allowed, you know, certain fixes, but it wouldn’t be quite, you know, fully automated process that you would expect if it was happening all the time ’cause it always seemed like it was gonna be the last time.

    Sam: And it’s probably always happening for a slightly different reason or in a slightly different way. It’s actually, it’s an interesting question to me in the subject of continuous delivery in general. People always talk about every time you commit you run your tests, every time you commit, you deploy.

    But that’s actually only one of the kinds of events that you’re dealing with in a software life cycle, right, when something’s actually running. I mean, sure you’re changing the code but also the world’s changing around you. And I feel like you always end up needing a little bit of both—you need something that’s gonna trigger when you change it, but you also need sort of an eye on everything you depend on. And, of course, even figuring out what all those things are can be interesting.

    Paul:

    So I wrote a blog post about this on the Heavybit Blog, I think a while back, and it was about third-party APIs you know, how do you test third-party APIs. And the conclusion that I came to was that, so if you run the tests as part of your test suite, then if something breaks, your test suite will be broken, but also some things will just like randomly break in production.

    And then of course you can’t deploy because your test suite is broken. So you’re in kind of the worst of both worlds. And so what I considered is, is that you need something which, you know, is just permanently monitoring. Like whatever API things you do constantly hits your personal accounts or your test accounts for those things, and then stub it out in your test suite.

    And that way you kind of get that best of both worlds, your test suite doesn’t fall over and you get an actual monitoring event when the third-party service goes down in production.

    Sam: I can tie this to an experience from LinkedIn, actually. So LinkedIn has the model where there’s a staging environment and a production environment. And the staging environment is—I don’t know if it’s technically called a staging environment because it’s not exactly production-like, but what they actually call it is early integration. Because it’s the environment in which teams integrate with other teams.

    And so the idea is you deploy your stuff into EI, early integration, as soon as you can so that if any other team depends on you or vice versa, you can find out there before the thing hits prod. But what you’ve effectively got in EI is this constantly changing environment full of dependencies and you have exactly this problem and different teams which using different solutions to, well, let’s write some integration tests which actually hit the dependencies and then we’ll run them in our test suite when we deploy to EI or when in the build step which—

    Paul: I can’t tell as you’re describing this whether this is a good thing or bad thing.

    Edith: Yeah.

    Sam: I think there wasn’t a clear conclusion, but my conclusion is that it doesn’t work for the same reason that you’re saying.

    You need to know that if your test suite fails, it fails because of something you did. And then you need something else that can fail because of something someone else did.

    But if you’re ever unsure which of things it is, then the signal is lost for both systems.

    Be sure to listen to the full episode. While you’re there check out Heavybit’s library, home to great educational talks from other developer company founders and industry leaders.