devopsy.kinsta.cloud

Author: Chris Corriere

  • Inclusion, Collaboration and DevOps: Seeing the Bigger Picture

    Inclusion, Collaboration and DevOps: Seeing the Bigger Picture

    The last half of 2017 was pretty amazing. I wrote a brief post about the Inclusive Collaboration Summit I attended in London last July. Sallyann Freudenberg and Katherine Kirk started this campaign to raise awareness around neuro-diversity in tech. With respect toward the silence experiments I kept my original post very short. It was great to participate in the silence experiment in London and to help Sal and Katherine with their workshop in Orlando after I returned to the United States, but that’s not the whole story. I offered to facilitate a Wardley mapping workshop while in London in an effort to help the Inclusive Collaboration Campaign form a strategy.

    Coincidentally, enough I also ended up helping Simon Wardley organize the first Map Camp in London last year. While I didn’t get a talk accepted into Lean Agile Scotland and wasn’t expected to speak at Map Camp either, I decided to make the trip back to the UK in October to attend both events as a volunteer. It’s great to get a chance to speak at conferences but I think it’s important to occasionally show up in another capacity as well. I’ve even gone so far as to buy a ticket and show up at an event as a garden variety attendee.

    Both of these events were the same week in October, with Map Camp right in the middle of Lean Agile Scotland. In what one might consider a lapse of judgment, I decided to spend that week traveling between Edinburgh and London on planes, trains and automobiles. Due to a speaker cancellation at Map Camp, I was able to give a short presentation on the mapping approach I used for the workshop at the Inclusive Collaboration Summit earlier in 2017. I also helped Sal with a workshop on sensory perception in Edinburgh. Simon was running a mapping workshop on the last day of Lean Agile Scotland as well, so we had a bit of a race from England to Scotland. He left on a sleeper train right after Map Camp while I took a plane out the next morning. Simon ended up beating me there and I owe him yet another cup of tea. Still, I somehow managed to make all my flights and appointments while I was in the UK, but I didn’t have much time to take in any of the scenery while I was out of town.

    At some point I accounted for this and booked a seat in the quiet coach on a train back to London from Edinburgh. I knew I’d be exhausted and would need some down time to decompile before heading back home. As the train left the station, I crossed off my last checkpoint for the trip and tried to relax a bit before flying back to the United States. As with any journey, this last checkpoint was an appropriate time to pull out the maps from the ICC workshop in London to see how we were doing and what might be next.

     

    Diversity and inclusion are a common conversation in tech these days. They remain a priority at Map Camp as well, which is why we had a pool of diversity tickets reserved and did our best to include as many folks from different walks of life at the event as we could. Diversity and inclusion can and should be improved continuously. If you can’t take my word for it, then maybe this compilation of G.I. Joe public service announcements will convince you. Like Gung Ho says, “Don’t judge people until you give them a chance!”

    Simon got right to planning the next Map Camp as soon as he left London for Edinburgh. I wanted to jump right back in to Map Camp, too, but hosting an Inclusive Collaboration conference was a stretch goal that came out of our mapping session in July, and I had devopsdays Atlanta 2018 to think about as well. The least complicated thing I could do was make Inclusive Collaboration the theme for devopsdays 2018, so that’s exactly what I did. I was geeking out about trains at the time, so of course devopsdays Atlanta will have a train theme this year as well.

    The train theme is more relevant than you might think at first glance. Trains, like any technology we have, are part of a larger socio-technical system. If your system doesn’t have any people in it I’d suggest taking a step back and trying to look at the bigger picture. Of course, you can look at just the train from a purely mechanical perspective, but cutting out the societal impact of expanding travel options and connecting people, not to mention the social impact the construction and operation of railroads had on so many communities, really understates the global impact of the technology. I looked for songs about trains to listen to on the way back from Edinburgh and was amazed at not only how many there were, but how many different decades, genres and cultures were represented in the list. I found this list of about 1,000 train songs, but even the article admits the set is incomplete and the total number of songs about locomotives is likely uncountable. The fact that we’re still writing songs about trains today really goes to show the permanence of the impact railroads have had on our human race.

    The DevOps community is adamant that DevOps isn’t about the tools. We’ve also heard time and time again that containers will not fix our broken culture. I had a long chat with Simon while in Edinburgh over his concerns that the community is about to make the same mistake again with serverless technology. No, serverless isn’t going to fix our broken culture, either. Similar to trains, it’s time to zoom out on the technical details so we can bring the social impact of modern computing into our frame of reference.

    This isn’t news, but like the title of the post says, knowing is only half the battle. We have to actually get off of our asses and try to do something about it if we want anything to change. I’m glad to have Sallyann Freudenberg traveling from England to Atlanta this April to deliver a keynote. I’m glad Jamie Knight and his friends, Lion and Mike Southgate, will be in town from London to spend some time with us in Atlanta, too. I’m excited to hear what Atlanta’s own Kim Crayton, a technologist with a background in special education, will bring to this conversation as well.

    I don’t have any silver bullet answers for the social issues we’re facing in tech today and I doubt we’ll get things sorted in Atlanta this April, but I do know we’ve got some very well-qualified individuals showing up and there will be a lot to learn over the course of two days. Cloud computing, CI/CD practices and the typical tech content seem like known, solved problems to me by comparison, but we’ll be sure to have some folks talking about these topics as well. It’s not a question of social or technical, but rather, How do we do a better job of building and operating socio-technical systems?

    In addition to Map Camp 2018 this October, there may be an Inclusive Collaboration Summit in London later this year. The details on both of those events are still getting sorted out. What I do know for sure is you can join us in Atlanta this April and I sincerely hope you can make it. Early bird tickets are still available and our CFP is open until March 1. Please join us.

    — Chris Corriere

  • In Building Communities, Our Preferred Pronoun is ‘We’

    In Building Communities, Our Preferred Pronoun is ‘We’

    It has been about 18 months since my last post. Toward the end of 2015 I started digging deeper into cybernetics and systems analysis, which led me to construct a model and write a paper. I haven’t posted it before now because it isn’t perfect. The funny thing is, I know it will never be perfect but I still have a list of edits to make. I always will.

    I would describe the paper as a form of hermeneutical phenomenology. The TL;DR on the model: it provides a basis to qualify systems for further analysis. Writing is a mechanism for me to study the model, but as my investigation varies so does my interpretation of what the paper should say. At the moment I’d like to add some observations around four-color mapping, thresholds and compactness. It’s a more rigorous argument than what I said about the continuum hypothesis, but I may leave that bit in anyway.

    Ultimately, the paper is my interpretation of the work I’ve studied. More than read my paper, I’d encourage anyone to review the sources themselves. It goes without saying that I couldn’t have written it in the first place if I weren’t standing on the shoulders of the giants that came before me.

    Self-Inflicted Othering

    I strongly identify as a technologist and a mathematician, but I would be neither of those things if it weren’t for the broader scientific community. Where I keep seeing people get stuck is celebrating our individual differences along with what we have in common. Humans have a tendency to form tribes and wage war. While I’m always glad to see the sense of belonging communities provide for their members, it’s the tendency to direct hostility toward other groups we should resist. J. Paul Reed wrote a post about this last year. We all have circles we run in where our faces are more familiar than others and it’s easy to get “othered” when we step outside of those environments.

    I meet a lot of really smart people at tech conferences who haven’t taken any trig or calculus. If I’m at a math conference I typically get a lot of beginner tech questions from people who are a different kind of brilliant. One conclusion from my paper is the operating environment is the ultimate constraint on a system. We can be local experts on whatever experience we have that’s not the focus of the event. We should also be able to find people that can teach us more about the event’s main topic while we’re there. If you find your experience in a field is making it difficult to learn, try spending more time with newer community members. Their challenges and experiences stand a good chance of providing a fresh perspective.

    We Rise

    This past week I spoke at We RISE Women in Tech Conference, which was hosted by Women Who Code Atlanta. I’ve been among the organizers for devopsdays Atlanta for the past two years and have been making an effort to engage the local community more.

    I spent most of my time at We RISE on the tools track. G. Ann Campbell gave a great talk on code quality called, “Refactoring with Cognitive Complexity: The New Option for Measuring Understandability.” I’ve used sonarqube for years and have been studying complexity science as part of my cybernetics journey. I felt like I was the target audience for her talk. I really liked Meera Subbarao’s talk, “Know Your Enemy, and Yourself: Demystifying Threat Modeling,” too. I enjoy any talk with maps and Sun Tzu quotes, but the highlights of Meera’s talk were the personal stories she shared about trust and failsafe experiments. Experimental methods are playing an increasingly important sociotechnical role in security as our systems become more complex and adaptive. I’m currently convinced all sociotechnical systems are cybernetic as a byproduct of their complexity and all cybernetic systems are sociotechnical at a coarse enough scale.

    Other Kinds of Othering

    It’s that kind of talk that got me grouped in with the “complexerati.” The first time I heard the word I knew they were talking about me (among others) and it wasn’t intended as a compliment. Nevertheless, I couldn’t deny it is who I am. I’ve also been called an insight junkie and street academic and I can’t deny those, either. I can get hooked on a phenomena and dig until I’m completely exhausted. I also never finished my undergraduate degree, which leaves me to perform my research in the street, like a busker. I’m not sure busting my chops over the part of my degree I’m missing is constructive feedback, but we can’t expect everyone to be nice to us all the time, either.

    I know we can catch people on a bad day so I try to be forgiving when someone gets a bit brisk with me. The complexerati comment led to some helpful advice after a diplomatic conversation. If I’m going to use language that’s specific but unconventional, I should make an effort to translate as I go along. People are comfortable with expanding their vocabulary but we shouldn’t introduce too much new material at once. What I said about cybernetics and sociotechnical systems translates into plainer English as, “any system with people in it is self-steering and any self-steering system will have people in it if we zoom out far enough.” The language isn’t quite as precise but it’s still accurate. Negative feedback can be helpful but we have to keep it constructive.fours

    Diplomatic Relations

    Churchill said, “Diplomacy is the art of telling people to go to hell in such a way that they ask for directions.” Communication is vital to trust and learning. Our human tendency to zoom in and steer toward our own familiar shores keeps us divided. This is an excerpt of my last post from December of 2015. It would seem my position has not changed.

    Instead of focusing on processes we should focus on enabling people to improve themselves. We improve when we learn and we learn when we communicate. If your culture does not foster communication can if foster learning? If you’re not open and honest with yourself, how does that effect your environment?

    I could go into more detail about my intentions for next year and may do so at a later date, but my plans are of no use to you as we are different people. I would encourage you to gain awareness of your own situation by challenging the assumptions that have bounded your operation.

    We can stay true to ourselves and our own identities while interacting with other communities. We’re members of some and just allies of others. It’s important to respect those boundaries and understand when our experience is welcomed and appreciated, when it’s time to listen and learn, and when it’s just not the time or place for us and we need to move onto something else for a while.

    We RISE was about technology and culture, which I believe are at the heart of what I’ve been studying for the past few years. The talks about DevOps and security couldn’t have been more relevant to my work. I felt some of the advice I heard in the keynotes from Jen Bonnett, Safia Abdalla and Estelle Weyl that was specifically targeted toward women was probably good advice for anyone, including me. Both Melanie Crissey and Sandra Persing shared their takeaways from the conference here and here, respectfully. Their posts should help shed some light on how I fit in and what I took away from this women’s tech conference more convincingly than I’ll be able to explain.

    I felt very welcomed and included at We RISE. Erica Stanley, Alicia Carr and Beth Laing threw a terrific event. We’re lucky to have them and Women Who Code as members of our Atlanta tech community.

    A Group Effort

    I think this slide I tweeted from Ashley Sun’s talk is the best I can do at summing it up. Ashley gave an awesome talk, during which she showed us how she’s using neo4j for cloud monitoring that gave developers in her organization the ability to manage more of their own infrastructure.

    Ashley shared Kelsey’s tweet in an effort to convince us that just because this solution was a good fit for her organization, it didn’t necessarily mean it would work for anyone else. Adrian Cockcroft had made the point that not everyone can be Netflix. Adrian used to work for them, so he might be a little biased. When Kelsey Hightower quoted him, it provided some external validation to the idea. Kelsey works for Google and they aren’t trying to be Netflix, either. They’re too busy being Google.

    Greg Poirier pointed out that not everyone can be Kelsey Hightower in his talk at devopsdays Silicon Valley a while back. I felt the point Adrian had about Netflix applies to us as individuals, too. Copying Kelsey’s results won’t give us the experience of walking Kelsey’s path. Some experience with Kubernetes may help you on your own journey, but it will never make you Kelsey Hightower. The same applies for Ashley Sun, Adrian Cockcroft and anyone who’s still reading this. We all have some experience others can learn from because we’re each on our own path. As things change over time and our memories fade, not all of what we remember remains how it was today. This is how the perspective of a novice can give fresh eyes to someone with more experience.

    One Step at a Time

    This is why Taiichi Ohno was unconcerned when people came from the United States and wanted to study TPS. By the time they had implemented what they had observed, the process at Toyota would have already adapted into something different (if you’d like to read more about it you should check out this book). The game we’re in has no finish line we can cross. The only successful move we can make is to try to support each other as individuals and communities as we’re able, sharing what we have in common.

    I had a great time at We RISE Women in Tech Conference and will look forward to it returning to Atlanta again next year.

    — Chris Corriere

  • Chronos, Kairos, & Janus: It’s Time to Seek Balance

    Chronos, Kairos, & Janus: It’s Time to Seek Balance

    We conflate idealism with naïveté and pessimism with ‘just being realistic’. But to deny the existence of perfection is to deny the evidence of our own lives.

    Was 2015 a win or a loss?

    It can be tough calling the winners and losers. The mathematics department I studied in had a habit of calling out the As in the class or saying nothing at all. One professor considered us lucky as his alma mater called out the Fs when handing back papers. It’s interesting how information presented from different perspectives can have such a dramatic impact on the range of emotions it invokes.

    A postmortem is an activity that focuses on the time line leading up to a failure. Etsy has done an excellent job of encouraging blameless postmortems in industry. It’s a matter of when things are going to fail, not if failure will occur, when we’re dealing with complex systems.

    Retrospectives are a similar meeting held at the end of a development sprint where team members try to identify actions they can take to have more successful sprints in the future. Looking back on 2015 I see the need to balance our successes with our failures. When we’re focused on optimization we’re too quick to celebrate our successes. When we’re focused on stability we tend to only see the failures and ignore the wins as we come across them. When we’re well balanced it can be hard to tell the difference between our postmortems and our retrospectives, but we definitely have both.

    No matter how you feel about it 2015 is quickly slipping through our fingers. Chronos & Kairos represent time as we typically think of it, mechanically, and the opportunistic time to take action, respectively. I happen to be a fan of Janus, the god of all beginnings and endings, as he was said to exist through all space and time simultaneously. While I do not have that type of experience or foresight I am capable of looking at this past year as I anticipate the next and try to adjust.

    I can remember students that had trouble making passing grades that would be excited about Cs. I can remember other students that would be disappointed if they made a B and utterly devastated if they failed a test. Odds are you’ve been both of these students as our performance is just as dynamic as the courses we’re faced with over time. While we celebrate As it’s really a sign that a student isn’t being challenged. An F is a sign that they need to slow down or maybe back up a bit, and C students represent those that are being challenged by the curriculum and are successfully making forward progress.

    Cultural Balance

    In early November I gave a talk called “Precision, Accuracy, and Debugging the Bimodal Model” at the devOps Summit during Cloud Expo in Santa Clara. My focus was on the granularity in which we measure things and our tendency to get caught up in sorting things as right or wrong, good or bad, new or old, and into other extremes instead of simply acknowledging them as being different. While boolean categorization can be accurate in some contexts there’s often circumstances that require the precision provided by defining a spectrum between two extremes. These intermediate states often increase our odds of implementing successful change as they provide gradual steps to follow instead of expecting a single instantaneous jump to occur.

    When undeniable failure occurs we find where to learn by following a time line of small successes. At some point our expectations will not accurately line up with the reality we’re faced with and this is where we need to take a closer look.

    I was able to catch Devops Days Silicon Valley while I was in Santa Clara. The conversations I had where intended to gather more feedback around culture hacking and how culture influences the artifacts an organization produces. The theme was “bridging devOps cultures”, which made it easy to get feedback on some models I’ve been working on as it was a general topic of conversation at the event.

    Matter of fact, concerns that cultural change is ultimately what is needed for sincere advancement echoed throughout various sectors of the science and technology industries I found myself in this year. Devops Days Silicon Valley made it a theme, but cultural change seemed to be the hallway track topic at most of 2015’s events. Dave Snowden‘s cynefin framework was also present at most events I attended this year. I was introduced to the model during Jabe Bloom’s talk in NYC. Both J. Paul Reed and myself mentioned it at devOps Summit, but Jason Hand went into it in more detail with his talk, “Flaming Poo and the Human Response” at DevOps Days Silicon Valley.

    Lean Agile Scotland had a lot of discussion around culture and cynefin as well. It was even in the carpet. This worked for me since I made the trip in an effort to contrast the cultural differences in tech communities across US cities I had visited with an international group. Outside of the presentations culture was a common topic among corridor conversations. Lean Agile Scotland had folks from all over the world in attendance. It’s often the little differences, like calling it a corridor instead of a hallway, that we tend to find amusing.

    Conway’s Law links the artifacts an organization produces to its communication structure, which ultimately is an artifact of the organization’s culture itself. I’m starting to look at culture as the context of an organization with respect to how it communicates. Jabe Bloom and Chris Matts both gave presentations on culture in Edinburgh but the videos have not been posted yet. Here’s a picture of Jabe providing a definition of culture during Chris’s keynote.

    culture

    I was really delighted with Chris’s keynote. It was well rehearsed and entertaining but it also provided a missing piece to a vector based model I’ve been developing in an effort to more consistently benchmark organizations. I had hit a wall as I couldn’t see how I could quantify something like culture without injecting my personal bias into the metric, but what Chris shared about Hofstede’s Cultural Dimensions easily drops into my model and satisfies the requirement.

    I’ve received a lot of feedback on my model between Pittsburgh, Edinburgh, and Silicon Valley and it’s all been very encouraging. I’ve seen initial doubt turn to optimism and have had lots of people offer to help me extend my efforts and research this further, which I intend to be a focus for me in 2016. I do look forward to benchmarking organizations next year in an effort to find more paths between safe game states in industry.

    Looking back at the communication I described earlier with professors handing back graded tests, it is clear to see that how we learn is influenced by the emotions that are emphasized when we communicate. Are we focused on failure or are we focused on success? Does the culture we’re in value stability or optimization more? Does your team have postmortems, retrospectives, or both?

    Katherine Kirk advocated “Why Team Happiness Can be the Worst Thing To Aim For”. Insight is enveloped in emotion and equanimity is the tool we need to access it. As emotions are fleeting we should not pursue them. Instead, by anchoring ourselves in equanimity we are prepared for the waves of different emotions that will come and allow them to pass without judgement.

    Richard Sheridan, author of Joy Inc and CEO of Menlo Innovations, offered a different perspective with his keynote on how to “Build A Workplace People Love – Just add Joy“. He describes an open, high trust community that’s centered on learning. This means an environment that’s comfortable with failure when it happens and embraces the opportunity to learn from it. To pursue an emotion may be folly, but when culture is so harbored on stability that the failure ultimately manifests in the human components of the system it is only natural to drastically course correct towards different bearings.

    To Err is Human

    Jonathan Disher‘s talk, “DevOps Army of One” at devopsdays Silicon Valley was an excellent blend of technical and cultural information. We automate in an effort to gain some room for balance but that space can quickly be filled with other work instead.

    Asking for certain kinds of help, like more team members to offset workload, can bring leadership to question an individual’s level of commitment. Fear of retribution may discourage us from taking the personal time we need even if we don’t have adequate depth on the team, but eventually something will give. This is why burn out rate is high in our industry.

    The intention of mapping is to clearly challenge our assumptions. This helps us to realize we’re off course as soon as we checking our bearings. Wardley Mapping is a value chain mapping approach developed by Simon Wardley, appropriately enough. His mapping is quite practical and provides an interesting contrast to Edward Tufte‘s techniques, another data visualization and mapping expert I was pleased to meet this year.

    It is unfortunate that the time needed to give the level of attention required for this type of insight is often a considered a luxury in corporate cultures. Thanks to automation and adaptive visualization frameworks the standards are slowly rising. Will Evans‘s presentation from Lean Agile Scotland provides a solid blend of Wardley Mapping and design as he explains how new practices like these enter our workplaces.

    A Culture of Learning

    As 2015 enters its final moments I feel successful in finding consensus that the cause of our failure is a cultural one. I believe we cannot improve what we do not measure, and therefore my plans for 2016 are to further study, model, and visualize organizations and their equilibrium states.

    I cannot help but form a hypothesis on how we fix the problem before I’ve gained the context to make a reasonable attempt, but mapping is intended to help us challenge our assumptions. I’m assuming the solution lies in our actions as individuals and we must want to improve ourselves if we expect to see improvement around us.

    Often the final state we are striving for is literally not achievable. John Willis introduced me to this concept of “True North” from Toyota Kata, by Mike Rother, and suggested I read it myself to learn more. We’re currently reading Toyota Kata in a devOps book club we started, now that we’ve finished reading Designing Delivery by Jeff Sussna.

    The book club is intended to help us discuss ideas as we move through a book. I’ve been surprised to find how asynchronously a book club can operate. Everyone reading the same book at the same pace is a more typical book club approach, but given we live in different time zones and keep busy schedules we’re adapting to a model that’s a better fit. If you’ve read Designing Delivery, Toyota Kata, or plan to read either book and would like someone to discuss them with feel free to ping me on twitter.

    The improvement kata from Rother’s book maps the scientific method into Plan, Do, Check, and Act. Instead of focusing on processes we should focus on enabling people to improve themselves. We improve when we learn and we learn when we communicate. If your culture does not foster communication can if foster learning? If you’re not open and honest with yourself, how does that effect your environment?

    I could go into more detail about my intentions for next year and may do so at a later date, but my plans are of no use to you as we are different people. I would encourage you to gain awareness of your own situation by challenging the assumptions that have bounded your operation over the past year. You have had successes and failures and both deserve attention. Reviewing your outcomes with equanimity will help balance your actions, and hopefully lead you into a successful 2016. Happy New Year.

  • I signed up for an ass kicking

    I signed up for an ass kicking

    I imagine most people are like me and have a lot of things they’d like to do if they could find the time and energy for them. Lately writing has been one of those things for me. I suppose we all have our priorities. Traveling for tech events and coaching the kids’ soccer team can make it seem like there’s no time to write, and then something legitimately disruptive happens like one of your kids landing in the hospital.

    I recently read Designing Delivery with a group of folks as the inaugural title for our devOps book club. In the book Jeff Sussna discusses delivering quality experiences to your customers as early and often as possible. Most people have been exhausted from traveling at some point in their lives. Many examples from the book talk about this shared experience. This allows us to empathize with a specific type of pain many airline and hotel customers are trying to manage while going about their business.

    Early engagement that sets expectations can make a huge difference. By providing realistic travel times that account for things like seasonal roadwork airlines are more likely to have their customers arrive on time and in a good mood. A hotel that knows they’re dealing with jet lagged, travel worn customers would be well served to have fresh coffee, water, and fruit for them when they arrive if not a drink at the bar.

    You can probably empathize with being travel worn if not jet lagged, but I doubt you understand the level of exhaustion and confusion that comes with having one of your children hospitalized in an emergency situation unless you’ve been there. Hospitals and hotels have a lot in common. One luxury a hospital doesn’t always have is setting expectations for the patients and families. In an emergency you could have less than an hour’s notice before the hospital makes its first impression. How do you make someone comfortable and help them relax when they’re quite reasonably stressed out and concerned?

    My twins were diagnosed with Twin to Twin Transfusion Syndrome (TTTS) before they were born. The illness required an emergency trip to Women and Infants Hospital in Rhode Island on short notice. Providence is a sixteen hour drive from Atlanta. Given I was still in college we stayed at the Ronald McDonald house which are around for families that have a sick child in a hospital away from home.

    A trip to an emergency room for a broken arm is one thing. Not knowing if or when your child is coming home is something else entirely. There were kids getting picked up by buses for school and parents that left for work while other members of their family where in the hospital or back home in another city. I’m thankful our stay was short and relatively uneventful. Women and Infants is an amazing hospital and the Ronald McDonald house really made us feel at home.

    My twins have had their lives saved at Children’s Health Care of Atlanta a total of three times since then. One was hospitalized with cardiac issues immediately after she was born and had a balloon catheter procedure done at Egleston. Both have been in intensive care at Scottish Rite for respiratory issues which includes a visit within the past sixty days.

    CookieOps was something I started with them over a year ago to try to raise money for these organizations. My girls sold me on the idea by telling me, “There are sick kids that still can’t go home and play with their brothers and sisters and we need to help them”. What started out as a bake sale turned into a model for approaching problems with a devOps perspective. For us that means embracing the Three Ways and following the CALMS model (Which is CAMS with an ‘L’ for Lean/Learning). We believe that every process has an artifact and there’s room to optimize both of them. It’s this type of thinking that leads to putting a mint on a pillow and understanding the cost involved far outweighs the related expenses. It’s often the little things we find delightful that make us loyal customers for life.

    parking1

    Like leaving cards near the elevators to help you remember where you parked coming in from the deck to find your child. I’ve wander around in a daze looking for my car in a hospital parking lot before. They’re big buildings that often have multiple decks. It’s easy to forget where you parked when you’re in a hurry to check on much more important things.

    nursesCard

    My daughter was really excited to get a postcard in the mail from her nurses after she was feeling better. I’m not sure how much time is spent writing these cards but I’m thankful for it. Extending the hospital’s experience out to when you’re home and things are back to normal is something that should be celebrated and is well worth the cost involved.

    I’ve investigated a few ways for my kids to raise money with cookieOps but so far we’ve been giving out cookies for survey responses. Publishing the data from those surveys is another thing I’d like to make a priority but haven’t had the time to complete. As raising money for the hospital was always our primary goal when we first started putting surveys out I had little choice but to do so as efficiently as possible when the opportunity presented itself.

    Brawl for a Cause allows regular people to raise money for causes they believe in by stepping into a boxing ring for their first amateur boxing match. A big difference between a road weary traveler and a family with a child in the hospital is the traveler usually signed up for the ass kicking. The hospitals are dealing with families that would have been fine without ever visiting, and the kids definitely didn’t sign up to fight with their illnesses.

    So I’m glad to step in the ring on November 25th, fight for the sick kids that can’t go home yet, and make good on the promise I made to my daughters. My opponent is boxing to raise money for his daughter’s college education. While I believe everyone deserves the right to an education I also believe they need to live long enough to make it through school. There are opportunities to take your education into your own hands through programs like Kahn Academy. The kids in the hospitals lives are in the hands of the doctors and nurses watching over them. Those kids didn’t sign up for the ass kicking they’re taking, but I’m willing to risk a beating if it can help Children’s successfully ease a family through such a difficult time.

    If you’re in Atlanta we’re boxing at the Hyatt Regency in Buckhead on the 25th. tickets are on sale now. You can donate to support these causes no matter where you live. I’ll take the punches but I need You to make a donation today.

    buckheadBrawl

  • DevOpsDays MSP 2015: Minneapolis is a Tech Hub?

    DevOpsDays MSP 2015: Minneapolis is a Tech Hub?

    I stopped in the Minneapolis Airport to make a connection earlier this year. I never spent much time thinking about Minneapolis given its reputation for snow and ice. I hadn’t ever planned to go there nor had I explicitly made plans to avoid it. That was until the lineup for devopsdays MSP came out.
    I remembered seeing videos from the previous year and being mildly surprised they had such a seasoned lineup. When the speakers were announced I wasn’t expecting to see such a solid group of presenters taking the stage again. I like going to events in cities with a thriving tech industry. It always makes for a good hallway track when a lot of sharp folks are in the area and can easily attend. So what were all these people doing in Minneapolis?
    It ends up a lot of tech heavy companies are in the area. Target, US Bank, Cargill, General Mills, Best Buy, Land O’ Lakes, and 3M are among them. None of these are pure tech companies like I would expect to find around San Francisco, but any company that’s competitive these days is probably leveraging a lot of technology in an effort to better serve their customers and keep things running smoothly behind the scenes. If you know anything about what I’ve been doing with CookieOps it should be no surprise I was thrilled to see technology applied to traditional business models so enthusiastically.
    Target not only sponsored the event but had employees show up in force, fully engaged. 2nd place highlight for this event was discussing devOps, agile, and lean principles with Mary and Tom Poppendieck, authors of Lean Software Development: An Agile Toolkit, along with a group of folks from Target. Mary’s talk, “The New New Software Game” focused on techniques that make it possible for small teams to deploy small changes to large systems very rapidly and very safely. More specifically, this was a talk on how to break monolithic systems into micro services. I really appreciated how her talk highlighted human interaction and how it incorporates with technology.
    Getting to listen to Mary and Tom talk shop with Target was incredible. They mentioned the shelf life of software on more than one occasion, and I may have gotten a little light headed when Mary used a cookie company as an example of a manufacturing process that’s deeply concerned with time to market.
    The top highlight of the event for me had to be Daniel Willis’s ignite talk, “Sports Stats 101: Putting the R in Sports”. As a mathematician and a father I think getting kids into technology early is one of the best things we can do for them. When I saw Daniel was giving a talk on baseball stats in R at the age of 12 I had no choice but to make plans for Minneapolis. Daniel didn’t disappoint either, cracking standard deviation jokes in his bow tie in front of a crowd of about 400 people.
    Colleen Velo’s talk, “Rolling Your Own vs SaaS” covered the tradeoffs between internally coding solutions, using open source software, and running infrastructure in the public cloud. Bloom Health, another Minneapolis business, is in the high risk environment of healthcare. I found Colleen’s perspective and experience on this subject to be extremely relevant given the new security challenges we’re facing in our industry and the trouble that higher risk sectors are having with innovation as a result of it. I wasn’t surprised to hear Bloom is moving to SaltStack giving the risk associated with the industry where they operate.
    I also encourage you to take a look at Josh Zimmerman’s presentation on “Why You Should Care about DevOps in the Public Sector”. Public sector IT employees face a different set of problems with respect to structure, motivation, and funding. DevOps has moved from startups towards the enterprise, but there’s still a lot of good these practices can do to solve problems in the public sector where much of the resistance can be attributed to the need for a cultural change. Josh and I had a chance to talk about the cultural perspective of libraries, and how they’re viewed as places to keep books instead of places to keep information. I try to check out a library when I’m in a new town and see if they have anything interesting in the archives. This trip landed me in the Minneapolis Central Library where I was able to look at an early edition of Walden that included signed letters from Henry David Thoreau. There’s a lot of good stuff at the Hennepin County Central Library including a Dunn Bros Coffee on the first floor, a Teen Tech Center sponsored by Best Buy, art exhibits, and even a sound proof piano room. I stayed in Special Collections the entire time  I was there, but if you’re in Minneapolis it’s definitely worth stopping by.

    thoreau
    I really appreciated Jason Clifford ignite on GameOps, volunteering it as an Open Space, and coaching a group of us through a few rounds of The Resistance at the evening reception, which was sponsored by Target. Games like The Resistance and Werewolf help us learn how to interact and reinforce the low trust commensalism environment I’ve spent a good bit of time promoting this year and will be presenting this October at Lean Agile Scotland. If table top gaming isn’t low trust commensalism I don’t know what is. You literally have to share a table.

    I also introduced the group to The Drawing Game during the open space. You start with a random doodle on a blank page and pass the paper and pen around to each person so they can add something to the picture. We made it around our open space twice before stopping and ended up with the picture below. This game is a life saver for any group stuck waiting to eat, especially if there are kids with you. A spare place mat and a pen borrowed from the staff have warded off more toddler meltdowns for me than I can count. It’s still fun for older kids and adults too, even if you’re just watching the scene unfold.

    drawingGame

    So how did all this awesomeness end up in Minneapolis? As far as devopsdays MSP is concerned I’d give a lot of the credit to Bridget Kromhout. It’s obvious she puts a lot of time into the devOps community at large and hasn’t forgotten to help build a strong community in her own backyard. The food, shirts, event communications, AV, and even the slides and videos on the website for the event can all be traced back to her and the rest of the amazing staff. Bridget promised an awesome event while in New York but I was skeptical she might be partial when she started talking about the wonderful warm weather in Minneapolis. I can confirm first hand it doesn’t snow year round in Minneapolis, and devopsdays MSP happens during the perfect time of year for a visit.
    We’re only two weeks out from devopsdays Pittsburgh, and I’m looking forward to attending the event, visiting this town for the first time, and meeting some new faces while visiting with some more familiar ones. My expectations are pretty high thanks to Minneapolis, Bridget, and the entire MSP crew. Is Pittsburgh a tech hub too?

  • The devOpsSec Dilemma: Effective Strategies for Social Networking

    The devOpsSec Dilemma: Effective Strategies for Social Networking

    I was sad to hear of the passing of John Nash and his wife Alicia this weekend. May they rest in peace. As a game theorist I am familiar with his work and it just so happens that Nash Equilibriums have been in the center of what I’ve been working with lately. It’s an honor to be in a position to build on his ideas and hopefully pass his legacy on to a larger audience at the time of his death. I’ve done my best to pay him my respect by carefully studying his work over the past year, given Velocity starts this week.

    devOps believes in intelligent actors. Security assumes the worst of intentions. Both risk an imbalance of trust. The Prisoner’s Dilemma and the devOpsSec Dilemma, defined here as a lack of cooperation that stems from a lack of trust in an hyper competitive environment, have the same flaw: There’s not a game position that’s safe for everyone because of obliviousness or malicious intent. What I believe is emerging is a low trust cooperative game state. In a devOps world everyone means Everyone both internally and externally. This includes the unique identities of you, your co-workers, the company you all work for, the customers that keep you in business, and even the people who you perceive as your competition.

    The oblivious trampling and opportunistic predation of other people  and organizations are really complementary set operations that mitigate any risk associated with a perceived threat, which is what security is ultimately about. Staying in the lowest risk, highest paying game state is a decent definition of a secure bet for the short game but may not be a safe one over time. If the strategy is damaging others you will eventually compete yourself out of existence.

    We have harassment problems in our industry. I want people to come to events and interact in an effort to network with folks because we grow faster when we share ideas. It would seem we have some of the wrong people showing up given the problems stemming from this type of interaction. If you’re passionate about technology and can respect other people and their skill sets you’re welcome in our community. If not, Keep Out. This applies to everyone, including pushy recruiters and vendors.

    The problem with amensalistic behavior is you’re not aware that your offending someone when it happens. I’ve talked with a lot of people about this and there’s definitely room for a misunderstandings and they’re often cultural. When it turns into harassing behavior and you start looking like a parasite is when it’s characteristically repetitive. The key here is to pay attention to people. Watch for non-verbal queues when people are trying to politely excuse themselves. If you don’t catch the hints people will tell you they don’t have time or the energy to talk and you really need to take No for an answer.

    I’d like to make it particularly clear that if you’re looking for a date you should go look somewhere else. Given the gender ratio issues at events your time would be best spent elsewhere. There are appropriate times and places for it but a professional environment is not one of them. Most conferences and events are in big cities. You can take your name tag off at any time and find plenty of trouble to get into outside of an event if that’s your thing.  That means no one should have to ask you twice, Period. If someone has to tell you three times expect a less than friendly visit from event organizers. Let’s be clear that there’s also zero tolerance for unwanted physical contact in our community. No one should have to explain that to anyone and there is no good excuse.

    With the exception of physical contact and other overtly offensive behavior, we also have to trust that people can make a mistake instead of automatically assuming the worst intentions. An intelligent agent will correct a poor behavior if they’re politely made aware of it. Again, if there’s a characteristic repetition in spite of someone asking them to stop their plausible deniability goes out the door with them. I noticed someone handing out business cards to a strip club in front of a conference recently. I told him that while it was mostly guys here it wasn’t the time and place to be handing those out and he’d have a problem on his hands if he didn’t leave. He apologized and left. Peer moderation can be effective and doesn’t have to hurt anyone. I feel confident this is a fair approach given the explanation of Blameless Problem Solving found on Etsy’s 2014 Progress Report:

    “Making mistakes is an inevitable by-product of doing innovative work. Accidents can actually be valuable and rich sources of learning. We strive to create a blameless culture, in which it is safe to make mistakes and to speak up about them. This allows us to gain as much knowledge as possible from our experiences.”

    We need to move out of no trust interference and do a better job of working together. This is a shift from no trust security towards high trust safety. This is also the split between devOps and security. DevOps believes in intelligent actors. Security assumes the worst of intentions. I covered this material at DevOpsDaysNYC. The video goes into more detail and has some ecological examples of the relationships on the chart, which I’ve added to since the talk.

    [youtube http://www.youtube.com/watch?v=O9hgYtNlo3o]
    The Prisoner’s dilemma is a Pareto inefficient Nash equilibrium because there’s no honor among thieves. Not defecting on the other prisoner leaves an opportunity for them to defect on you, walk out of jail, and leave you incarcerated for the longest period of time. Both prisoners defect and they both end up serving a longer sentence than if they had taken the risk of cooperating.

    Prisoner B stays silent (cooperates) Prisoner B betrays (defects)
    Prisoner A stays silent (cooperates) Each serves 1 year Prisoner A: 3 years
    Prisoner B: goes free
    Prisoner A betrays (defects) Prisoner A: goes free
    Prisoner B: 3 years    		 Each serves 2 years

     

    A significant change in a system replaces the original with a different, adapted system. A minimal number of local changes can be accommodated without inherently changing a system but enough local change in a short period of time is equivalent to a global shift. The devOpsSec Dilemma is the type of systemic failure that demands a systemic change. So I came up with a new game board.

    Here we move from no trust competitive environment where everyone gets hurt towards a Pareto improved equilibrium. By attempting to move out of competition we risk being stepped on or preyed upon if our counterparts don’t make the shift to a higher trust relationship with us. The good news for our game board is that I don’t think anyone has to die or go to jail. While there is some risk here it’s most often a blow to one’s ego, which is a risk I can entertain for something I care about. The solution here is more diplomatic peer moderation and the ability to take constructive feedback even if it’s uncomfortable. We need to move out of safety and danger towards cooperative learning. When you move out of learning and into danger we end up hurt, but it isn’t always competitive.

     

    adaptiveTrust

    Cooperation is at the heart of John Nash’s work. Trust is fundamental to our survival and is built by showing what you can contribute, not haggling and spit-balling in the bike shed. The more we trust the more we learn and the less we get hurt but it takes a combined effort. If your patterns as an individual or an organization don’t demonstrate these qualities over time then you’re done. As Deming put it, “Survival is not mandatory.”

  • DevOps Spring Tour 2015: Keeping Cool with ICE

    DevOps Spring Tour 2015: Keeping Cool with ICE

    One thing I love about conferences is finding out what other people are doing in industry and getting exposure to new ideas. I like to get external feedback on my work because I find it to be helpful. Here’s some highlights from where I’ve been and what I’ve heard so far in 2015, including Alterconf, RSA, A Data Presentation Course with Edward Tufte, and DevOpsDays NYC.

    AlterConf

    My first stop was Alterconf in Atlanta this year. I try to support local events and knew I would gain a lot of perspective from the presenters that I probably wouldn’t find elsewhere. There was a lot of discussion about age bias and discrimination for both younger and older people as we all move through those stages during our lives. Starting a culture of continuous learning early in life was a consistently supported goal among the presenters and attendees.

    My other take away from alterconf is people may or may not be comfortable representing or discussing the different groups they belong to with anyone. Introverts can have a very difficult time talking tech at an event, for instance.  Several speakers at the conference made note of the fact that we will occasionally step on each other’s toes due to what are ultimately misunderstandings and we should be quick to apologize, provide space, and try not to get too upset when someone makes a mistake or appears uncomfortable. I was encouraged to see the consensus lean towards discretely and respectfully bringing any issues up if necessary instead of public exposing someone. Even snapping at a manager in a meeting over a wisecrack about a father ‘babysitting’ his own kids does more harm than good. You shouldn’t stomp on anyone with both feet over failed build either. Having a calmer conversation with a smaller audience if possible was the advice I heard repeatedly throughout the day.

    We’ve been having more conversations around anxiety and burn out as a community and talking a lot about comfort, learning, and danger zones. I’m going to openly admit I was more nervous than usual going into alterconf as a white cisgendered heterosexual male. It may have had something to do with the fact that I didn’t know what cisgendered meant before the event. The anxiety quickly passed after I found my seat and realized there were plenty of people there just like me. At a minimum all of us were passionate about technology. It was a very open and welcoming space and I gained a lot of perspective that I wouldn’t have otherwise. There are 5 more sessions scheduled this year if you may be interested in gaining some perspective from a broader range of people that are also passionate about tech.

    DevOps Connect @ RSA

    DevOps Connect @ RSA was a uniquely awesome experience. I met a lot of security folks coming into the DevOps space for the very first time which was encouraging to see. I also got an up close look at the security industry and why they need to gain some insight from a DevOps perspective. The expo floor was the most commercialized one I’ve ever seen at an event. It reminded me of a midway at a state fair. I wouldn’t have been shocked if someone had called me in for a go at the ring toss. Most of the sales pitches sounded like magic beans to me, bolted on for instant risk mitigation.

    The one vendor that separated themselves as an exception to this was Tufin. Tufin’s security orchestration suite is a comprehensive solution for network security management providing visibility, change tracking, analysis and auditing for firewall policies, network devices and cloud platforms. It also provides automatic firewall change management and application connectivity management. It assures a tight security posture, rapid service delivery and regulatory compliance across all enterprise platforms.

    I haven’t taken it for a spin myself, but Tufin definitely impressed me with their demo and orchestrated approach to the problems security is currently facing.

    Certification was also a consistent topic of conversations at RSA. I think there’s a lot of misunderstanding in this space between arbitrary compliance and compliance that legitimately mitigates risk or provides a return on invested time and energy. This sentiment echoed through Secretary Jeh Johnson’s keynote, as he announced the opening of a satellite cyber security office in Silicon Valley and an invitation to cooperate with concerns surrounding encryption.

    Data Visualizaiton Workshop with Dr. Tufte

    My next stop was Boston for a data visualization workshop with Dr. Edward Tufte. When I told him I was a devOps engineer working on dashboards in an enterprise environment I could tell he was curious but may have not heard the word devOps before. The person I was speaking with before Dr. Tufte came by mentioned a health care system that still faxed data between offices and the problems it caused with younger employees that had never used a fax machine before. It was obvious to Dr. Tufte that they should switch it to a web based system, and the frustration with compliance and risk concerns that prevent that change from happening were not lost on him. For lack of better words and opportunity, I offered this as the type of problem devOps is currently trying to solve. The perspective I gained from the session that followed was from an experienced professional that can fully appreciate the current state of the art. I believe I appreciate it now more than ever, for which I’m thankful.

    DevOpsDays NYC

    The topic of discussion was ICE, or Inclusion, Complexity, and Empathy, based on a post by Dave Zwieback from earlier this year. The talks focused on how to try to cooperate better between teams both at home and abroad and suggestions on how to handle departments that are more territorial in addition to dealing with our own stress levels.

    Attendees seemed particularly grateful towards John Willis and Justin Lintz for talking about burn out and anxiety in our industry among heavier topics. If you’re inclined to start a conversation about these types of health issues in your community then you probably should. I try to shut up and listen as much as possible, and that’s often helps more than one would think. If you have a venue to show John’s video before an open spaces event all you have to do is book the room and be available. Creating an opportunity to discuss these issues is a huge step forward in my opinion.

    But wait! There’s More! I’m giving an Automating Enterprise Security Talk at Gluecon outside of Denver later this month. I spoke about this at devOps Connect @ RSA but I always update my content between talks as I get more feedback from folks. If you happen to be at Gluecon this year we should have a conversation over coffee, a beer, or a meal. From my experience a chat tends to be more fun in person than online, as I for one get tired of staring at screens all day.

  • Balancing Between Impostor, Blowhard, and Burnout

    Balancing Between Impostor, Blowhard, and Burnout

    I’m glad John Willis wrote his post on karojisatsu, or suicide from overwork, and that it has been getting the attention it deserves in the community. If it weren’t for the push behind the article that has kept it surfacing in my feeds the past few days I definitely wouldn’t have realized I needed to stop and write this right now.

    It’s great that this issue has been brought to our attention and the community response has been awesome. I’ve seen a lot of people offer their availability for anyone that needs someone to talk with, but there’s definitely some frustration surrounding what can be done to help without offering useless platitudes, as Gene Kim so eloquently put it.

    I have noticed a cycle I can get caught up in that can leave me in a bad place, but now that I’m aware of it I’ve been better at avoiding it. I wanted to share my experience so that others dealing with any of these problems have something they can try that may help.

    Be Confident, but not a Blowhard
    Christina Xu recently posted her thoughts on blowhard and impostor syndrome. I agree we all have areas we’re comfortable in and shouldn’t take on any work we don’t feel comfortable doing. It doesn’t matter if it’s because we don’t feel we’re qualified to do it yet or if we just aren’t interested in a particular space. Anyone that doesn’t immediately jump head first into a technology isn’t suffering from impostor syndrome. Anyone that does or says they can is probably being a blowhard.

    I can and do move in and out of a lot of different spaces in tech but I definitely know my limits. I never really learned much C++ and have no business writing it. However, if the need came up and it made dollars and sense I would try to take some time to transition into C++. This would take months, not weeks, and it would be a lot of hard work. I would even argue it would be painful, but I’m not scared of hard work if it’s worth it.

    I Allow a Reasonable Amount of Time to Transition
    I have no plans to touch any C++ any time soon. I am presently moving back into JavaScript though which shouldn’t be too bad. I’ve been working in JavaScript since I was in middle school but I haven’t done any deep diving in this space in years. I remember wanting to learn backbone and marionette but the need and the time to transition never lined up. I’ve been working with mostly back end code like services, DAO layers, and build tools the past few years. I’ve used JavaScript sparingly for things it does well, but I haven’t had to think very hard about JavaScript in a while. I really need to gather my self confidence as I dive back into JS, and that means I am at risk of being a bit of a blowhard. Allowing time to transition can keep me from getting too far ahead of myself.

    Asking someone if they “ know” JS is a very vague question that could lead to a lot of misunderstandings these days. I need things like angular, ember, node, and D3 but they’re all pretty new to me. It’s really awesome that there’s all this new stuff to play with and better tools to use. I feel I can do more with less thanks to the changes that have happened in the JavaScript ecosystem since the last time I was here.

    True Impostor Syndrome
    I didn’t always have such reasonable expectations about how fresh a particular knowledge base was in my mind. Years ago I would have dove back into JS head first expecting things to be where I left them and then panicked when I realized the landscape had changed and my skills were rusty. If this realization came late at night after a particularly hard day (never mind week or month) it could leave me in a very bad place. In that moment it’s easy to declare yourself a failure of a human being and vow to never code again. This is true impostor syndrome. At it’s worst it can lead to thoughts of ending one’s own life.

    I Limit My WIP to Account for Adversity
    Impostor syndrome can attack when you’re working in a domain your comfortable in as well. Because there’s so much work available it’s easy to take on too much. When you’re spread too thin, teetering towards burn out and then slip on a banana peel of an issue with no slack in your WIP to accommodate it you can end up in the same pit of despair.

    How I Deal with Burnout
    I’ve come to the conclusion it’s better to be proactive in avoiding these scenarios than reactive in responding to them but we need to do both. Let’s assume you’re burned out from trying to transition too quickly or taking on more work than you should have. These things often lead into each other I’ve come to find. Here’s what I do when I know I’ve bagged my limit.

    Step Away from the Keyboard
    Taking time off seems obvious but we can’t always drop everything and take a week or even a whole day off. There are things I do to mandate several daily breaks during different parts of the week. The key here isn’t to decide if I have time to do them, I just drop what I’m doing and go do them on principle. Whatever I’m putting down should be able wait. This is about establishing a lifestyle change more than just taking a break.

    Exercise
    If you don’t do anything start with daily walks outside or yoga and meditation. I’m better off doing these twice a day for 15 minutes than going to a class once a week. It wasn’t until I started doing these things this regularly that I started to really feel the benefits. Even a stroll around the office building for some fresh air when the heat gets hot can do you a world of good.

    Eat Right
    GIGO in it’s truest form. Eat healthy stuff and splurge in moderation. My family eats fish and vegetables almost every night because it’s healthy, quick, and relatively inexpensive (if we buy the vacuum sealed frozen fillets). I’ve been working on drinking more water and less coffee. We do have a larger meal filled with red meat and chocolate desserts every couple of weeks at least. I believe any food that’s good for your soul is soul food but it’s usually not very healthy from a nutritional perspective. You should definitely eat it in moderation anyway.

    Go to Sleep!
    I used to get home and poke at work in front of the TV until late at night with the DVR spewing garbage at me. I got rid of the DVR and only watch shows as it’s convenient. I get to bed by 10pm more frequently and feel like I’m making better use of my evenings. I have trouble sleeping more than 6 hours at a time, but I try to leave room for 8 in case I need it and I definitely take it when I do.

    Find a Computer-Free Hobby
    I happen to be very passionate about BBQ. I try to smoke some brisket or pork at least once a month. I also play a handful of instruments very casually as I don’t want it to turn into more work. Band practice and playing gigs takes a lot of energy and would not result in the desired effect in my particular scenario.

    Volunteer
    Nothing will make you feel like you matter in this world than giving back and being of service. I coach my kids’ soccer team but the sincerity of the “thanks Coach!” I get from some of the kids and their parents feels really good. I completely wasn’t expecting it but I now understand why some coaches stay in the program long after their kids have grown up and moved away.

    There are a lot of people eager to learn to program that would love your help. Look in your community and see who you can help with HTML and CSS. It doesn’t have to be children either. A lot of adults are look to refresh these skills or are trying to move into technology from another field. They would love a mentor they can learn from and you’ll end up being better at what you’re teaching too.

    They also need help at food banks, animal shelters, and many other places. Find something you care about and try to get involved. Trust me, you’ll feel better about life.

    No Burn Out BBQ and Free Cookies
    I am also here for you if you need me. Specifically, if you’re in the Atlanta area and need someone to talk to hit me up. You could live here or be through on business but I will absolutely break out and come meet you if it’s at all possible. I feel the face to face interaction is needed for me to really connect with someone and give them my undivided attention.

    On that note, I’m interested in starting a No Burn Out BBQ to provide an opportunity to do all of these things in Atlanta. Having a BBQ one weekend a month would get us away from the keyboards and outside into the fresh air. It takes a lot of volunteers to get the meal together, and it would provide a safe place to come talk about burn out on top of serving as a monthly reminder to unplug and decompress a bit.

    I have a side project with my kids’ that involves sending people cookies and a drawing for filling out a survey. I’ve opened a survey to gauge the level of interest in a No Burn Out BBQ. Feel free to answer this even if you don’t live in Atlanta. If I get a large enough response from a community we’ll figure out how to hold a BBQ there too. There’s also a fair chance that some cookies and a picture in the mail will do you more good than you realize.

    Please don’t be shy.  You’re in a community that really does care and wants to help. If you prefer something anonymous the Suicide Prevention Life Line is available. Even if you’re not at risk and just want to talk to somebody you can reach out to them 24 hours a day, 7 days a week: 1-800-273-TALK

  • Expert, like Master, can be a Dangerous Word

    Expert, like Master, can be a Dangerous Word

    William Sealy Gosset was a mathematician that published while working at Guinness. The company was concerned about trade secrets being leaked so all brewers had to write under a pseudonym. Given he was publishing strictly for academic purposes, he decided to write under the name “Student”. If you have ever taken a statistics course you’ve probably used his Student’s t-test. I couldn’t find a completely consistent account of why he selected the name, but Student seems to have been chosen in the spirit of continuous learning.

    I often confuse the L in the CALMS model for Learning, but it’s actually Lean. At first glance it may appear that continuous learning and lean practices are in opposition of each other, but in fact continuous learning is an operational expense of implementing and maintaining lean practices along with devOps as a whole.

    The Old Way of Learning

    Up until the 1900s brewing was a trade that operated off of an apprenticeship system. I’ve known a few people in apprenticeship positions and they were treated horribly for the most part. If they swept enough floors and cleaned enough toilets they may graciously be given the privilege of getting tenuously involved with some side work. Often the hazing got pushed to the point that the apprentice quit before they were given any work that wasn’t custodial in nature. Only a couple of times did the expert realize the apprentice left because they weren’t being respected. Most of the time it was chalked up to an inadequacy of some sort.

    There was some of this mentality at Guinness. Newly hired positions were instructed that if they were lucky enough to meet a brewer in the hallways they should not identify them but lower their eyes until they had passed.

    If you’re in an toxic environment you should quit and go work somewhere else if at all possible. I’ve heard it’s easier than you think from many walks of life, but Katherine Daniels (conveniently known as @beerOps) was a recent guest for a podcast on developing careerOps with @SoberBuildEng, @eciramella, @sacha_d, and @SonOfGarr on the The Ship Show. You should check it out for additional information on finding the best fit position and employer for yourself.

    Transitioning to a New Way

    I’m guessing this brewer centric culture at Guinness had caused enough problems for Cecil Guinness to get sick of it. With the assistance of the managing director at the time, a Mr. Christopher Digges La Touche, Guinness started hiring STEM graduates from Cambridge and Oxford and making them brewers after a two year apprenticeship. They had decided to make the brewing of beer scientific.

    The culture and environment sound like they belong to a start up. These newly hired brewers lived together in a house at St. James’ Gate and spent a lot of free time on team outings. A group of really passionate people with similar interests will start to brainstorm and collaborate if they spend enough time together. They started to figure out ways to measure properties of the raw materials used in brewing that had always been qualitative in the apprentice program, such as subtleties in the grains that had to be learned from experience under the tutelage of a master brewer.

    I’ve never brewed any beer and I have no idea what a brewer would mean by the rub of the hops or the texture of the barley, especially when described as milky or steely. The real problem with subtle nuances that are driven from intuition instead of data is the difficulty in comparing results. I certainly couldn’t explain how one barley was milkier than another barley. By identifying a method of measuring these properties consistently and accurately the brewers were then able to compare different sourcing options empirically instead of off their gut. It also helped the brewery identify how to store their materials until needed while maintaining the quality of the ingredients.

    Op-ex Well Spent

    The discovery process wasn’t cheap or easy. Facilities and laboratories were built. Tours of fields and networking events where brewers could meet and converse with the farmers and maltsters where arranged. Outside of finding the right hops to grow they studied the cost associated with producing them. I suspect it was worth the cost of growing their hops if they could control the quality consistently. Guinness eventually bought enough farms for these young ‘brewers’ to experiment with that Guinness became one of the largest hop growers in Ireland and England.

    Not only was this expensive, but it was culturally difficult. Getting brewers and farmers to change traditional practices that had been handed down over generations is not easy. Especially when your reasoning depends on science and mathematics that aren’t always easily understood.

    Good Experts Collaborate

    Each of these young ‘brewers’ did become a resident expert that could help the other ‘brewers’ when they encountered a problem in a specific area. Not only was Gosset good at listening to their concerns but he made sure he helped them find a best fit solution they understood. He even built tools to help solve their problems when needed, the t-test being the most famous among them.

    A maltster named E.S. Beaven was a bit of a Ron Swanson and had no interest in statistics or statisticians. Gosset was so fascinated with his skill in cross breeding barley though that he eventually overcame the social hurdle of being a mathematician and became close friends with Beaven. Gosset was then able to help him set up several nursery experiments, including a group of experiments where farmers in different areas swapped seed to see how the yield varied with agricultural methods and environment.

    These young ‘brewers’ Gosset was working with were good experts. They were always learning more about brewing by taking measurements and sharing their findings, even if they had to do it anonymously. This was all done in an effort to eliminate waste and improve quality. I’m not sure how focused they were on automating everything, but this was just before the time Henry Ford’s assembly line started rolling. Automation probably came earlier at Guinness. A Brewery, unlike many other factories, can power automation using only gravity instead of steam or electricity. Scaling one up is easy to do compared to something like the Ford factory.

    The First DevOps Engineers

    I see the CALMS model being followed here. I see a systems perspective being embraced, feedback loops being completed, and a culture that understands the connection between experimentation, failure, and eventual improvement. This is consistent with the Three Ways. Was Guinness an early (albeit unaware) adopter of devOps? In my opinion yes. Not only was Guinness transcending industrial engineering by running one of the first devOps incubators in the 1900s, but Gosset and his group of young ‘brewers’ would be more aptly titled DevOps Engineers.

    DevOps requires production. It is a lot like industrial engineering, but the current revival of these practices under the banner of devOps is a result of the rapid changes in technology we’ve seen. Viable applications can be provisioned at scale using these techniques, but this isn’t a job just for software companies any more. If you produce or sell something and aren’t optimizing from a systems perspective, leveraging technology to automate and measure where ever you can, and keeping your employees and customers satisfied in the process then your business could probably be making more money.

    There Are Bad Experts Too

    I have walked through the debate on hiring devOps engineers with a few folks. Being a devOps engineer myself, I obviously think there’s a place for it and it can have merit. Growing a devOps advocate internally can be really difficult because of history, baggage, and biases that are already in place. A devOps engineer, much like Gosset and his buddies, really needs to have a fresh set of eyes on the work flow and where the bottlenecks, waste, and quality concerns are located. Guinness was smart to hire more than one of them though. If you’re going to have experts or masters in your organization you should have more than one expert in any single subject.

    I’ve encountered a lot of resident experts in enterprise sized companies that have become the single source of truth on a particular subject. When brought a problem that needs a solution it’s very easy for them to declare it “as designed” or a job for someone else because it’s not quite specific enough for their level of expertise. If there’s no one to give a second opinion on the matter how can they ever know when they’re wrong?

    I had a boss that used to tell people, “A Master electrician can still change a light bulb” when they complained someone was doing work outside their area. If people are showing up with problems and someone can easily help them they should help them. If your expert isn’t getting their job done then their customers are going to take the demand somewhere it can be met. A good expert should have the awareness to collaborate with someone who can help them manage their workload instead of expecting people to wait or just deal with support issues.

    Which is why Gosset and Beaven could become friends. Beaven was truly a master maltster and an expert in cross breeding barley. Gosset respected this and wanted to learn from him. If Beaven had been mediocre at his craft I doubt Gosset would have cared what he thought of statistics, but when Beaven saw the mathematics was a tool Gosset used to understand and confirm why he was good at cross breeding barley he then became interested in it. A demonstration of utility is the best sales pitch you can make, as seeing is believing.

    Collaboration Makes the Difference

    A good devOps Engineer must become an expert in many things. It helps us help each other, it fosters empathy, it keeps our career options open, and it keeps us honest. If we get bored or frustrated with a job where there’s no traction then we can take our diverse set of skills and do something else. Sometimes this can even be done in the same company. Most companies would rather re-purpose a talented individual if they can move them into a role they can perform with some passion again than see them leave. There’s strength in diversity and it ensures continued survival, even when talking about what you do day to day in the line of your work.

    If you’ve landed in a position of absolute authority you run the risk of degenerating into a bad expert that’s mostly sitting around, punching a clock, and telling people to get lost instead of helping them. It’s not that you’re a bad person, but absolute authority corrupts absolutely and you’re still human. If there’s any question whether you might run this risk yourself then find a colleague you can collaborate with that can help you learn as much as you teach in complementary subjects. Find a Gosset to your Beaven (or a Beaven to your Gosset). Cross training will help manage your work flow but it’s a balancing act between optimizing hand-offs and individual task transition. Still, if you diversify your skill set it will definitely be good for your own self-preservation in your current role and in your next one.

     

    SOURCES/ADDITIONAL READING

    Guinness, Gosset, Fisher, and Small Samples
    Author(s):
    Joan Fisher Box
    Source:
    Statistical Science,
    Vol. 2, No. 1 (Feb., 1987), pp. 45-52
    Published by:
    Institute of Mathematical Statistics
    Stable URL:

     

     

    Guinness’s Brewery in the Irish Economy 1759-1876
    By Patrick Lynch, John Vaizey

  • Trolling PuppetConf: Surveying Configuration Management

    Trolling PuppetConf: Surveying Configuration Management

    Troll (Internet)

    In Internet slang, a troll (/ˈtroʊl/, /ˈtrɒl/) is a person who sows discord … by starting arguments or upsetting people,[1] by posting inflammatory,[2] extraneous, or off-topic messages in a… community …with the deliberate intent of provoking readers into an emotional response[3] or of otherwise disrupting normal on-topic discussion.[4] 

    -Wikipedia

    When I stopped by the Chef booth at Velocity Santa Clara this year Nathen Harvey was out of XL t-shirts. I happened to have a Puppet shirt on at the time too, but it was just a lucky coincidence. Nathen’s always advocated using something (anything) to automate your infrastructure, as the ‘A’ in the CALMS model for devOps stands for automation, but the look on his face when I crossed his line of sight was as delightfully awkward as it could have been. Nathen told me he’d get me a Chef shirt if I got him my address back in Atlanta. I had been meaning to do it before I made it out to PuppetConf, but a tweet from Mandi Walls ultimately ended up prodding me into action.

     cheftroll

     To Nathen’s word, he got me a shirt in under a week, and in plenty of time for #PuppetConf.

    chef

    chefShirt

     

    I think “Stirring up delight” was the right shirt to send me too. I was half worried I’d get a shirt that was much more confrontational, putting me at risk of getting jumped and having my lunch money stolen. Lucky for me pretty much everybody that noticed the shirt thought it was funny. At least anyone who said anything to me about it. A proper troll is going to get at least one person a little bit angry though, right? Anyone that recognized I was in a Chef shirt happened to create an opportunity for me to ask a few questions, so I did.

    1. Have you used Chef, Puppet, CFEngine, or any other configuration management tools?
    2. Which do you prefer?
    3. Why do you prefer that particular tool?
    4. Why does the competition between these tools help or hurt our industry?
    5. Where do you see these tools in five years?

    I got pretty consistent answers just talking to people, and the Q&A often evolved into a natural conversation, which I prefer. (Another devOps principle I like to practice is shut the hell up and listen). I also put up fliers with a link to a survey while at the conference in an attempt to gather some empirical data. Truth be told, the survey was the real troll.

    One of the best talks I saw at PuppetConf was by Lindsay Holmwood, call The DevOps Field Guide to Cognitive Biases. I wasn’t familiar with the term before the conference but I think I had the idea. I also know I still don’t fully understand it. Watch the video and check out the books Lindsay suggests at the end. It’s pretty deep stuff. I really like his advice on having engineers manage and managers engineer to help break these biases down, as it creates a lot of empathy between the two groups.

    Given my own cognitive biases, I wouldn’t have filled out the survey if I had seen one of the flyers I posted. The flyer said nothing about a home school project for twin girls, which would have had a much better chance of getting my attention. Instead, it just asked folks to fill out the configuration management survey with the promise of cookies for their trouble. I’m pleased to say the tech community will respond to surveys for cookies even when crudely marketed. While I have enough responses to do sound statistical analysis, I know my data isn’t sufficiently stratified and probably doesn’t represent the larger population of the configuration management community. In other words my data lacks diversity.

    My daughters’ home school project is to start a business, and the twins decided to launch Gemini Snacks. Their company is pretty much an online bake sale to allow them to give back to all the hospitals and organizations that have helped them and cheer up the kids that can’t go home yet.

    The deal I made with my daughters was simple. I would print out flyers for PuppetConf to get survey responses to help me write this article. I would thank as many people as we could for filling out the survey by sending them some cookies, and I would hire their company to bake and ship those cookies. If people filled out the survey my kids would be in business.

    There was a strong enough trend in the responses that I can discuss those now with confidence despite a lack of diversity in the data. Of course I got a lot of responses from Puppet users, but I also heard from people using CFEngine, Chef, Ansible, Salt, and bconfig2. A lot of people weren’t using anything and just recently picked up Puppet.  I ran into a few folks that had used three or more tools at one point or another and could give comparisons which were really interesting. Here are the main conclusions I’ve been able to draw so far.

    A Rising Tide Floats All Boats

    Most organizations are still sshing into a lot of hosts to handle things, sometimes they do it with scripts. Configuration management (CM) tools are still becoming a fully adopted standard. I have heard users of every CM application in the market say it really doesn’t matter what tool you’re using, automate your infrastructure with something. As a result this is still very much a growing sector in the tech industry.

    Don’t get into a Configuration Management Bake-Off

    Bake-Off is the word that was used, and more than one person put it that way specifically. I don’t know what you’d call setting up test environments to try different CM tools before making a decision, but I’d call it a bike shed of yak shaving myself. Bake-Off is probably more familiar people anyway. My opinion is you should either be using Puppet or Chef, but if you’re not using any configuration management tool at all it’s advised that you start.If you can’t help but take more than one for a spin before committing then keep it short and sweet. You’ll be better off taking the time to commit to one and put the work into making the transition quickly than trying to figure out the easiest way to do it. This automation will take some effort. The key thing to understand is the right tool depends on your environment.

     

    There’s a Lot of Respect and Appreciation for CFEngine

    CFEngine is a trailblazing academic project that gained a lot of popularity in the early days of CM. The adoption of the software in the open source community really proved the viability of CM as we know it. While the application isn’t seeing continued growth now that there’s competition with newer tools in this space, several people went out of their way to emphasize the immense respect they have for the product and its founder, Mark Burgess, as CM wouldn’t be as evolved without them.

     

    Puppet: Preferred by Ops Folks because it’s Easy to Configure

    The main thing the ops crowd seems to prefer about Puppet is that it’s easy to configure. I had a book recommended to me  at the conference called, “Switch: How to Change Things when Change is Hard”, by Chip and Dan Heath. The book describes a “Project Mood Chart” used by IDEO, a product design firm, to brace their clients for adversity.

    productMoodChart

    When a new project starts we’re optimistic and hope for the best. We inevitably face adversity during the course of our work, and while this adversity is always frustrating it does lead us to insight. Empowered by our new perspective on the problem we can proceed to a viable solution with confidence.

    I believe this is how Puppet evolved out of CFEngine. Luke Kanies, the founder and CEO of Puppet, was an early contributor to CFEngine. I can see Luke following this curve, gaining insight on the configuration difficulties the CFEngine community was experiencing, and going on to found Puppet, confident he had a product better adapted to industry since it was easy to configure.

     

    Chef: Preferred by Devs because it Allows them to Treat their Systems Like Objects

    I can see this same curve being followed as Chef came out of Puppet as well. With ease of use comes a lack of robustness. Puppet’s advantage over CFEngine is it’s easier to configure, but there seems to have been a point of frustration within the development community that lead to the creation of Chef. Developers like to treat things like objects, which should be obvious if you have any experience with object oriented programming. Abstracting the complexity of the software we write so it can be configured by end users that are not developers is challenging. Since Chef allows its users to treat their systems like objects, the difficulty surrounding the abstraction of any complexity into a declarative model is circumvented.

    Which is the main reason ops folk don’t like Chef as much, because they’re not programmers. If you’re in an organization with a mature operations department that’s not so interested in learning to code, Puppet ‘s declarative model for describing systems is going to be a better fit. However, if you have developers doing most of the operations work out of necessity in a small shop or an operations group that’s doing more software development for internal tools, especially if they like Ruby, Chef may be a better fit for your organization.

     

    Ansible: Symbiotic Orchestration

    The best things anyone had to say about Ansible is that it’s great for orchestration. It seems to be competing more directly with mcollective, a framework Puppet Labs supports for orchestration tasks. Mcollective can be used with both Chef and Puppet, and the same appears to be true for Ansible. Ansible is also agentless, managing nodes over SSH. Depending who you talk to this can be a good or a bad thing. By not running an agent the nodes aren’t dealing with the overhead that would come with it, but you’ve also lost the ability to monitor your systems as tightly as you could if they were running an agent. I’m under the impression CM is trying to move us away from SSH loops. While I don’t think it’s fair to write Ansible off as an overblown SSH loop, I am of the opinion that the advantages provided by agents running on each node in your environment are well worth the overhead. It will be interesting to see if Ansible’s relationship with other configuration management systems continues to evolve into a symbiotic orchestration provider or if another product, like mcollective, continues on to fill that void.

    Salt Isn’t Really Configuration Management Software, but it’s Fast

    Salt is a remote execution platform build on zeroMQ that was designed with speed in mind. Everything salt’s being used for, including CM, has been an afterthought to the fast remote execution platform at the heart of the project. The constant connection the nodes (called minions) maintain with their master definitely contribute to the speed of the system. This would lead me to believe Ansible is the slowest of the four technologies discussed here, since it doesn’t run an agent at all, and Chef and Puppet would fall somewhere in the middle since they run an agent on their nodes but it isn’t on constant communication. (This is an educated guess on my part. Please let me know if you can speak to the speed concern from personal experience.)

    I have run into a few businesses that deal with high speed financial systems that use salt for a lot of things, including CM, but only because it’s fast. If you have systems that demand split second responses to failure and change, salt may help solve a lot of problems your organization is currently facing. I’m not convinced it’s that easy to use.

    Salt seems popular with groups that want to avoid Ruby like it’s the plague or folks that are so horribly comfortable in a CLI that using a GUI just seems unnatural. I think these are terrible reasons to select a tool, since it’s directly avoiding the ‘adapt or die’ mentality that the devOps community embraces. Still, depending on your environment and the adversity you’re dealing with, these may be entirely reasonable grounds to use salt instead of another technology. I’d be less concerned if you told me you picked salt because it’s fast though.

     The Survey is Still Open

    I hope you like what you read here but I’m sure somebody doesn’t agree with me at all. The good news is you can still fill out the survey and let me know. When my data looks sufficiently diverse I’ll publish another update, and there’s a decent chance you’ll get some cookies for you time too. I’d also like to thank everyone that took the time to discuss this with me. Gene Kim, John Willis, Chris Webber, Charles Johnson, Dawn Foster, Kara Sowles, everyone at Puppet Labs, and of course Nathen Harvey and Mandi Walls, all helped to make PuppetConf a huge success and a memorable experience for me. I’ve learned way more than I expected from the experience and I’m excited to see how the competition in this space plays out in years to come. Competition is a good thing. As different challenges arise and as projects go through mood swings I believe we’ll see two things happen: The communities for these tools will adapt them to adjust to the environment they’re operating in, and at some point enough adaptation will lead us to a new set of tools all together. I for one am really excited to see what happens.