Tag: all day devops

  • Why You Can’t Afford to Miss All Day DevOps on October 10th

    Why You Can’t Afford to Miss All Day DevOps on October 10th

    In a tech world full of events and conferences, one stands out, not just for its longevity, but for its global influence and accessibility—All Day DevOps (ADDO). Since its inception, ADDO has become more than just another virtual conference; it’s a cultural touchstone for the DevOps community. On October 10th, ADDO returns with its signature 24-hour “follow the sun” format, proving once again that DevOps is more than a buzzword—it’s a movement that never sleeps.

    A Global Stage for DevOps Excellence

    For those unfamiliar, All Day DevOps was one of the first large-scale virtual DevOps conferences, launched when most people thought virtual meant second-rate. Today, it’s the grandaddy of DevOps events, boasting a platform that connects people across every time zone, allowing them to tune in whenever they can, from wherever they are. It’s about democratizing access to cutting-edge insights that were once limited to those who could travel to physical events.

    This year, the lineup is stacked with must-see sessions that reflect the urgency and evolution of DevOps in the current landscape. More organizations than ever are grappling with software supply chain security, AI’s rapid integration, and the balancing act between innovation and risk in open source. If you’re in DevOps, security, or software development—or simply interested in how these forces shape the future—you’ll want to mark your calendar for ADDO 2024.

    A Few Sessions You Can’t Afford to Miss

    While there’s plenty to take in over the 24 hours, some sessions will set the tone for the year ahead in DevOps. Here’s a taste:

    • 3:00 – 4:00am ET: The Invisible Threat: Open Source Malware is Silently Sabotaging Your Software

    Open source has always been a double-edged sword—rich in innovation but fraught with vulnerabilities. This session digs into how malware in open-source projects is becoming an invisible threat to your software supply chain. If you’re sleeping on open-source security, this is your wake-up call.

    • 8:00 – 9:00am ET: Revolutionizing Software Development: The Critical Balance Between AI and Human Insight

    In an era when every tech headline seems to scream “AI,” this session tackles the reality of AI’s role in software development—not as a replacement for developers, but as a tool that needs to be wielded with human insight. Striking this balance could be the key to staying ahead.

    • 11:00am – 12:00pm ET: A Decade of Transformation – Unveiling the 10th Annual State of the Software Supply Chain Report

    One of the most anticipated moments of ADDO 2024: the unveiling of Sonatype’s 10th Annual Software Supply Chain Report. This isn’t just another report—it’s the pulse check on how supply chain security has evolved over the past decade, and where it’s headed next.

    • 1:00 – 1:30pm ET: Open Source in Financial Services: Balancing Innovation and Security in a Highly Regulated Industry

    Financial services are often seen as late adopters of open source, but this session shows how they’re balancing innovation with security in an industry that simply can’t afford to take risks. It’s a compelling look at how even the most regulated sectors are embracing open source.

    • 4:00 – 4:30pm ET: Achieving Quality at Scale: Revolutionizing QA with GenAI

    Quality assurance at scale is hard enough without the looming pressure of accelerating release cycles. This session promises to demonstrate how Generative AI (GenAI) can become a game-changer for QA, achieving high quality without compromising speed.

    Why ADDO Still Matters

    In a tech landscape cluttered with conferences, All Day DevOps holds a special place for one simple reason: it consistently delivers on its promise of community-driven, relevant, and actionable content. Year after year, industry veterans and newcomers alike find value in ADDO’s free, global, and virtual format—proving that physical distance or cost should never be a barrier to learning and growth in DevOps.

    This year’s focus on AI, software supply chain security, and open-source innovation reflects the real-time challenges practitioners are facing in an environment that is rapidly changing. More than just theory, ADDO offers concrete solutions and case studies that you can apply in your organization today.

    For many, All Day DevOps has become an annual touchstone, a way to measure how far the DevOps movement has come and to gain insight into where it’s going. Whether you’re new to the field or have been part of the DevOps revolution since the beginning, ADDO is the place to connect, learn, and get ahead of the curve.

    Join the Global DevOps Movement

    So why not tune in? ADDO 2024 kicks off at midnight ET on October 10th and runs for 24 hours straight. With the world waking up to new challenges every day, DevOps is more important than ever—and so is this event. You can register for free at https://alldaydevopsy.kinsta.cloud and be part of a global movement shaping the future of technology.

    Don’t just take our word for it—log in, learn, and see how ADDO continues to drive the conversation in DevOps forward. I will be on line for sure.

    Also, shameless Techstrong plug, we are featuring our newest video series called “From the Source,” featuring Sonatype CTOs Brian Fox and Ilkka Turenen talking about DevSecOps, software supply chain security and SBOMs around many of these topics. You can watch on techstrong.tv, or it’s coming soon to the podcast platform of your choice.

  • DevOps Chat: All Day DevOps Preview

    DevOps Chat: All Day DevOps Preview

    All Day DevOps, the virtual event featuring more than 100 sessions from practitioners around the world, is happening very soon. Now in its third year, this year’s event will take place Wednesday, Oct. 17, starting at 3 a.m. Eastern. But if you don’t feel like getting up, each 30-minute session will be available on demand as well.

    In this DevOps Chat, All Day DevOps founders Derek Weeks and Mark Miller, both also from Sonatype, offer a preview of what’s in store during this year’s virtual event, which is free to attend.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Audio

    Transcript

    Alan Shimel: Hey, everyone. It’s Alan Shimel, devopsy.kinsta.cloud, and you’re listening to another DevOps Chat. Really happy to have two friends of mine join us on the chat today. We have none other – those crazy guys from All Day DevOps and Sonatype, Derek Weeks and Mark Miller. Mark, Derek, welcome to DevOps Chat.

    Mark Miller: Thanks, Alan.

    Derek Weeks: It’s good to be here. Always happy to chat with you.

    Shimel: Yeah.

    Miller: Thank you, Alan. Welcome to New York, welcome back home.

    Shimel: Yeah, it’s good to be home. So Mark, you know, this is an audio only, people don’t know, but you have a Fighting Okra hat on, is that correct?

    Miller: You know, some of my followers from around the United States are sending me hats and right now the Delta State Fighting Okras have sent me a hat so I’m gonna be giving them a good callout during All Day DevOps. Derek, you haven’t seen this one yet.

    Weeks: No, I haven’t. That’s pretty cool-looking.

    Shimel: All right, a shout out to the Fighting Okras.

    [Laughter]

    But Mark, you mentioned All Day DevOps. We’re getting close, right? What’s the date?

    Miller: We’re next – we’re gonna be on Wednesday, October 17, and as we looked at this morning, as we looked at it, we’re over 19,000 registrations this morning. Once again, the community has gotten behind it. Derek, you’ve got a bigger hand on it, too. You’ve got some ideas about what’s going on with the viewing parties even, right?

    Weeks: Yeah, I think the – you know, we’re at 19,000, I think we’re easily headed over 30,000 by the time we get to the conference, and I think part of that is what we’ve seen with the viewing parties this year. So, you know, it’s an interesting kind of evolution of the conference where we started this thing as, hey, let’s have this virtual conference so that everyone can attend wherever they sit, and then people said, “You know, I want to participate but have others in my company or my community participate as well, so we’re just gonna gather in a place to have this viewing party.” In the first year that we did it there might’ve been 10 because it wasn’t a formalized thing. Last year we kind of encouraged people to do it but didn’t have anything on the site that really mentioned it too much, and I think we had 35 viewing parties.

    This year we have 115 viewing parties that have registered in 24 countries, and the interesting thing is not only has that changed the conference and how it evolves and how people share this education within groups but we’ve heard some amazing stories just come out of this community where organizations like RBC in Toronto, they’re planning for up to 1,400 people in RBC to attend their viewing party. State Farm in Dallas has 500 people that they’re planning in their viewing party. National –

    Miller: They went up to over 800 this weekend.

    Weeks: Yeah. National Australia Bank; you know, they’re planning, I think, 250 people in their viewing party. And when you think about the size of some DevOps conference that all three of us have been to, anyone having 500 people at a DevOps conference has a successful DevOps conference. The fact that one company, you know, is setting up a viewing party just for our conference that’s as large as some DevOps conferences out there it’s pretty interesting. But, you know, the same point – and this is a conversation I had with someone this morning in Munich, he said – you know, we’ll get into talking about Club 20 – but he said, “Hey, do we have 20 people registered yet because we want to get our company into Club 20,” and I hadn’t heard of this organization really before.

    And I looked; they had 16 people. Now they actually have 20 so they’ll make the list. But I said, “I’m curious; how many people in your organization could attend All Day DevOps that, you know, are DevOps practitioners?” And he said, “You know, like I want to get on the Club 20 list, but I think we could get 100 people at our viewing party and in a perfect world we could have 1,000 people participate in All Day DevOps.” And that – you know, Mark, as we started this whole thing, that was really the aim, like how do you get to those 1,000 people and give them some free education?

    Miller: Yeah. One of the topics that you just touched on – and you and I talked this morning too as we were putting together how our day was gonna go – Alan, when you think about it, can you imagine 20 people showing up at a live conference? I mean think about that. If you get four or five people at a live conference from one company you’re blowing out the budget, right, already. In this case, we can literally allow dozens and dozens of people from the same company to attend 120 sessions in a live conference. It’s a pretty phenomenal way to look at it.

    Shimel: And, you know, so one way of looking at it, Mark, is wow, what would the cost be to fly in a dozen, two dozen, three dozen people to a physical conference? The other thing, you know, to make it a truly win-win is what are the benefits of a team that size sharing that common experience, right? It’s almost priceless, right? It’s like the MasterCard commercial: $45 for the conference ticket, $60 on Uber, $20 on lunch. Sharing it as a team? Priceless.

    Miller: Yeah.

    Shimel: And that to me is the real one plus one equals three of it.

    Weeks: Yeah, I think – and that’s something that we see – you know, as we’ve seen the past two years that we’ve done this and I think we’ll see again in spades, is that we have the Slack channel set up during the conference for conversations, and last year during the 24 hours of the conference we saw 30,000 conversations on Slack, and that ability to exchange information, whether it’s with people on your own team and you have your internal Slack channel set up through your own company because, you know, you’re State Farm and that’s what you did and how you shared information, or you’re participating in the global channel, you know, I think it’s all about that kind of exchange of information and also the anonymous – somewhat anonymous level of that communication.

    We all know it’s a lot easier to ask a question in a conference if no one can see you’re the person raising your hand or asking the question; you’re just one of many on a Slack channel saying, “Hey, I have this question,” and there’s – you know, it opens people up to sharing a little bit more or asking questions they might not otherwise get to ask.

    Shimel: Absolutely, absolutely. Guys, I want to turn the conversation a bit if I can from numbers and viewing to content. Let’s talk – you know, people out listening to this, some of them may already be registered, some may not, some may not even know of All Day DevOps, right?

    Weeks: Yeah.

    Shimel: Mark, if they register – and registration’s free. You don’t have to be part of Club 20, you don’t have to have your whole company. If you’re just one person who’s digging DevOps you can register for free for All Day DevOps and sit in on 24 hours – up to 24 hours of DevOps, right? They don’t have to listen to all of it; you pick and choose what you want. But give us some of the highlights. What are some of the big hot topic speakers that you’re looking forward to?

    Weeks: Mark, why don’t you go –

    Miller: One of the things that you just – sure. One of the things that you just said is interesting to me, Alan. You said 24 hours of DevOps. Actually, when you do the math it’s more like 50 to 70 hours of DevOps because we have simultaneous channels happening. The things that I’m excited about seeing is, one, we have David who runs a CRE team and an SRE team at Google coming to talk about site reliability engineering. That’s a new topic for us this year, and to have Dave step in from Google, who is actually running that whole initiative for Google, it’s gonna be an exciting one for me. The other one that I’m – how do I say it?

    I’m exciting that Cindy’s coming, Cindy Healy. She is the one that wrote some of the code that went on the Mars Pathfinder to send to Mars. Now that’s – the personal story behind how Cindy did that and how she became involved in it, what’s she’s doing now to encourage women in technology, that is a story that I really want people to be able to hear, so we’ve got her set up as a keynote. Derek, you’ve got some interesting things that you’re looking forward to, right?

    Weeks: Yeah. You know, one is a conversation I just had last week with one of our keynote speakers, George Swan. He’s at Autodesk. Super interesting story of DevOps in the enterprise. So these guys knew, you know, the needed to transform the entire company – and I think they have about 200 products, 4,000 developers in their organization, and they needed to transition from an Agile organization to an organization that had DevOps practices top to bottom. At the same time that they were doing this they needed to transition into the cloud. They also needed to transition from a traditional license business to a subscription license business.

    They had, like, four or five transformations going on at the same time – you know, become cloud-based infrastructure, transition to SaaS, transition to DevOps and some other things, and it was like, oh my gosh, one, this is a multiyear journey he’s gonna tell his story on, so it will be fascinating to kind of get lessons learned and also understand it’s not just hey, you know, transition to DevOps; it’s there are a lot of other things that are moving parts of our business that have to change, you know, along with this as the marketplace evolves for them and other participants. The other one that I’m really excited about, and a guy that I’ve got to see a couple of times at live conferences, is Navin Vembar. He’s the CTO over at GSA, the Government Services Organization.

    They do all the main procurement for the U.S. government. But he is going to talk about how they transformed their organization into a DevOps practice and kind of looking at the maturity models that they use and kind of used as milestones for measuring their progress within their organization and also, you know, coming from an organization that loves silos, they love regulation, [laughs] they overly love governance sometimes, but to hear his journey, he’s a fascinating speaker and someone that I quote often, so always looking forward to hearing people like that join in.

    Shimel: Sure.

    Miller: Alan, the other thing is as Derek was talking I went over to the All Day devopsy.kinsta.cloud site and clicked the speaker link. We’ve got 124 speakers this year. We had so many good submissions that we actually went above our quote. We had set it at 100 and we’re at 124 right now. It’s hard to go and cherry pick out of those. What we want to make sure people understand is two things; No. 1 is we want to bring 124 practitioners to everybody’s desktop. We want you to be able to pick and choose what you’d like to hear.

    No. 2, Derek and I guarantee to the community is there are no vendor pitches here. Alan, you’ve been with us long enough you know. We really hold to that creed, that what we want is practitioners, and when you come here I think that you can trust us to provide that to you.

    Shimel: Absolutely. That’s another aspect I want to just highlight too, guys. You know, we – you both highlighted what – some of the ones you’re – some of the talks you’re looking forward to most. Well, Mark, there’s 124 as you mentioned. All Day DevOps is a global phenomenon, and though we may have GSA and we may have some U.S.-based practitioners, Mark, I’ve been around the world with you, right? You and I have traveled – [laughs] literally traveled the world, and DevOps is a worldwide thing, and I think the speaker lineup reflects that. Have you had any stats, like how many different countries are represented?

    Miller: Yeah, Derek’s got that, but before he jumps in with the stats I got a story that you’ll recognize. You and I went to Singapore, Bangkok, Sydney and Melbourne about two months ago.

    Shimel: Yep.

    Miller: While we were having dinner with Rob England in Sydney, I went, “Man, you’ve got a great story, Rob. Why don’t you speak at All Day DevOps?” Meaning because of our travel, because of our ability to meet people in the community, it gives all of us the ability to say this guy’s got a story that deserves to be told, and I think that that’s one of the things when Derek and I were first putting this together we actually talked to – amongst ourselves and said what we want to do is give exposure to people that are doing DevOps that deserve more recognition for what they’re doing, and our global platform has allowed us to do that. Derek, how many countries are we dealing with now?

    Weeks: I think, you know, we’re around 130 different countries participating.

    [Laughter]

    It’s amazing, you know, to see that list grow because some of these – you just imagine like who’s practicing DevOps, and in my head like maybe 22 countries and maybe if I stretched it like maybe 40, right? But to see participants from all over the place – you know, and I just did a quick look at the registrations in the top cities. So No. 1, Bangalore, second, London, Toronto is high on the list, Melbourne, Australia; Sydney, Australia, Atlanta, New York, Istanbul, Chicago, Singapore, Amsterdam, San Jose, Seattle, Santiago, Jakarta.

    Shimel: That’s amazing.

    Weeks: You know, in the top 20, right? So it’s really, you know, a global kind of phenomena that we’re seeing. You know, where is the DevOps kind of center of expertise? Where are these people around the world? It’s pretty interesting to see where they all come from.

    Shimel: Agreed. You know, Mark, listening to you talk –

    Miller: I don’t want to leave anybody out but when we –

    Shimel: Go ahead. You go, Mark.

    Miller: Go ahead, Alan, go ahead.

    Shimel: No, no, you go.

    Miller: Yeah, I just – I don’t want to leave anybody out, but there’s some names I’d like to give out a call to, is like Midu in India, who’s helped us so much with the community in India. Look at Bangalore; it’s No. 1 on the list. When you look at Michael Mann, you look at Mark Cluett in London; those are community people that have said this is something worthwhile and I want my community to know about it, and that’s kind of – I don’t know if heartwarming is the word, but it means that we have actually resonated with various communities, cross-cultural around the world, and that means a lot to me.

    Weeks: Yeah, I just – you know, curious, I just looked at the list of what cities have 40 people registered – Alpharetta, Georgia – that’s near Atlanta – Johannesburg, South Africa, Istanbul, Dubai, Barcelona, Seoul, Sao Paolo, and McLean, Virginia, where I’m sitting.

    Shimel: Seoul, I haven’t heard Seoul yet.

    Weeks: Yeah. So fascinating, just that list of only cities with 40 is extremely diverse.

    Miller: Wow.

    Shimel: You know, that’s really – that’s the power of the internet and it also speaks, Mark, to the power of DevOps, is if it was all centered in San Francisco or London, great. All the cool kids, right, are doing it. But it’s not; it is – it’s diversified, decentralized. You couldn’t snuff it out if you tried. You know, Mark, listening to you talk though a little bit about we met this one here and this one there, I can’t help but think of you as sort of the Anthony Bourdain of DevOps, venturing into parts unknown.

    Miller: You know, I’m a story guy. I gotta jump in there. When I was in Manila, we got Anthony Bourdain’s driver to take us around to all those places. It was fascinating.

    Shimel: And I’ve been on those trips with you. You like to off the beaten path. But, you know, this is a PG-rated show and, you know…

    [Laughter]

    Weeks: I’m gonna give Mark a new title of the Anthony Bourdain of DevOps.

    [Laughter]

    Miller: I’ll take it, I’ll take it.

    Shimel: Anyway, hey guys, we’re about out of time here, but I wanted to check in ’cause we are getting close. We would be remiss if we didn’t at least mention where can people go to register for All Day DevOps? Derek?

    Weeks: AllDaydevopsy.kinsta.cloud. It’s that easy. It takes 15 seconds to register. You’ll get a link to the schedule once you register so you can start to build your own schedule for yourself and your organization. And like we always say, once you register yourself encourage your team to register so that everyone can watch together.

    Miller: Alan, the last thing that I’d like them to look at on the site too, is once they register go to the viewing party page. With over 110 viewing parties there has to be something local, that’s relatively local, if you would like to participate with other people.

    Shimel: Fantastic.

    Weeks: You know, the other thing just really quick before we go is one way that we do this conference for free for everyone is we have a lot of sponsors, like devopsy.kinsta.cloud, Scaled Agile, Cisco, Puppet, Chef, GitLab, Sonatype, CloudBees and other organizations that help fund the conference to actually put this on and make this happen. We also have 245 community groups around the world that are helping us spread the word to say here’s free education for our members, and that’s really at the heart of this conference is the financial support to make it free for everyone and the community’s support to really spread the word to everyone that this is available to them.

    Shimel: Excellent. Absolutely, Derek. These – as someone who’s done enough conferences, it really – it doesn’t happen without the sponsors, and a lot of people say, “Oh, the sponsors just want to spam me or contact me.” Yeah, you can talk about that all you want but they make the world go round on these things, right?

    Weeks: Yeah.

    Shimel: We can’t do it without them and don’t lose sight of that.

    Weeks: Yeah. It doesn’t cost $10 million to run but it’s not free to run. [Laughs]

    Shimel: Exactly. But we all know that. Anyway, guys, two of my good friends in the DevOps – well, not only in the DevOps space, in the world in general. Mark Miller, who I have traveled around the world with and eaten in some very –

    Miller: Thank you, Alan, so much.

    Shimel: – fun places – Anthony Bourdain there – and Derek Weeks.

    Weeks: Thanks, Alan. Always a pleasure.

    Shimel: Always a pleasure, man. I haven’t seen you in too long.

    Weeks: Well –

    Shimel: But hopefully we’ll see each other soon.

    Weeks: Yeah.

    Shimel: All Day DevOps coming to a computer near you Oct. 17, right?

    Weeks: Yep.

    Shimel: All day.

    Miller: Starting at 3 a.m. New York Time, Alan.

    Shimel: Excuse me?

    Miller: Starting at 3 a.m. in New York. I’m sure you’ll be with us for the opening.

    Shimel: Yes, I’ll be up bright and early at 3:00. But if you want to register, go to AllDaydevopsy.kinsta.cloud, guys. It’s free. Jump on it. It’s a great event. Have a great day, guys. Thanks for joining us. This is Alan Shimel for devopsy.kinsta.cloud and you just listened to another DevOps Chat.

    — Alan Shimel

  • Reducing Risk in Applications Using Docker Containers

    Reducing Risk in Applications Using Docker Containers

    Docker. It seems like in this day and age you are either using Docker containers or you are going to use Docker containers. If you haven’t jumped on the bandwagon yet, check out a previous article, “Docker: The New Ordinary.” If you are on the wagon or are thinking about it but have concerns about their security, it’s time to read on.

    José Manuel Ortega (jmortega.github.io ) is a software engineer and security researcher in Spain. I recently watched his presentation online titled, “Testing Docker Images Security.” He gave an overview of typical Docker deployments, explained the attack surface and threats, presented how to detect vulnerabilities and outlined a couple of best practices. In short, his advice will help you learn how to better secure your Docker containers.

    (New to Docker? Read this paragraph; all others skip ahead.)  If you aren’t sure what Docker is, José offered this explanation: “Docker containers wrap a piece of software in a complete file system that contains everything it needs to run: code, runtime, system tools, system libraries—anything you can install on a server, regardless of the environment it is running in.”

    That is, containers are isolated but share an operating system and, where appropriate, binaries and libraries. Docker provides an additional layer of isolation, making your infrastructure safer by default. This makes the application life cycle faster and easier to configure, reducing risks in your applications.

    For starters, José laid out Docker’s default mechanisms for security:

    • Linux kernel namespaces
    • Linux Control Groups (cgroups)
    • The Docker daemon
    • Linux capabilities (libcap)
    • Linux security mechanisms like AppArmor or SELinux

     

    Screen Shot 2018-04-08 at 7.17.58 AM

    José walked through others tools, add-ons, best practices, etc., to increase Docker container security. I will cover most of them here.

    Docker Inspect Tool. The Docker Inspect Tool is built into Docker. It provides information about the host name, the ID of the image, etc. and it comes up when you start Docker.

    Docker Content Trust. It protects against untrusted images. It can enable signing checks on every managed host, guarantee integrity of your images when pulled and provide trust from publisher to consumer.

    Screen Shot 2018-04-08 at 7.19.01 AM

    Docker File Security. Docker files build Docker containers. They should not write secrets, such as users and passwords. You should remove unnecessary setuid and setgid permissions, download packages securely using GPG and certificates, and try to restrict an image or container to one service.

    Container Security. Docker security is about limiting and controlling the attack surface on the kernel. Don’t run your applications as root in containers, and create specific users for testing and policing the Docker image. Run filesystems as read-only so attackers can not overwrite data or save malicious scripts to the image.

    José provided a useful checklist to check the security of a Docker container, but it’s not a short one. Remember, if you are going to deploy hundreds or thousands of these containers, you’ll want to ensure consistent handling of security concerns to keep the hackers at bay:

      • Do not write secrets to Docker files
      • Create a user
      • Follow version pinning for base images, packages, etc.
      • Remove unnecessary setuid, setgid permissions
      • Do not write any kind of update instructions alone in a Docker file
      • Download packages securely
      • Do not download unnecessary packages
      • Use COPY instead of ADD
      • Use the HEALTHCHECK command
      • Use gosu instead of sudo whenever possible
      • Use no-cache (if applicable) when building
      • Enable Docker Content Trust
      • Ensure images are free from known vulnerabilities
      • Ensure images are scanned frequently throughout your DevOps pipeline
      • Ensure your images, packages are up to date
      • Use file monitoring solutions for image layers (if required)

    Auditing Docker Images. You can scan your images for known vulnerabilities with a wide variety of commercial and open source tools such as:

     

    All of these solutions can be integrated in one element of your CI/CD pipelines — some can be integrated in multiple places.

    José explored these solutions and best practices in more detail and offered up technical implementation tips in his full talk, available for free here. If you are working on or interested in Docker security, his talk is worth your time.

    Thirsty for more DevOps discussions led by practitioners (not vendors)?  You too can binge watch 99 other talks from the All Day DevOps conference here.

    — Derek E. Weeks

  • Challenges in Leading a DevOps Team at a Fortune 100 Company

    Challenges in Leading a DevOps Team at a Fortune 100 Company

    A manager, an evangelist and a godfather all walk into a bar.

    Okay, this isn’t a bad joke, this is an article about how managers can implement DevOps in large organizations, especially where culture and organizational change hamper efforts. But, I promise, we will talk about managers, evangelists and godfathers.

    Uldis Karlovs-Karlovskis (@UldisKK) works at Accenture Latvia. Accenture is a global consulting giant, employing 400,000 with teams and clients across the globe. Uldis helped bring DevOps into the organization and shared his journey—the good and the bad—at theAllDayDevOps conference in his talk titled, “Lessons in Leading a Fortune 100 Team to a DevOps Philosophy.”

    Uldis joined Accenture in 2012 and faced three top challenges:

    1. How to maintain quality at such growth?
    2. How to enable continuous learning?
    3. How to stay fast and flexible at their size?

    His team was small, they held daily stand-ups that were productive, and they were flexible. Life seemed good, and then his team grew into a “spider.” They still held daily stand-ups, but they were useless because it was just people reporting and no one understood what is going on.

    What did Uldis do? Well, he didn’t freak out, if that is what you are wondering. Actually, he did. One morning he, “woke up, realized I had 25 people under me, freaked out and skipped the job for the day.” He walked around, thinking about it and wondering what to do. He thought about his three to four types of DevOps projects and all of the non-project activities that were monopolizing his time. He realized he focused on the urgent over the important tasks, working 10 to 12 hours per day.

    What is a manager to do? Have people help him, of course. Uldis decided to create an unofficial role that sounds interesting: the subject matter expert (SME). They would work on non-project activities, increase team engagement, improve satisfaction and assess DevOps maturity in projects. Initially, people were motivated to be SMEs, but they were always busy with other things. So the effort failed.

    Uldis had a new idea: Instead of him leading the SME team, he let them decide who wanted to do what, which SME team they wanted to join and who would be the lead SME for each team, and let them set their own goals and how they wanted to work, etc. This helped, but still didn’t solve the problems. Uldis was still burdened with the tyranny of the urgent.

    Back to the drawing board, and a new idea was born: evangelists. Uldis broke down 50 SMEs into seven subgroups and added a little hierarchy because the flat structure wasn’t working. Each evangelist led a guild—one of seven. Team members were to talk to their evangelist before coming to Uldis with questions.

    But, alas, Uldis was still just handling escalations and talks.

    So, what do you do to bring order? Bring in Godfathers.

    Now, team members could always go to the Godfather for help. Uldis decided that, generally, each person would have four topics they may want to talk about. For each topic, they have a person to talk to:

    • SME addresses how to contribute to the company in general
    • Godfather addresses escalations, such as not happy with the team lead
    • Evangelist addresses questions about the person’s skills
    • Team Lead addresses project tasks

    Uldis now realized the benefit of this approach: In a way, now, everyone is focused at some level on sales. As we all know, this is important yet very difficult in any organization—but especially when everyone is an engineer.

    While his approach has helped him and his team, he also acknowledged, “If I did the same thing today, I would fail, but maybe for different issues.”

    Todat, Ulids is looking at improving his culture. As we talk about at AllDayDevOps time and time again, good culture is key to successful DevOps. Uldis, looking at Accenture, noted it doesn’t fit into a typical organizational structure. Reading literature, industry buzz, etc., he realized something was missing from these models, but he wasn’t sure what. He knew they needed to be better.

    He discusses three dominant types or organizational cultures:

    • Pathological – Eight hours of work and your job is done
    • Bureaucratic – Two to 12 hours of work if the process requires it
    • Generative – Two to 12 hours of work and I’ll deliver it

    Uldis is trying to be at the third approach, and the key to achieving it is knowing that if you are having a bad day, you should leave your work for tomorrow when you will be more productive. Tomorrow it may take more than eight hours, but you will put your best into your work.

    His journey is ongoing, but he finished his talk covering a few things they do on his team:

    • Onboard new joiners through a continuous delivery pipeline
    • Meet weekly at lunch
    • Half-yearly performance assessments

    He also provided four takeaways:

    1. Everyone by default is good
    2. Intrinsic over extrinsic
    3. Engagement is your work responsibility
    4. Let them lead

    Uldis told an honest and open story about his journey with the hope it will help others. To hear it firsthand, watch it for free here. You can watch any of the 2017 AllDayDevOps sessions free of charge here.

    — Derek E. Weeks

  • DevSecOps: Don’t Invest In Hope

    DevSecOps: Don’t Invest In Hope

    A successful DevSecOps approach is rooted in action, not hope.

    There is a lot of investment in hope.

    I hope we won’t get breached.

    I hope our DevOps teams aren’t deploying thousands of vulnerable containers.

    I hope our developers aren’t downloading millions of vulnerable open source components.

    I hope our developers and security teams will figure out how to work together.

    I hope we won’t be fined under GDPR.

    I hope the hackers don’t notice.

    Yet, the fact remains that hope will not reduce breaches. Hope does not safeguard your containers or components. Hope cannot achieve collaboration. Hope won’t prevent a fine. Hope is not a strategy.

    Many organizations are considering their approach to DevSecOps. They are moving beyond hope to strategy. They are choosing action over indifference. They are taking the first step of their journey.

    Here are three ways to start your journey:

    Picture This

    We all learn from others. Here is a collection of 20 DevSecOps reference architectures. They reveal the choices and priorities others have made ahead of you. Look at what they did. Choose to do something similar, or create your own path from a mix of their ideas. If your canvas is blank, use these to draw something.

    Watch This

    The pioneers are the ones with the arrows in their backs. Another word for DevSecOps pioneers is practitioner. And many practitioners have navigated their journey successfully enough to share it. More than 15 practitioners shared the tale of their DevSecOps journey during All Day DevOps. Every session was recorded online. Every session is free. Start with this one from DJ Schleen at Aetna and then pick your next one.

    Start There

    In a panel discussion I sat on recently as a prelude to DevSecOps Days at this year’s RSA Conference, I heard some very practical advice (find the recording here). One of the other panelists, Stephanie Derdouri from Fannie Mae, advised our audience to start with one thing. Don’t pick a bunch of strategies and tactics. Just pick one painful, suboptimal or annoying practice tied to security and improve that element within your DevOps practice.

    Pick only one. Start there.

    Hope is not a strategy. Action is your only safe option. Take the first step.

    — Derek E. Weeks

  • DevSecOps: Digging into Root Cause Analysis

    DevSecOps: Digging into Root Cause Analysis

    We have all been there in a postmortem when someone says, “Let’s get to the root of the problem.” And, we all know what that means: Who or what is to blame? We also all know that no one wants to play the blame game, yet we all do. But it isn’t our fault (no blame, see what I did there?). It has been the default system for solving problems in business for decades. It is called root cause analysis.

    We can change—for the better.

    I recently watched a presentation from Matthew Boeckman (@matthewboeckman) titled, “There is No Root Cause: Emergent Behavior in Complex System.” Matthew is a developer advocate with VictorOps and a technology strategist with Dryan.io. He grew up a systems guy and jokes that he has been in DevOps for 18 years, even though DevOps wasn’t around, because he has always been nice to developers.

    Digging in (pun intended), root cause analysis focuses on what went wrong, and how we can prevent it from happening again.

    null

    The core problems with root cause analysis for development is that it doesn’t provide for enough complexity and its natural focus is blame, which can undermine a positive DevOps culture.

    Root cause analysis was more applicable when waterfall was the development methodology because states stayed consistent for months or even years at a time. In the age of Agile, DevOps, CI/CD, microservices, etc., states of work are in a constant flux. Root cause analysis can’t provide solutions quickly enough. As Matthew notes, in root cause analysis, things are either good or bad, working or broken, uptime or failure. The reality is that our world is more nuanced.

    What Matthew recommends is to look at it through the principle of emergence because it, “separates judgment from the good and the bad binary approach to our system health, and instead focuses on behaviors and interactions, patterns and complexities of our system. With practice and effort we can manage them to more desirable states.”

    But what does this look like in practice?

    Getting back to the analogy of the tree and its roots, the answer is more of a forest than a tree. Trees are one living organism, forests are ecosystems.

    Matthew takes this philosophy and mental picture and gives us a better system: Cynefin. It is a Welsh word that means habitat, and was created by Dave Snowden (@snowded), originally for managing IBM’s intellectual capital. It draws on research in systems, complexity, network and learning theories.

    null

    Starting in the bottom right quadrant, working counter-clockwise, its goes from simple to more complex.

    Simple

    These are patterns or behaviors that don’t require a great deal of understanding. DevOps is increasingly setting up automated systems to respond to simple issues.

    Complicated

    These are known unknowns. You can imagine a set of realities where they can occur and they are probable, but not certain. For instance, a busy harbor might get a storm that causes damage to boats, docks, etc. It is hard for the harbor manager to manage and they need to think about it. This requires people to do some thinking, and it is difficult—if not impossible—to automate.

    Complex

    This is where we start to see emergent behaviors occur. We don’t have the metrics need to understand or manage these problems or you haven’t looked at that metric before. We start with probing, going into the system and exploring. Think of any collection of humans at any scale. Things are still in the scope of probable, but things change quickly. There are many moving parts that aren’t predictable and that we didn’t fully encounter in our test methodology.

    Chaotic

    This is, well, chaos. Matthews’ real-world example was when an entire region for AWS went down, causing other regions to be overloaded as system admins were moving services. In chaos, you act, then get a sense of where things are and then respond.

    Disorder

    In DevOps, this is where you have lack of communication and collaboration. Here teams need to: reduce: figure out what you agree on; analyze: build consensus; and, iterate: move to a quadrant and continue.

    Matthew notes that knowledge and practice move patterns toward more favorable quadrants. But, complacency erodes the process. Complex systems left poorly managed will create increasingly complex processes to manage.

    null

    How to adopt Cynefin

    • In the moment: Ask, what quadrant does this map to?
    • In the post incident report: How did we manage the pattern? Was it complicated, complex, simple? What can we do to change it?
    • In your sprint planning: Devote time to manage your patterns clockwise. What can we move with a little bit of work?

    null

    The reality is that root cause analysis is really only present after the fact. Cynefin calls us to action.

    Convinced that Cynefin might be just what your organization needs or want to dig a little deeper? Share and watch Matthew’s full talk here. You can watch any of the 2017 AllDayDevOps sessions free of charge here.

    — Derek E. Weeks

  • DevSecOps: If You Build It, They Will Come

    DevSecOps: If You Build It, They Will Come

    Spring training for Major League Baseball in the United States has begun. Millions of people share my love for baseball; however, the same can’t be said for security and compliance—well, at least not yet. Perhaps one day.

    Much like in the immortal baseball movie, “Field of Dreams,” if you build a friendly security and compliance system, the developers and operators will come. At least, that is the contention of Julie Tsai (@446688), senior director of security operations at Box.

    GettyImages-509936368

    I first met Julie at the RSA conference’s DevSecOps Days three years ago and have been following her musings since. I recently watched Julie’s presentation, “Build It and They Will Come-pliant: DevSecOps in the Real World,” from All Day DevOps. Here is a summary of what I learned:

    Good Architecture Sustains Sound Applications and Security

    Julie’s contention is that you can build a system that might actually bring a little joy to developers and operators, and it starts with realizing that, at the end of the day, we are all looking for good architecture. Good architecture sustains sound applications and security. It makes everyone’s life easier, so we all have a little time for baseball (or football or board games or even curling).

    The good news is that DevOps organizations are ahead of the game. Julie pointed out the old joke, “DevOps … isn’t that just where you give the keys to the developers?” Well, no, and it isn’t a specific tool or deployment. For security, it is about being lean on requirements, building compliance in from the start and integrating it across the development life cycle so you can build something that is secure and performant. Besides these being best practices for developers, they also allow you to use your code and policies to appease your auditors, as compliance is built-in. Bottom line: DevOps puts the rigor around security and compliance.

    It Takes a DevSecOps Village

    Julie pointed out a couple of takeaways from DevOps that help security and compliance:

    1. You can own a problem and be an individual leader. You can think globally but act locally by seeing across the silos and bringing a message of empowerment.
    2. It provides a path toward integration to internalize other groups’ values, bring them into your own words and ways and mutually reinforce and thrive. It also informs how people work together.

    Remove Friction to Scale DevSecOps

    The goal here is to ultimately get us something easier to scale and maintain and be compliant and secure. Julie outlines steps to get you closer to this idyllic system:

    1. Use configuration management because it leads to precision and more unified and verifiable work.
    2. Get rid of whatever you can to eliminate mistake factors. You will also get to a place to streamline workflow so there are fewer mistakes.
    3. Extend infrastructure as code. It can become configuration as policy and your audit trail can connect the intent to the execution.

    DevSecOps: Where Things Go Wrong

    With these in place, Julie contends you need to realize the world isn’t perfect, so you need to build a system that injects security and compliance where it is most effective. To find the areas in the development life cycle that it is important to inject, ask these questions:

    1. Where is someone intending to make a change?
    2. When you are releasing it live into your production stack?
    3. What is happening when something isn’t going as expected? Is it being changed back to where you need it to be, are you monitoring it, or are you doing nothing?

    DevSecOps: String Together Wins

    You also need to make it simple. She quoted Mark Burgess, “IT has a detail sickness,” noting that, “We are often burdened by complexity—we love it and dig right in, but it is important to understand the level of granularity you need. You need to look for the things that can make a critical uplift that gives you an incremental improvement. String together the wins.”

    In the end, Julie said to “try to reach a goal of being visible and streamlined and leverage the automation and technology in a way that is joyful in how we use it. It is not about a rigid process that is going to die soon; it is about what we are trying to bring to the whole world so that we work more efficiently and more technical.”

    The entirety of Julie’s presentation—including some live Q&A—is available here. If you missed any of the other 100 speakers at All Day DevOps, you can find their 30-minute presentations here.

    — Derek E. Weeks

  • DevOps Heals at NIH

    DevOps Heals at NIH

    Cancer sucks. But with folks like Sarah, DevOps is helping make a difference in the race to a cure.

    Sarah Elkins is not curing cancer herself, but she is employing DevOps practices to help those who are. Sarah supports the technology infrastructure for those who are trying to cure cancer at the National Institutes of Health (NIH).

    Screen Shot 2018-01-17 at 6.56.30 PM

    Sarah Elkins (@configures) configures technology solutions at the National Cancer Institute (NCI), where she has worked for nine years. NCI is a federal agency, part of the National Institutes of Health. It supports more than 700 websites, from basic HTML to content management systems to complex bioinformatics systems that have multiple tiers and thousands of servers. It uses multiple operating systems, containers and physical and virtual machines (both on-premises and some AWS). Developers range from lone scientists to large teams.

    Sarah and I have been following one another on Twitter for several years, but I recently caught the session she delivered in the “DevOps in Government” track at All Day DevOps. Sarah led a discussion about how the National Cancer Institute is automating its builds and deployments—showing it is possible even in a large bureaucracy.

    Her talk covered the processes and technologies that enable software to move from source code repositories all the way to production servers at nci.nih.gov, including the use of GitHub, Jenkins, Nexus and other technologies, with a variety of teams involved.

    We Need to Talk

    Sarah’s cure begins with engagement: development, infrastructure and security all working together, just as teams of specialists work with patients to help them beat cancer.

    To help speed development and approvals—a necessary evil that can grind development to a halt if not managed well—they agree that applications drawing on an existing technology catalog are approved as operational and security teams provide guidance to development on what is required for an Authority to Operate (ATO). In other words, they are trying to preapprove as much as possible and communicate earlier in the process.

    DevOps Practices are Maturing

    Source code is primarily kept on GitHub, including a public repository for non-proprietary source code, and NCI primarily uses Maven or Apache Ant for build scripts. Infrastructure teams provide XML templates to provide consistency.

    Most NCI software relies on build dependencies on open source software components. The organization uses artifacts during builds and utilizes Sonatype’s Nexus as its repository manager.

    All of this supports the automation of builds and deployments at NCI. For builds, most active projects are either on or migrating to Jenkins. Build artifacts may be .zip, .war/.ear or Docker images.

    For application deployments, NCI uses development, quality assurance, stage and production stages. Most teams use some form of automation, from simple (copy content, stop/start container) to more complex scripts. Developers are allowed to perform deployments for robust applications, and some manual orchestration is required—for instance, for database timing or related applications.

    As an organization, NCI is moving toward continuous integration (CI), with varying progress among teams.

    Building In Security

    NCI also now includes security early, often and automatically. The organization has deployed self-serve/on-demand Nessus scans, which allow developers to see, on their own, how they are doing as soon as the application is stable. Security teams run AppScan and Twistlock for Docker image scanning during Jenkins builds, just before deploying and frequently scanning the repository in between. For issues found, development and infrastructure teams work together to remediate security concerns.

    At the end of her talk, Sarah offered up three takeaways:

    1. DevOps at NCI is a work in progress (isn’t it everywhere?)
    2. The wide-ranging needs at NCI require flexibility, communication and teamwork
    3. There is no shortage of work

    Since it is a work in progress, what opportunities are ahead at NCI?

    • More automation for individual applications with Jenkins
      • Developers can perform container restarts on lower tiers
      • Database updates (some projects are using Liquibase already)
      • Some applications are still manual/batch scripted
    • More Docker and improving Jenkins/Docker instances
    • More orchestration automation and Puppet/Ansible integration
    • Security improvements through
      • Scan dependency artifacts before building
      • More integration and automation

    Sarah dives into more details in her talk, which you can watch here. If you missed any of the other 30-minute long presentations from All Day DevOps, they are easy to find and available free-of-charge here. DevOps in Government is one of the five tracks.

    — Derek E. Weeks

  • All Day DevOps, Viewed from the Outside

    All Day DevOps, Viewed from the Outside

    This year was my first All Day DevOps event. I loved it, as much as I was able to attend. I was especially interested in how DevOps worked with government, as I work in state government. We don’t use DevOps. My guess is that if you work in state or local government, your group also may not use DevOps. This may also be true of some federal agencies.

    I’ve not been fortunate enough to work in an organization that used DevOps. So, when I learned of the All Day DevOps event, I had to join.

    I’m a DevOps outsider looking in. Where I work the waterfall project management paradigm is followed, mainly because waterfall includes the word, “requirements.” I’ve been told that scrum and, more generally, agile project management includes the concept of requirements, but they’re just not called that.

    Since we use waterfall, our progress in delivering software solutions is lengthy. I appreciate the need to get a very good understanding of what the users want, but it can be difficult to meet so many needs in a timely manner. Consequently, many users will strike out on their own to come up with a software solution. In our state department, that means using Microsoft Access. Sometimes a division will get someone to write an Access app to quickly satisfy their needs, which they start to use. This works for a while until the demands on the app become too great and they come to IT begging for help.

    I really should mention that not all of this is caused by IT being slower to respond than we would if we were using DevOps. A large part of this is a consequence of not having enough money to purchase a commercial, off-the-shelf software solution, if such exists. Fiscal limitations coupled with a long queue of projects to work on and a project management process that tends to be slower makes for aggravation on both sides.

    Another thing I’ve noticed about my department is that both IT and development don’t work together as well as they could. I wouldn’t characterize it as adversarial—more like just not entirely trusting of each other. This IT/Dev department has more than 200 people throughout the state, so I find it interesting to see how it plays out. Before this job, I worked in a very small organization with just two IT people—and I was one of them. It was easy to coordinate and there was no divide between IT and development. In my current position, however, everyone has their assigned tasks: IT, development, business analyst or security. There is very little overlap. I very much would like to see more.

    When I learned about the All Day DevOps event, I was thrilled with the prospect of being able to learn more about DevOps. Where I work, no one ever travels to any conference, so having All Day DevOps online was a huge bonus. Also, it was free to attend, which was essential. I caught a few of the sessions throughout the day. This may sound trite, but I was awestruck with the idea that All Day DevOps was going on worldwide! This demonstrates just how widespread the DevOps movement is. When I start promoting All Day DevOps to my colleagues, I will stress this fact.

    I understand that All Day DevOps will be done again in 2018. I look forward to that because this year I learned of the event only the day before, which was hardly enough time to really prepare. If you’re like me, working in a government agency—or any organization, really—and you desire to do a better job of providing your users with software solutions they’ll enjoy and in a more timely manner, as well as work better with your colleagues, I recommend that you join in at the next All Day DevOps event. I’ll be there, so I hope to see you there as well.

    About the Author / Rod Falanga

    I’m a longtime software engineer working primarily in the Microsoft stack. Writing applications for both the desktop and web, with some experience in writing mobile apps. I’m also a longtime fan of the BBC’s Sci-fi show, “Doctor Who.” Follow me on Twitter @Doctor_Who.

  • Scaling DevOps at Pearson

    Scaling DevOps at Pearson

    Pearson might be one of the more influential companies you have never heard of. Its footprint in the educational publishing marketplace is expansive, and its scope was just one challenge when it made the move to DevOps.

    Sean D. Mack (@SeanDMackNYC) is formerly the VP of Operations and Application for Pearson. To give you an idea of the scale of the company’s operations when it began implementing DevOps, Sean oversaw 110 development teams and 3,000 different products across the globe. To further complicate things, Pearson is a mature company with both legacy and new products. This is a case study of how the company approached implementing DevOps with the goal of helping you learn how to do it. Sean presented it at the 2016 All Day DevOps conference.

    As they say, you often learn your biggest lessons with your failures. Sean noted that, ultimately, DevOps was a failure at Pearson due to organizational change. But, he presented the case study with the hope you will learn from what Pearson did and take it forward. He continues to be a passionate evangelist for DevOps.

    Pearson had its challenges. It was a siloed organization operating at a large scale, and had a toss-it-over-the-wall mentality in dealing with problems. Out of sight, out of mind.

    seanmack1.png

    In addition, all of Pearson’s teams were at different levels of maturity, from legacy teams running .Net on Windows 2000 servers while others doing CI/CD to AWS. Obviously, this creates challenges when implementing DevOps, but we will see later how the company tried to tackle that, specifically.

    At the high level, the solution involved several aspects:

    • A DevOps practice area
    • A DevOps steering committee
    • Measuring progress based on global KPIs focused on measuring business impact
    • Measuring teams based on a DevOps scorecard
    • Push from the bottom – plenty of developers and operations who embraced and were eager for the change
    • Push from the top – support from key executives

    Pearson’s DevOps Steering Committee set up several workstreams:

    • Tracking tools
    • Metrics
    • Deployments automation and monitoring practices
    • Test automation
    • App engineering resource contention
    • Operationalization
    • Communications

    While these are specific to Pearson, many may be applicable to your organization. In any case, it is important to step back, look at your organization, and decide what your organization needs.

    Pearson also looked at four core competencies to track and measure: automation; independence (how independent is the feature); operationalization (how easy it is to maintain); and continuous improvement. The below chart outlines each area in more detail.

    Automation

    Continuous Improvement

    Independence

    Operationalization

    • Unit tests
    • Critical Scenarios
    • Dependent Systems
    • Continuous Build
    • Continuous Deployment
    • Test Automation
    • KPI
    • SLA
    • Metric Tracking
    • Metric Reporting
    • Remediation
    • Isolated Configuration
    • Code Dependencies
    • Isolated Infrastructure
    • Monitoring and Alerting
    • Tier 1 Service Desk System Knowledge
    • Tier 2 Service Desk System Knowledge
    • Documentation

    To help each team measure their competencies and improvement, Pearson built a self-reporting tool to empower teams but encourage visibility, and developed business metrics and then a competency model to measure teams against to see how they were progressing. As Sean noted, “We were not trying to get everyone to a 5. Some teams, yes, but we also had legacy products that were running for years, successfully, with little or no maintenance. The investment to bring the teams up wasn’t worth the effort. There had to be a business reason.“

    seanmack2.png

    In the end, Pearson raised DevOps maturity for all of the teams. As noted earlier, though, this was eventually phased out, but Sean presented this case study with the hope you found something to drive DevOps in your company.

    In the end, if you have just a couple takeaways, they are this:

    1. Begin by determining what your desirable outcomes are.
    2. Don’t just do DevOps to be cool. Do it to drive business results.

    You can watch Sean’s entire talk online here. If you missed any of the other 30-minute long presentations from All Day DevOps, they are easy to find and available free-of-charge here.  Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here.  This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed). It’s all free and online on Oct. 24.

    — Derek E. Weeks