devopsy.kinsta.cloud

Category: Events

  • Gene Kim, DOES17 Speakers Discuss the Deployment Age

    Gene Kim, DOES17 Speakers Discuss the Deployment Age

    In a special episode of Electric Cloud’s Continuous Discussions (#c9d9), Gene Kim and speakers from the upcoming DevOps Enterprise Summit San Francisco (DOES17) discussed the deployment age, and how DevOps practices and automation can unburden teams and boost productivity.

    The panel included: Mik Kersten, CEO of Tasktop; Robin Yeman, fellow at Lockheed Martin; Courtney Kissler, VP of digital platform engineering at Nike; and, our very own Sam Fell and Anders Wallgren.
    Continue reading for some of their top takeaways:

    Kersten discusses the importance of spreading the word about DevOps tooling: “I think the best thing we can do is accelerate this turning point by giving the rest of those 15 million developers working in all of those other companies the tools that they need to apply these technical practice and to reorient their organizations to actually leverage them.”

    Yeman on the multiplier effect of automation: “DevOps is definitely all about the people, but the automation enables people to have such a higher outcome and better business results. It’s almost like a force multiplier for companies.”

    It’s crucial to look at the bigger picture of software delivery, advises Wallgren: “If you don’t have a big picture of what your value stream is from one end to the other, it’s hard to convince people that the problem they’re looking at is in fact not the right problem to be focusing on, or whether that is the right or wrong problem.”

    Organizations that employ transformational leaders succeed, claims Kim: “We see people driving DevOps transformations in large complex organizations, specifically the DevOps enterprise community, and my claim would be that these people are being promoted at a rate far higher than the population at large. My interpretation of this is that the value that’s being created by these leaders is appreciated and they’re being put into roles where they’re being asked to make an even greater contribution across even a broader portion of the organization.”

    You can’t just optimize for cost and efficiency anymore, explains Kissler: “There’s this realization that we’re all shipping software and really embracing that and understanding that organizations that have typically optimized for cost and efficiency are not going to win. And so, optimizing for speed to value, the value stream is a huge shift as well.”

    Yeman explains the awesome leadership structure at Lockheed Martin: “At Lockheed, most business leaders were engineers at one point in time. So, if you’re looking at the President or the Vice President, they started out in software, or systems, or as a tester. They have to have depth in their area because then they can really empathize with what we’re building and with, you know, the people that are building it.”

    More on the wonders of automation, per Kersten: “Agile has scaled up, but it kind of hit a brick wall until automation came into play and organizations realized that they needed reliable, predictable, and safe ways to deploy. I think one of the most interesting things that happened when the technical practice matured around that automation layer is all of the other gaps it uncovered.”

    Wallgren on the ironic manual nature of the software delivery industry: “I think that one of the paradoxes of what we do in software teams is we don’t use software at all, most of the time, to help us produce our own software. We use people.”

    Kim reflects on revelations after reading “Disrupt or Die” by Jedediah Yueh: “The real value that DevOps will create is not going to be in the FAANGs, the Facebook’s, Amazon’s, Apple’s, Netflix’s, and Google’s of the world, but it really is in the large brands in every industry vertical as they either disrupt, get digitally disrupted, or digitally disrupt themselves.”

    Kissler on how to respond to the exponential growth of digital disruption: “I think the acknowledgement of where we’re at and where we need to go and how fast we can get there is a huge component of responding to the digital disruption.”

    Watch the full episode:

    Want more Continuous Discussions (#c9d9)? We hold our #c9d9 podcast every other Tuesday at 10 a.m. Pacific. Each episode features expert panelists talking about DevOps, Continuous Delivery, Agile and more.

    Check out all past episodes and panelists here.

    About the Author / Anders Wallgren

    Anders Wallgren is chief technology officer at Electric Cloud. Anders has over 25 years’ experience designing and building commercial software. Prior to joining Electric Cloud, he held executive positions at Aceva, Archistra, and Impresse and management positions at Macromedia (MACR), Common Ground Software, and Verity (VRTY), where he played critical technical leadership roles in delivering award winning technologies such as Macromedia’s Director 7. Anders holds a B.SC from MIT.

  • DevOps Connect: DevSecOps Days @ RSAC Call For Speakers Is Open

    DevOps Connect: DevSecOps Days @ RSAC Call For Speakers Is Open

    For the 4th year devopsy.kinsta.cloud is proud to be producing DevOps Connect: DevSecOps Days @ RSA Conference. The event will be on Monday, April 16th, 2018. RSAC is the world’s largest security conference and so the perfect place to hold what we think of as the best day of DevSecOps. You can get full details at the DevOpsConnect.com site. The call for speakers is open now through December 15th.

    devops connect: DevSecOps Days 2018 @ RSAC

    Why Speak at DevOps Connect: DevSecOps Days @ RSAC

    If you have a great DevSecOps story to tell, this is the conference to do it at. Last year we had almost 1000 people stop in during the full day of sessions to catch some of the top speakers in the space, as well as many up and coming thought leaders giving us a fresh look at the latest trends in DevOps and security. This year we expect over 1200 people or more to attend the sessions. Sessions are also video taped and then put up on DevOpsTV for later viewing.

    It is a large stage that gives our speakers a great pulpit to share their expertise and experiences. The fact that it takes place on the backdrop of the largest security show in the world means the spotlight is even brighter.

    More importantly, this is your chance to lead. Your chance to give back to the community as we forge a road ahead towards better, faster more secure applications. Unlike some other conferences, there is a strong preference here for speakers to be practitioners in the space rather than vendors selling to it (though not all vendors are bad, they speak with more practitioners than anyone and do add to the body of knowledge). So there is a strict, no vendor pitches policy.

    What Are We Looking For?

    In fact here are some of the guidelines we are looking for:

    We are looking for practitioners to discuss the unique challenges of adopting security into the DevOps pipeline, the obstacles to security being a contributing and valued partner in DevOps and the ways to overcome them. Specific topics we are looking for include, but are not restricted to:

    • How did you get senior management buy-in? What did you do? How did you quantify the business value?
    • Examples of security teams being integrated into the CI/CD or DevOps process
    • How did you overcome objections inside the organization?
    • How did you design and implement effective controls to mitigate security risks?
    • Which business area (and application) did you start with, why, and what did you do?
    • What were the top challenges, and how did you overcome them?
    • What were your top mistakes, and what advice would you give as a result?
    • In-progress stories (and even nightmares) are welcome!

    Also, beyond the practitioner preference, we are looking for a diverse set of speakers that closely matches the real world. There will be something approaching 50,000 people attending RSA Conference this year. We want DevSecOps Days to reflect that diversity.

    All speakers receive passes to DevSecOps and an expo pass to RSAC itself.

    How You Can Attend DevOps Connect: DevSecOps Days @RSAC

    If you aren’t ready to share your DevSecOps story with the world quite yet, you can still participate and attend DevSecOps Days @RSAC.  All you need is any kind of pass for RSAC. If you are not speaking and have not purchased a ticket yet, or would like a complimentary Expo Hall pass on us, please use these codes.

    • $100 Discount off of Full Conference passes – 18UDEVOPSFD
    • Complimentary Expo passes – 18UDEVOPSXP

    The free expo pass is enough to get you into DevSecOps Days and expo show floors all week, as well as the keynotes. The $100 discount is off a full conference pass good for all sessions.You can register using the codes above at: https://www.rsaconference.com/events/us18/register

    Wait, there’s more

    Following the day sessions, the 5th annual No Wine-ing DevSecOps Cocktail party will take place @ Jillians, San Francisco, 4:30pm till 6pm, down the block from the Marquis. Registration for the No Wine-ing event is required and attendance is free. Register here.

     

     

    The Security @ the Speed of DevOps panel and luncheon, featuring an all star lineup of DevSecOps experts will take place on Tuesday, April 17th, 11:45am, again @ Jillians. The panel will be based on the 2nd annual Security @ the Speed of DevOps reports by George Hulme. Once again admission and lunch are free, but registration is required. Printed copies of the report will be available to attendees.

    So if you would like to tell your DevSecOps story to the world, please apply to speak at DevOps Connect: DevSecOps Days @ RSAC. But whether you do or do not, if you are going to be in San Francisco on April 16th, please join us for what should be another great day of DevSecOps learning, networking and growing at RSA Conference.

    Also many thanks to our sponsors, Sonatype, White Hat Security, Aqua Security, CA Veracode, Checkmarx. If you are interested in sponsoring, write sponsor@devops.com

    — Alan Shimel

  • DOES17 Speakers Share Insights on Live Podcast

    DOES17 Speakers Share Insights on Live Podcast

    As DevOps Enterprise Summit San Francisco (DOES17), slated for Nov. 13-15, inches closer, event founding partner Electric Cloud hosted a special episode of Continuous Discussions (#c9d9), featuring some of the conference speakers. During the discussion, the speakers shared some of their recent DevOps transformation stories and a bit about what to expect during their talks at the conference.

    The panel included:

    • John Esser, senior director of IT and data center operations at AdvancedMD;
    • Gary McKay, scrum master at Somos;
    • Aloisio Rocha, agile product owner at NetEnt;
    • Manish Aggarwal, software engineering manager at Intel;
    • Marc Priolo, global software configuration manager at Urban Science;
    • J. Paul Reed, managing partner at Release Engineering Approaches;
    • Marc Hornbeek, principal consultant – DevOps at Trace3;
    • Gary Gruver, founder and CEO of Gary Gruver Consulting; and
    • our very own Sam Fell and Anders Wallgren.

    Here is a list of the most memorable takeaways from the discussion:

    Every company is different, so approaching a DevOps transformation will vary based on the organization, explains Esser. “Every company has a different set of constraints, a different culture, different things they value. So you can’t just say, ‘Oh, I’ve got this DevOps hammer, and I’m going to start pounding away.’ We understand the principles, but how to apply those and how to execute those principles are quite different.”

    McKay on the success of moving from mainframe to microservices with his “Team Dragon” concept: “Team Dragon was formulated to make sure that our CI/CD process as well as our deployment and release processes were consistent. Coming from a mainframe environment to, say, a distributed microservices environment, there are differences not only in culture, but processes and technology. We wanted to make sure all that was managed efficiently and effectively, so that we could do deployments when we needed to do deployments.”

    Rocha on the NetEnt DevOps journey and what inspired the transformation: “We’ve gone from a place where the business was expanding, but deployments and deliveries were not scaling with the business to working on how to tackle it with automation, working with the deployments and making the deployments visible to everyone.”

    Aggarwal will be talking about the DevOps journey at Intel at DOES17: “I will be covering as to how a lot of organizations, including Intel, are trying to board the journey of DevOps. They are successful to an extent, but how to make sure that the quality, which is the sole purpose of this journey, does not get left behind.”

    Priolo on his pipeline-as-a-service strategy: “We have a pipeline-as-a-service strategy. That’s where we want to make sure that our developers can concentrate on developing instead of trying to figure out how their code is getting into our production environment.”

    Gruver’s simple advice on starting your DevOps transformation: “In the beginning, you need to create some positive momentum. You need to find the things that are the most broken in the development and deployment operations. You need to highlight them. You need to get them fixed. Then you need to market those changes, so that you get some positive momentum for the transformation.”

    Don’t forget about the human factor of release engineering, says Reed: “[At DOES17] we’re going to be going through a little bit of the human factors that are related to release engineering. There’s a couple of facets of that. It’s sort of, ‘How do you go about doing the work,’ and then also a bit about how we treat others, the net system.”

    It’s important to include all teams on your transformation journey, per McKay: “The secret is inclusiveness. Making sure that the mainframe team is part of the microservices process and making sure that we, in our haste to get things done very quickly and very effectively, make sure that we bring them in. We make sure that their roles are purposeful and that they’re part of the process and not just, ‘Oh, you’re just some mainframe guys. We’re just going to ignore you.’”

    Esser on the core values of DevOps: “DevOps as a whole is not limited just to an architecture. DevOps, at its core, is about working better together and increasing the flow of value across the whole value pipeline.”

    It’s important to find the right balance between allowing your developers to be unique, but also adhering to the rules, says Priolo: “The real challenge is trying to find out a way to allow developers to still have their snowflake ability, while still complying with the standards of the company. It’s not to make everything stringent and follow the same exact path. It’s to create a pipeline that gives them the flexibility, but still gives us the ability to do auditing, to make sure that the code is being checked at its proper place.”

    Sometimes it takes a big mistake or bug to make you realize you need DevOps, explains Rocha: “Our process was intense before, to say the least, especially when some scary bug came or a heavy compliance requirement came in that we had to fulfill in a short amount of time. That made us realize we have to fix this.”

    Wow – Aggarwal’s team at Intel went from almost complete manual testing to 100 percent automation in seven years: “When we started about seven years ago, 99 percent of our testing was manual, and everything was repetitive. The work was getting more challenging, and the sheer amount of it was growing with every release. So it was very important to bring that under control. The only way of doing that was end-to-end automation. When I say, ‘end-to-end,’ it’s not 99 percent. It’s 100 percent totally touchless automation.”


    Reed on the human factor of versioning: “Versioning really is about humans communicating with each other. They’re trying to encode information into that version number, which is, I think, one of the reasons you see people bikeshedding that a lot.”
    DevOps can expose your longstanding problems, says Gruver: “When you start trying to increase the frequency of build/test cycles or deploy cycles, you see inefficiencies repeating. You see the trends. There’s a bunch of tools in the DevOps toolbox that enable you to go after that.”


    Moving from monolith to microservices is not an overnight transition, explains Wallgren: “Time and time again we hear the difficulties that everybody has with legacy applications or monolithic applications. The transition away from mainframes is not an overnight kind of thing. They’re not going anywhere, anytime soon.”

    There’s still value in mainframes, and getting that to customers is key, explains Fell: “There’s a ton of value locked up in mainframes and being able to get that out to other people in a service-oriented sort of way. It’s a hard migration to do, but that cultural aspect of making teams feel like they’re comfortable working with one another is critical to getting through that part of the process.”

    Watch the full episode:

    About the Author / Anders Wallgren

    Anders Wallgren is chief technology officer at Electric Cloud. Anders has over 25 years’ experience designing and building commercial software. Prior to joining Electric Cloud, he held executive positions at Aceva, Archistra, and Impresse and management positions at Macromedia (MACR), Common Ground Software, and Verity (VRTY), where he played critical technical leadership roles in delivering award winning technologies such as Macromedia’s Director 7. Anders holds a B.SC from MIT.

  • Microservices Days coming to a city near you

    Microservices Days coming to a city near you

    Microservices is rapidly becoming a design paradigm for developing software. While there are many benefits to microservices designs, it is more complex than traditional application design. For this reason we are formulating a new community around microservices architecture. This community will explore design issues and the intersection with Mobile/IoT, Containers, Cloud, DevOps and APIs.

    Microservices Journal is a newly-launched online community where practitioners can share their experiences and learn from others.

    Microservices Days are a series of regionally delivered one-day seminars led by practitioners.

    Who should attend: CIO/CTO, Application Development Management, Software Engineers, DevSecOps, and IT Operations.

    For our 11/16 program we are lucky to have some strong industry experienced speakers come speak:
    • Michael Cote – Pivotal
    • JP Morgenthal – DXC.technology
    • Deepak Kulkarni – Charles Schwab & Co
    • Irakli Nadarishvilli and Mark Donovan – CapitalOne
    • Ryan Bagnulo – Guarden.io

    This is a great opportunity to come hear from early adopters and learn from their experiences.

    Register Now: http://www.innotechconferences.com/austin/about-2/microservices-day/

    Microservices Days, Austin – Agenda

    Time Presentation 
    8:00am
    9:30am – 10:15am
    10:20am – 11:05am
    jpmorgenthal
    11:10am – 12:00pm Deepak_Kulkarni

    12 Factors for the Enterprise
    12:00pm – 1:00pm Lunch and Expo
    1:00pm – 1:45pm Microservices_Irakli_MarkImplementing Microservices at Capital One
    2:30pm – 3:20pm TBA
    3:30pm – 4:20pm RyanBagnulo
    Microservices Maturity Matrix, Token Authentication & API Gateway Mediation

     Register Now: http://www.innotechconferences.com/austin/about-2/microservices-day/

    — JP Morgenthal

  • Learn DevOps from 57 Experts for Free

    Learn DevOps from 57 Experts for Free

    Need to brush up your knowledge of continuous integration/continuous delivery (CI/CD), automated security, modern infrastructure or cultural transformations?

    All Day DevOps just released its Summer School program offering 57 online sessions delivered by DevOps practitioners.  Every session is free to watch, and absolutely no registration is required.

    Image title

    Watch as many or as few sessions as you would like at: https://www.alldaydevopsy.kinsta.cloud/addo-on-demand

    Courses are delivered by these awesome DevOps practioners who will share their real-world experiences and lessons learned with you.

    Image title

    — Derek E. Weeks

  • Characterizing and Contrasting Container Orchestrators

    Characterizing and Contrasting Container Orchestrators

    Admiral Calcote, also known as Lee Calcote (@lcalcote) or the Ginger Geek to his friends, gave a presentation titled, “Characterizing and Contrasting Container Orchestrators,” at the 2016 All Day DevOps conference.

    Okay, he isn’t really an admiral—nor does anyone call him that—but he used the title “admiral” to describe what container orchestrators do, relating it to an admiral directing a fleet of container ships. You could also say that they are like the conductor of an orchestra, directing the individuals to work together as a group toward a common goal while each musician is still able to play their own instrument.

    lee1.png

    Lee is the head of Technology Strategy at SolarWinds, and for his talk, he discussed four open-source container orchestrators: Nomad, Swarm, Kubernetes and Mesos-Marathon.

    He emphasized the obvious: There is no one perfect solution. Each organization is different, so for each solution, he looked at:

    • Genesis and purpose
    • Support and momentum
    • Host and service discovery
    • Scheduling
    • Modularity and extensibility
    • Updates and maintenance
    • Health monitoring
    • Networking and load balancing
    • Secrets management
    • High availability and scale

    Lee noted that while there are many core capabilities, any orchestrator must have cluster management and scheduling.

    lee2.png

    He then dove deeper into the four solutions. Below are summaries (the full talk is chock full of information and is online here):

    Nomad

    • Designed for both long-lived and short-lived batch processing workloads
    • Cluster manager with declarative job specifications
    • Ensures constraints are satisfied and resource utilization is optimized by efficient task packing
    • Supports all major OSs and workloads
    • Written in Go and with a Unix philosophy
    • Host discovery: Gossip protocol – Serf is used; servers advertise full set of Nomad servers to clients; creating federated clusters is simple
    • Service discovery: Integrates with Consul
    • Scheduling: two distinct phases – feasibility checking and ranking; optimistically concurrent; three scheduler types when creating jobs
    • Uses task drivers to execute a task and provide resource isolation, but it does not support pluggable task drivers
    • Built for managing multiple clusters/cluster federation

    lee3.png

    Docker Swarm 1.12

    • Simple and easy to set up
    • Architecture is not as complex as Kubernetes and Mesos
    • Written in Go – lightweight, modular and extensible
    • Strong community support
    • Host discovery: used in the formation of clusters by the Manager to discover Nodes (hosts); pull model – worker checks in with the Manager
    • Service discovery: Embedded DNS and round-robin load balancing
    • Scheduler is pluggable and is a combination of strategies and filters/constraints
    • Ability to remove “batteries”
    • Rolling updates are supported
    • Managers may be deployed in a highly available configuration, but does not support multiple failure isolation regions or federation

    lee4.png

    Kubernetes

    • An opinionated framework for building distributed systems
    • Written in Go and is lightweight, modular and extensible
    • Led by Google, Red Hat and others
    • Young – about 2 years old
    • Robust documentation and community
    • Scheduling is handled by kube-scheduler
    • Pluggable architecture and an extensible platform
    • Choice of database for service discovery or network driver and container runtime
    • Supports rolling back deployments, automating deployments and rolling updating applications
    • Inherent load balancing
    • Uses Pods, an atomic unit of scheduling. Each pod has its own IP address, no NAT required and intra-pod communication via localhost

    lee5.png

    Mesos-Marathon

    • Mesos is a distributed systems kernel
    • Mesos has been around the longest (since 2009)
    • Mesos is written in C++
    • Marathon is a framework that runs on top of Mesos
    • Mesos is used by Twitter, AirBnB, eBay, Apple, Cisco, and Yodle
    • Marathon is used by Verizon and Samsung
    • Mesos-DNS generates an SRV record for each Mesos task
    • Marathon ensures that all dynamically assigned ports are unique

    lee6.png

    Wrapping up, Lee provided the following overview comparing the different container orchestration solutions.

    lee7.png

    Lee packed a tremendous amount of information into his talk. If you use containers, his talk is worth your time and is available online here. If you missed any of the other 30-minute long presentations from All Day DevOps, they are easy to find and available free of charge here. Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here. This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed).  It’s all free and online Oct. 24.

    — Derek E. Weeks

  • Speaking for the Dead: Are ‘Agile’ and ‘Monolithic’ Actually Good?

    Speaking for the Dead: Are ‘Agile’ and ‘Monolithic’ Actually Good?

    What if Agile is the product of getting bored at work, microservices are the result of us needing a challenge and scrums are the result of an obsessive project manager? Do we all think waterfall development is a dinosaur whose extinction is deserved because of what we have been told and not because of reality? Do we think Agile is better because it simply sounds desirable?

    Michael DeHann (@laserllama), creator of Ansible, spoke at the 2016 All Day DevOps conference, which espouses the benefits of DevOps, and, by association, Agile and microservices. It essentially says, “Let’s step back and evaluate the past. Maybe waterfall development and monolithic architectures are actually a good thing?”

    Michael has been a software engineer and created a key tool in the IT automation toolbox: Ansible. His arguments deserve a look, because re-evaluation can keep you from making mistakes and improve processes going forward.

    Michael started his talk by making the point that in this day and age, it is waterfall versus Agile and monolithic architecture versus microservices. The assumption is that if you are still developing software with the waterfall method and/or developing monolithic applications, you might as well be riding a horse and buggy to work and dialing up coworkers on your rotary phone.

    Throughout his talk, he asked questions to make his points, beginning with, “Is big design up front bad?” with the conclusion, “I don’t think so.” His argument is that design and requirements at the beginning of the process enable a better production. Marketing and engineers hammered out requirements based on needs, abilities and budgets. There was plenty of time to research and design solutions. Both sides worked together, raising questions and comments and documenting mutual agreement. This presented an accurate view of the strengths of various team members and dependencies between tasks and task orders. And, perhaps most important, engineering was engineering—the engineers had greater responsibility and less hand-holding.

    Contrasting that with his view of Agile, big design up front is labeled as bad or counterproductive, engineers are viewed as interchangeable and scope is more unpredictable. Estimates are encouraged to not be time-based, which often results in a constant crunch time. Finally, meetings can consume a significant portion of team members’ time and often include “games reminiscent of Kindergarten, such as Fist of Five or Planning Poker.” Even in lightweight Agile systems, testing can suffer, requirements are more vague and there is no mutual sign-off.

    Michael argues that there can, in fact, be a middle ground where teams:

    • Do share design
    • Have an accessible, effective product owner
    • Know hard requirements versus soft requirements
    • See test-driven development and test automation as super important
    • Understand that Agile and iterative development can be closer than we think

    Agile and DevOps are also often associated with microservices versus monolithic applications, but according to Michael, “What’s so bad about a monolith?”

    Sample Monolithic Diagram

    dehann1.png

    Sample Microservices Diagram

    dehann2.png

    An example microservices shop has 150 services, many thousands of Jenkins builds per day, eight Amazon regions including dev, staging and production and thousands of EC2 instances. This can result in Jenkins configuration and security enforcement drifts, suboptimal code sharing and DevOps writing lots of custom code to manage it.

    Michael contrasts that with monoliths, which have:

    • Increased ability to develop on one laptop
    • Interfaces that benefit from static typing versus design-by-contract
    • Very small numbers of different server/image types in production
    • Local function calls that are much faster than RPC
    • Few instances sitting around with 5 percent utilization
    • No need for “containers” to get around above utilization problem
    • Greatly simplified debugging
    • Easy code-sharing
    • Very simple version upgrades

    Rather than argue that all applications should be monolithic, he makes the point that many of the concerns that resulted in microservices could be solved with improved communications. He contends that microservices are a tech solution to a people problem.

    Michael concludes that:

    • Labels cloud true meanings,
    • Monolithic architectures are more efficient,
    • Agile can be less efficient,
    • Microservices attempt to solve people problems but introduce new technical and people problems, and
    • We should try and separate the labels from the true meaning of things.

    Most importantly, though, is the fact that there is something to learn from everything. So, even if you don’t agree with Michael’s conclusion, you should listen to his talk online here. You might learn an unexpected lesson.

    If you missed any of the other 30-minute long presentations from 2016 All Day DevOps, they are easy to find and available free of charge here.  Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here.  This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed).  It’s all free and online Oct. 24.

    — Derek E. Weeks

  • Liberty Mutual: Taking an Organic Approach to DevOps

    Liberty Mutual: Taking an Organic Approach to DevOps

    Some people are directors, managers, and administrators. Others are disrupters.

    Eddie Webb (@edwardawebb) is an IT disrupter for Software Development Platforms at Liberty Mutual and was a presenter at the 2016 All Day DevOps conference. His talk, “Organically DevOps: Building Quality and Security into the Software Supply Chain at Liberty Mutual,” looked at Liberty Mutual’s transformation to continuous integration, continuous delivery and DevOps. For a large, heavily regulated industry, this task can be not only daunting, but also viewed by many as impossible.

    Eddie’s talk looks at the origins of DevOps at Liberty Mutual, the challenges of centralizing DevOps, and how to make it easy for developers and operations to do the right thing. He noted that DevOps is not Liberty Mutual’s goal; continuous delivery is the goal and DevOps is the enabler.

    It all started at Liberty Mutual in 2004 with a vision: “Black-box reusable services that benefit the business.” The company also had a concept that “developers own the quality of the code; we have no QA team.” Employees were empowered with this “concrete vision but malleable strategy,” as Eddie said. The company gave them time and space to explore solutions, because ownership increases adoption.

    The team started with fixing what frustrated them, including CI/CD pipelines, artifact and code repositories, elastic cloud-native runtime and Agile planning and delivery.

    But what about centralizing DevOps?

    The reality is that tools can be centralized, but culture cannot.

    Eddie laid out an example of a large organization’s IT structure, saying that this is how things work at Liberty Mutual. Eddie’s team is the Central IT, and their customers are the Market IT team, not Liberty Mutual’s end customers. As Eddie pointed out, the more layers you create, the more friction that enters the system. Often, organizations try to reduce the friction through microfixes, but Eddie’s team asked how to change the culture to reduce the friction.

    webb1.png

    This required distributing innovation and centralizing the tools that work. As an example, if you build a manufacturing plant, you wouldn’t build your own power plant. If you are in IT, why build your own cloud?

    webb2.png

    This is the Netflix way. At Netflix, teams are independent, but responsible. Engineers can choose the best tools for the job. No one single team is responsible for innovation. However, the Engineering Tools team helps direct other teams’ experimentation toward new products without getting in the way of innovation.

    Eddie quoted the book, “Drive,” when looking at how to motivate innovation in complex work environments. The author says if you want to incentivize or motivate cognitive tasks, you need three things: autonomy, mastery and purpose.

    When you centralize non-value added platforms, it enables teams to:

    • Master their specific domains of business;
    • See clear purpose in everything they build; and
    • Maintain autonomy of choice of frameworks, patterns, and approach.

    When you decentralize, it raises the question: What about security? Developers are humans, and humans inherently try to avoid tasks they see as unnecessary or overly burdensome, and this becomes easier when the compliance person isn’t there (think of how we handle speed limits). Traditional security measures erode profit and market share, and, from a stat Eddie quoted from Gartner, 77 percent of IT security professionals agree that security slows down IT.

    webb3.png

    Whether you are centralized or decentralized, security (and other compliance-oriented tasks) can increase adoption rates by making it easy to do the right thing. How do you do this? Eddie proposes governing with tools (think of speed governors in fleet cars).

    You also need to have immutable environments, because, “We can’t trust humans to manipulate production.” So, again, you have to make the path you want them to take, and make this the easiest path. As Eddie notes, “You can’t put too much friction into the pipeline, because developers are good at bypassing friction.” You have to make your CI/CD pipelines the easiest way to get ideas to production by. You also have to abstract complexity and automate policies. You have to have an “opinionated platform, and observe your customer so you can address their pain.”

    webb4.png

    Eddie concluded with the following final points:

    • Don’t mandate DevOps. Give employees the chance to master their discipline with examples to set and follow.
    • Favor deep end-to-end accomplishments over broad but incremental steps forward. Focus on taking the right teams far before encouraging broad adoption.
    • Centralize the platforms and tools that your teams shouldn’t be thinking about. Provide foundational services/commodities and let teams stay on purpose.
    • Incorporate contributions from everyone; don’t stifle autonomy. Stay open to new ways of working.
    • Challenge security policies, but respect intentions. Find new ways to enforce concerns without abandoning precaution.

    You can listen to Eddie’s entire talk online here. You might learn an unexpected lesson.

    If you missed any of the other 30-minute long presentations from 2016 All Day DevOps, they are easy to find and available free of charge here. Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here.  This year’s event will offer 100 practitioner-led sessions (no vendor pitches allowed). It’s all free and online Oct. 24.

    — Derek E. Weeks

  • DevOps and Security is Like Smoking Meat

    DevOps and Security is Like Smoking Meat

    It isn’t everyone who thinks, “Doesn’t Ubuntu remind you of wild boar?” Or labors over his pit of slow-roasted pork shoulder while contemplating containers. Or dwells on e2e testing while mesquite smoke permeates spareribs.

    But, Apollo Clark (@apolloclark) does. Apollo is a foodie of smoked meats (is that a meatie or smokie?), working to master the craft: understanding different cuts, what each type of wood adds, and the subtleties of sauces. But, alas, it is a hobby; security and DevOps are his career.

    Being passionate about both, he naturally sees parallels between the two, so Apollo presented at the 2016 All Day DevOps conference with a session titled, “What Smoking Meat Taught Me About DevOps and Security.”

    Understandably, you are asking yourself, “What does smoking meat have to do with DevOps and security?” Apollo notes both have tremendous complexity and nuances, there are multiple ways of getting the job done, lots of ways to mess it up, a couple of ways of doing it right, and you are always learning.

    apollo 1.png

    For both, there are many tools and processes to get the job done. For smoking, your wood is a critical component, along with time. Let’s look at some parallels:

    • Oak = unit testing
    • Maple = coverage
    • Apple = dynamic analysis
    • Peach = static analysis
    • Mesquite = e2e testing. People love it, but it is difficult to handle
    • Wine barrel = browser support. This is when you are doing it really well
    • Bourbon barrel = device support. Pretty complicated
    • Smoking time = testing. You can test or smoke for five minutes or five hours, but there is a sweet spot. You can oversmoke meats and you can overtest.

    Getting hungry?

    apollo2.png

    Meat is critical to a meal of smoked meat, but so are many other components. Likewise, software applications take a suite of other tools and components to deliver the full package. Get your taste buds ready, because Apollo likens all of the goodness that goes with smoked meats to tools you use to keep applications running:

    • Sauces = auto-scaling. They both can get pretty complicated. Always a little different, but you have to make it work for you.
    • Bread = monitoring, which is the bread and butter of infrastructure. Make sure you have it up and down your stack.
    • Salad = system logs. Not the sexiest things, but you can rely on them.
    • Fruit = custom application logs. Takes a lot of time to pair them, but takes a really good thing and makes them better.
    • Beer = firewall. You should always have both.
    • Wine = Intrusion detection systems. These are your fine wines.
    • Whiskey = IR training. It takes time and there are so many ways of doing it. When things break, we have procedures on how to deal with them.

    When smoking meats and in DevOps and security, Apollo asks and answers, “Do we have to do everything? No, but the more we do, the better it will taste.

    “Does it cost money and take time? Yes, but you can do great things even without money and time. You can use cheap meats and you can use open source. Start simple, build up complexity, and always be learning.” Constantly ask yourself, “Am I better today than I was yesterday.” If not, be better.

    You can watch Apollo’s entire talk online here. If you missed any of the other 30-minute long presentations from All Day DevOps, they are easy to find and available free-of-charge here.  Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here.  This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed).  It’s all free and online Oct. 24.

    — Derek E. Weeks

  • All Day DevOps: Taking Lambda to the Max

    All Day DevOps: Taking Lambda to the Max

    Lambda is the 11th letter of the Greek alphabet, and it is also the name of the Amazon Web Services (AWS) service that lets you run code without actually configuring a server. Amazon chose the name Lambda because—well, okay, I have no idea. Comment here if you know.

    Thankfully, you don’t need to know that to understand how to utilize Lambda. Matt Williams, DevOps Evangelist at Datadog (@technovangelist), outlined how to set up AWS Lambda at the 2016 All Day DevOps conference in his talk titled, “Hold the Infrastructure: Getting the Most Out of Lambda.”

    First, to understand what Lambda is, we must look at the evolution of servers.

    williams1.png

    We started with physical boxes—we had to buy and maintain these boxes, and no matter how much we used, we had to pay for all of it. We then evolved to multiple virtual machines on a physical server so that we could maximize the resources of physical boxes. Now, AWS and similar cloud services are common. We create a virtual machine that we need, and the service provider maintains the physical servers. We also have containers to keep environments identical across machines and, finally, we arrive at Lambda, which is service that runs code but doesn’t require you to set up the server. It takes care of all of that automatically, and you only pay for the time that the function is actually running. So, if your function takes 500ms to run, you pay for 500ms to run it once.

    Lambda is trigger-based. When it receives a trigger, it executes a specified function. The trigger can be a variety of actions with AWS services, such as when a file shows up on S3 or an Alexa skill is requested.

    Lambda is serverless, as far as you are concerned.

    Lambda is stateless. This means you can’t rely on local states, so, if you need to retain information between function calls, you have to save that data somewhere else.

    Lambda executes functions. Each function:

    • Is code in a container on LXC on AMZN Linux
    • Does not have restrictions on what it can do
    • Is priced based on exec time, quantity and memory

    Lambda links to various AWS services, such as:

    • Files (S3)
    • Database Updates (DynamoDB)
    • Streams (Kinesis)
    • Messages (SNS, SES)
    • IoT (Echo, API Gateway)

    Matt mentioned several good use cases, such as bootstrapping (the first 1M requests per month are free), cron jobs, file manipulation, API translation, social media stream processing, scaling infrastructure, mobile/IoT backends and Alexa skills, to name a few.

    He also mentioned three specific examples to explore:

    Matt then outlined how you can get started and noted that setup is simple as long as you think of application development as simple (write the code, set up on AWS, and debug). He walked through a specific example of automatically resizing images on S3 when they are added. You can watch his full presentation with screenshots here.

    Matt also provided a list of some AWS Lambda Tools:

    Matt rounded out his 30-minute presentation with a discussion regarding the need for monitoring and how to use Lambda for monitoring.

    You can watch Matt’s entire talk online here, where you can dig deeper into this subject. If you missed any of the other 30-minute presentations from All Day DevOps, they are easy to find and available free-of-charge here.  Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here.  This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed). It’s all free and online Oct. 24.

    — Derek E. Weeks