Tag: immutable

  • All Day DevOps: Taking Lambda to the Max

    All Day DevOps: Taking Lambda to the Max

    Lambda is the 11th letter of the Greek alphabet, and it is also the name of the Amazon Web Services (AWS) service that lets you run code without actually configuring a server. Amazon chose the name Lambda because—well, okay, I have no idea. Comment here if you know.

    Thankfully, you don’t need to know that to understand how to utilize Lambda. Matt Williams, DevOps Evangelist at Datadog (@technovangelist), outlined how to set up AWS Lambda at the 2016 All Day DevOps conference in his talk titled, “Hold the Infrastructure: Getting the Most Out of Lambda.”

    First, to understand what Lambda is, we must look at the evolution of servers.

    williams1.png

    We started with physical boxes—we had to buy and maintain these boxes, and no matter how much we used, we had to pay for all of it. We then evolved to multiple virtual machines on a physical server so that we could maximize the resources of physical boxes. Now, AWS and similar cloud services are common. We create a virtual machine that we need, and the service provider maintains the physical servers. We also have containers to keep environments identical across machines and, finally, we arrive at Lambda, which is service that runs code but doesn’t require you to set up the server. It takes care of all of that automatically, and you only pay for the time that the function is actually running. So, if your function takes 500ms to run, you pay for 500ms to run it once.

    Lambda is trigger-based. When it receives a trigger, it executes a specified function. The trigger can be a variety of actions with AWS services, such as when a file shows up on S3 or an Alexa skill is requested.

    Lambda is serverless, as far as you are concerned.

    Lambda is stateless. This means you can’t rely on local states, so, if you need to retain information between function calls, you have to save that data somewhere else.

    Lambda executes functions. Each function:

    • Is code in a container on LXC on AMZN Linux
    • Does not have restrictions on what it can do
    • Is priced based on exec time, quantity and memory

    Lambda links to various AWS services, such as:

    • Files (S3)
    • Database Updates (DynamoDB)
    • Streams (Kinesis)
    • Messages (SNS, SES)
    • IoT (Echo, API Gateway)

    Matt mentioned several good use cases, such as bootstrapping (the first 1M requests per month are free), cron jobs, file manipulation, API translation, social media stream processing, scaling infrastructure, mobile/IoT backends and Alexa skills, to name a few.

    He also mentioned three specific examples to explore:

    Matt then outlined how you can get started and noted that setup is simple as long as you think of application development as simple (write the code, set up on AWS, and debug). He walked through a specific example of automatically resizing images on S3 when they are added. You can watch his full presentation with screenshots here.

    Matt also provided a list of some AWS Lambda Tools:

    Matt rounded out his 30-minute presentation with a discussion regarding the need for monitoring and how to use Lambda for monitoring.

    You can watch Matt’s entire talk online here, where you can dig deeper into this subject. If you missed any of the other 30-minute presentations from All Day DevOps, they are easy to find and available free-of-charge here.  Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here.  This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed). It’s all free and online Oct. 24.

    — Derek E. Weeks

  • All Day DevOps: Modern Infrastructure Automation

    All Day DevOps: Modern Infrastructure Automation

    “We’re no longer an airline. We’re a software company with wings,” claims Veresh Sita, CIO, Alaska Airlines. The success of today’s businesses rests on software. It is an integral part of infrastructure, so we need to always ask, “How can it be better? More reliable? More secure?”

    Nathen Harvey (@nathenharvey) is the VP of Community Development at Chef and the co-host of the Food Fight podcast. He is also an evangelist of automating software operations, which is what he presented on at the 2016 All Day DevOps conference.

    Nathen started his talk by stating that “automation at scale is required for high velocity IT,” making the point that in today’s environment, you can’t wait months between deployments. Enterprises are moving to continuous integration/continuous deployment, and he advocates for continuous automation.

    harvey pic1.png

    To implement automation at scale, you have to start with a dynamic infrastructure by:

    • Provisioning and setting up environments
    • Implementing dynamic scaling of compute resources
    • Migrating legacy workloads to the cloud
    • Deploy in multi-cloud and hybrid-cloud environments
    • Support heterogeneous environments

    harvey2.png

    Several speakers of All Day DevOps talked about automation, and all of them said a critical step is to treat infrastructure as code. According to Nathen, this means that to programmatically provision and configure components, such as servers, databases, firewalls, etc., you also have to treat it like any other code base. This includes version control and automated testing, but also:

    • Automated, full-stack application policies
    • Package and service installation
    • Versionable, testable and repeatable workflow
    • Scalable application policies
    • Management of interdependencies across nodes

    When you treat your infrastructure as code and deliver at scale, you can scale elegantly from one to tens of thousands of managed nodes across multiple complex environments.

    For software to be effective, users have to trust that it will do what they expect it to do. This is true from handheld calculators to self-driving cars, and it is also true when we automate our development and operations. Nathen advocates that we have to build a pipeline to production that creates faith in us that our code is ready for production.  This pipeline must:

    1. Test the code/verify locally
    2. Verify in pipeline with automated testing
    3. Approve with another engineer (aka code review or Nathen’s “four eye rule”)
    4. Build an artifact
    5. Deploy into acceptance environment to automatically test the artifact
    6. Ask someone else, “Do we want to ship this?”
    7. Ship the code with a click of a button
    8. Union (bring together dependencies and run tests)
    9. Rehearsal
    10. Delivered (ship to customers)

    What does the future hold?

    Infrastructure automation is NOT enough. Why? Auditors show up to audit your infrastructure and applications. The problem is, they use different tools—Excel spreadsheets, notebooks, PDFs, etc. This makes it very difficult to audit. Nathen advocates managing compliance documents as code, too. When we translate compliance into code, it can be executed as part of the pipeline so we can test for compliance throughout the entire life cycle.

    Finally, Nathen stresses the need to have application automation, too. Modern applications look like infrastructure as code.

    harvey3.png

    Developers can make applications that operations can deliver and run in production. This makes it easier to consume, run, and keep applications healty. Operable applications need to be:

    • Isolated
    • Immutable
    • Configurable
    • Built on a common interface for monitoring health
    • Rebuildable from sources
    • Reside in common packaging
    • Maintain runtime independence

    As enterprises implement infrastructure, application, and compliance automation, it creates a freedom for people to focus, and people are the key.

    Nathen notes that, “DevOps is a cultural revolution and professional movement, focused on how we build and operate high velocity organizations, born from the experiences of its practitioners.” He underscores that people are the future of automation. Automation places the priority on people, which enables great companies and great products.

    harvey4.png

    You can watch Nathen’s entire talk online here, where you can dig deeper into this subject. If you missed any of the other 30-minute long presentations from All Day DevOps, they are easy to find and available free of charge here.  Finally, be sure to register you and the rest of your team for the 2017 All Day DevOps conference here.  This year’s event will offer 96 practitioner-led sessions (no vendor pitches allowed).  It’s all free and online on Oct. 24.

    — Derek E. Weeks