devopsy.kinsta.cloud

Category: Blogs

Coaching DevOps Baseball

Coaching DevOps Baseball

Every technology gets surrounded by hype during its maturation, and DevOps is certainly not immune to that phenomenon. If you’ve spent more than five minutes looking into it, you know that, because the current discussion sounds something like this:

“All right, there’s other stuff, but it’s just support, so I want you all to go out there and practice the important part! You only win the game if you score runs, and you only score a run if you run from third to home, so get out there and run from third to home! You’ll be the best team ever on the only part that matters, running it in!”

“What? That hitting the ball and making first, second, third base? That’s supporting stuff, doesn’t really matter, master running from third to home, that’s how you score runs, and runs win games! Oh sure, some hotshot can hit a home run, but then the team relies on the guy who can get home runs, so forget about it, just focus on running third to home!”

Yes, the vast bulk of punditry sound like this right now, and yes, it is as damaging to your DevOps efforts as this coach would be to a baseball team.

And yes, you need to care about all the other parts of the game – because it isn’t baseball if you only do one part, and it isn’t DevOps if you only do one part. No matter how much the coach thinks that part is the important part. After all, in the game, you’ll never get the chance to run third to home if you’re not able to handle the preceding parts of the game.

So when you read pundits at the moment, pay more attention to what they minimize than what they consider the end game. The benefits of putting scripts and configurations into version control, building automated testing, taking the time to write automation scripts you’ve been thinking about off and on for years (and putting those into VC also), all of these things are not only necessary prerequisites to the nirvana some foresee if only you do DevOps “right”, but they also hold benefits all their own. Harness your hotshots to automate what’s hard, and make that automation adaptable enough that it can change with the times. The “right” way to do it is methodically. Indeed, I would add the Etsy slogan “measure everything”. You can’t tell how DevOps is impacting the org if you’re not measuring things.

And doing it methodically is actually pretty easy. Pick a team or (my preference) a project, and tell them to do it. Set standards for keeping each change to configs in version control, demand that they automate as much as possible, but leave points in the process for human checking, require that automated tests are complete and run successfully before the code is considered complete (and set a standard for maintaining tests in the future, since that’s where most orgs have issues). Involve security deeply, because a failed drive, or a server dropped from a bad config can be recovered quickly, a security breech not so much. Then take what you learn, and apply what works well for your org again, perhaps to a larger team or project.

The whole point of DevOps is to improve IT’s responsiveness to changing business needs, increase operational capacity, and reduce human error. It’s not to achieve some goal set by people outside of your org, it’s to do it within the confines of your org.

So go hit some home runs, run the heck out of the first three bases, make IT more efficient and effective. Worry about things like “IT Transformation” when they come up and make sense, not when you’re just learning the rules of the game.

July 9, 2014
The Delivery Pipeline is your DevOps Signature

Where is DevOps? Searching for it is like querying a rainbow for pots of gold. No single element in a development team’s lifecycle defines them as a DevOps-enabled organization. Were anything the rainbow and the gold, it would be the delivery pipeline. The deployment pipeline centers the team and the process, functioning as the signature of any DevOps shop.

(more…)

July 8, 2014
Part Dev, Part Ops

July 7, 2014
Building Security and Compliance into the SDLC

As the concept of DevOps has become more popular in recent years, there are some common themes that always seem to come up in discussions about it. For example, how often do we talk about ways to speed up the software development life cycle (SDLC) in order to bring innovative applications to market faster and meet ever-increasing customer demands?

As important as this and other popular DevOps themes may be, though, they often have the side effect of overshadowing some of the less glamorous topics, such as security and compliance, which is often viewed as a sort of “necessary evil.” That said, the risks of putting an unsecure application into production can create massive challenges for organizations – from breaches to loss of company IP to brand damage – making security and compliance a topic that must be tackled as enthusiastically as accelerating innovation.

Three Critical Areas of Focus

When it comes to considering how security and compliance can become more integrated with DevOps practices, there are three critical areas on which all organizations should focus: shifting security left in the SDLC, building security into policies and creating an audit trail throughout development.

Shifting Security Left in the SDLC

Organizations that incorporate security in the earliest stages of the SDLC will ultimately benefit from products and applications that are secure by design. But in order to accomplish this, they can’t wait until the software hits production to test it for vulnerabilities.

Service virtualization can help by simulating the application delivery run-time environment and its data, allowing developers and testers to focus on security immediately, instead of waiting for access to crucial resources. In addition, a release automation tool can help collapse development cycle times from days to minutes, enabling IT teams to spend more time improving the functionality, quality and security of their applications – and less manually promoting them from stage to stage.

Building Security into Policies

It used to be that every element of application development took place between an organization’s four walls, but with the rise of cloud and software-as-a-service (SaaS) technologies, it’s not uncommon for different stages of the SDLC to occur on different infrastructures. So if the environments are always changing, how do we ensure security is consistently addressed? By building it into the policies that comprise our continuous validation methodologies.

Release automation enables development teams to create policy-driven infrastructure provisioning rules that automatically determine what application elements can and cannot be staged on specific infrastructure types. For example, a common policy would be to never let customer or database information move to a public cloud environment, where sensitive information could be lost or stolen.

Creating an Audit Trail through Development

To this point we’ve centered the conversation mostly on security, but being able to maintain – and prove – compliance throughout the SDLC is an equally important challenge that IT development and operations teams must solve.

So when we think about compliance, we’re really asking questions like “What application moved from this stage to that stage?”, “Who released the application into production?” and “If there was a rollback, what was the cause and who rolled it back?” In other words, we’re talking about audit trails and logging all activities within the SDLC – something that can be difficult when numerous manual steps are involved. However, a release automation tool will capture all of this data as a matter of course as it automates release deployments through orchestration and promotion of applications from development through production. This means you’ll always be able to answer those who, what, where and when questions that pop up during audits.

Eliminating Vulnerabilities and Getting Back to the “Good Stuff”

I’ll admit that security and compliance are not the most exciting topics in DevOps. If application development were a meal, they’d be the pile of lima beans you have to eat before you get to the dessert that is innovative feature development. The good news is, service virtualization and release automation tools can help you speed and simplify the process of building security and compliance into the SDLC, which ensures your applications will hit the market without vulnerabilities – and lets you get back to the fun stuff.

July 7, 2014
A/B Test with Caution

When the morning news starts talking about A/B Testing of course my ears perk up, and I eat my cereal a little slower. But when it becomes a conversation about how ethical it is to use customer usage data to iteratively improve the product, it spawns this blog post.

(more…)

July 2, 2014
Flexible and Scalable Multi-tier Application Deployments

This is a guest post from Alberto Arias Maestro, CTO and co-founder of ElasticBox

One of the biggest problems facing software engineers since the dawn of the multi-tier application is, well, how to make it multi-tier.

It’s more than just having several supporting applications – it is about connecting the layers correctly and allowing them to communicate with each other. All this is to create a scalable and responsive deployment that can be easily updated and adapted to changing business needs.

It is about ensuring that your infrastructure can co-ordinate the order in which your application tiers are spun up, even when the apps themselves have not been designed to perform these critical dependency checks.

What Are You Looking For?

Intelligently handling your solution’s dependencies is an inherent problem in multi-tier deployments – at whatever scale you are operating. For instance,

As a developer working on a project:

“I want to ensure that my database is up and running before my web-server is deployed.”

As the CTO of a rapidly growing startup:

“I want to bootstrap on basic AWS services (such as managed cache, load balancing, and managed databases), but as the product evolves, I want to give myself the freedom to evolve the services I connect to and consume – experiencing as little downtime as possible.”

As the Head of IT Operations of a large-scale enterprise:

“I need to reign in a heterogenous environment where I have hundreds of highly inter-dependent applications that range from just-released to legacy and everything in between.”

At ElasticBox, we’ve found a way to solve this problem using the concept of Boxes, and now, Bindings, a feature we are launching today.

‘Bindings’

ElasticBox lets you build and deploy complex, multi-tier applications using boxes as basic building blocks. Specifying a ‘binding’ from one box to another is a way to declare that there exists a dependency between the boxes.

It is a construct that allows the different tiers of a multi-tier applications to communicate with each other and exchange data.

Bindings enable users to introduce not just structure, but also flexibility to their multi-tier deployments. For example,

Our developer’s Apache Web-server Box can use a required binding to specify, that to launch correctly it needs a connection to a running MySQL DB instance.

Our startup CTO, who has been trying to make a choice between AWS SQS (Simple Queue Service) and RabbitMQ, can simply switch bindings, with his business application experiencing little to no downtime.

Our Head of IT Operations, can chain his applications so that they come up in the right order. He has the flexibility to modify this chain at anytime – to accommodate new applications or possibly remove old ones, without having to bring the entire system down. If you are thinking about deadlocks – don’t worry, bindings are designed to handle these.

On-Prem? Cloud?

For enterprises, bindings can be a very powerful way to evaluate different cloud strategies for business solution with minimal disruption to the existing deployment. Bindings allow switching between an AWS RDS instance and an on-prem database all with a single click.

As you can see from some of the use-cases above, ‘bindings’ is a powerful concept that solves a very prevalent problem in advanced deployments and provides users a simple way to evaluate different strategies. To learn more about how to incorporate bindings into your deployment, please visit our documentation. For a free consultation about how bindings can benefit your particular use-case, please email us at support@elasticbox.com.

July 1, 2014
Infrastructure-as-code, new rules for the old game

There’s been a lot of discussion regarding DevOps and its role in fostering quality and continuous improvement through increased levels of automation. In turn, this automation is fostered through emerging infrastructure and software architectures that are exposing control plane and management programming interfaces, which has been dubbed in many circles as Infrastructure-as-Code.

This movement allows development and operations personnel to have real-time interactions with the underlying environment—the network, servers, operating systems, and storage—that are used to deliver applications and data. It also means that as the environment becomes more programmatic the business must treat this this effort as a first-class application development process and ensure that the corresponding assets follow common coding practices and meet with a consistent and sustainable architecture.

Those that are unfamiliar with the activities of system administrators and data center operations may have an ill-conceived notion after reading emerging literature on DevOps that operators have been running the business on the backbone of purely manual processes. Even as a life-long IT person, until I started working with infrastructure and operations as part of my role, I was unaware of what happens behind the great curtain that keeps the digital bits flowing through the organization.

The truth is that operations and administrators have been developing scripts for years to automate routine tasks. These scripts have typically been used to dynamically generate or update configuration files that were then applied after a restart. This change to Infrastructure-as-Code is significant because now these configuration changes can be made on the fly and are often immediately acted upon without having to take the system offline. If you’re now putting your hand over your mouth upon the realization of the potential disastrous effects this can have, you understand the impact of this power.

Thanks to Software Defined Networking, hypervisors, and even Software Defined Storage, an operator can programmatically change the entire environment underlying the application without having to notify the application of this change. Architecturally, this is a wonderful thing. Pragmatically? Well, there’s a lot of bad code out there that makes assumptions about the underlying infrastructure and is about to be exposed when migrated to these modern infrastructures.

Hence, building, testing and deploying infrastructure control applications must become part of the overall Software Development Life Cycle (SDLC) and begin to be built, tested, deployed and managed using the same controlled techniques that are hopefully being used for any applications that are being developed by application engineering.

You may be asking, “Wait, doesn’t that mean my QA environment must have the same infrastructure components as my production environment?” To wit I respond, “Maybe!” In some cases this may be advantageous for ensuring that production deployment can support continuous deployment processes, which will ensure limited human intervention from the point of user acceptance through deployment, which will, hopefully, lead to limited opportunity for production failures. However, because we’re moving to a more programmatic infrastructure many vendors can also supply virtual instances of their infrastructure code for purposes of testing and this may be satisfactory for your needs.

Of note, there are many businesses out there that do not develop their own software. They primarily use Commercial-Off-The-Shelf (COTS) applications. Thus, these businesses may not have an established SDLC practice. Likewise, these businesses may not believe DevOps is relative for them since they do not do “development”. If your business falls into this category, the move to Infrastructure-as-Code and Software Define Data Centers should be perceived as a driver for DevOps adoption as well as formally adopting one of the well-defined SDLC approaches. Even if these components won’t impact you for a few years until your next refresh cycle, the advantages DevOps brings around improving quality and consistency in delivery is of definite value to your business now.

Infrastructure-as-Code is a significant change in capability that operations has for delivering compute and storage resources to the business. It will allow them to have greater insight into the hardware allowing them to do predictive maintenance, migrate applications to various resources based on demand, and tune the overall environment to meet the business’ needs at a given moment. In short, it will allow IT to deliver compute services with a high degree of availability and security. It will also expose shortcomings in existing applications that have made assumptions about the underlying environment on which it is operating. Thus, if operations and application engineering are not working together and the entire application environment inclusive of the infrastructure control apps is not tested and deployed as a unit, the likelihood for failure will be high.

June 30, 2014
DevOps Rock Star dinner
I have been so busy running from conference to conference lately. Next on the agenda is the Velocity conference in Santa Clara, Ca. While I am sure the conference is going to be great, I am super psyched about attending a dinner on the 24^th that promises to be a blockbuster. It is a “State of DevOps” event with an all-star cast including:
Sounds like DevOps Rock Star panel doesn’t it?

Part of the agenda is:
- The Why andWhatofDevOps?
  - Who is DevOps for? Startup, Enterprise, Unicorn
  - Business drivers?
- What will theimpactofDevOps be 10 years from now?
  - ROI and business value
  - How will DevOps cross the chasm?
  - What job functions go away? What methodologies / tools go away?
  - DevOps as a title?
  - Investment landscape?
- How do we get there?
Sound interesting? I am really looking forward to this one

I will be following up with a report from the dinner later next week.
June 20, 2014
DevOps, Growth Hacking & other ways to rule the world

DevOps will disrupt management styles, speed the end to ancient IT practices, and flatten away processes that don’t focus on doing more with less. But DevOps is not the only dev-led movement that will impact and reshape the High-Tech industry. So called “growth hacking” will also disrupt organizational charts by bringing development tools, methods and practitioners into roles previously filled by marketing, sales, and customer success pros. As DevOps and growth hacking become more mainstream, the inevitable result is developers ruling the world.

(more…)

June 18, 2014
The web-scale avalanche

Old-school businesses may not like it, but they’re competing with Google, Facebook, Amazon and other Internet powerhouses because those players have grown from day one with an eye on rapid, scalable, cloud-based and continuous IT operations.

Companies have been running application delivery aligned to an aging model based on low frequency release cycles. For decades, the major IT suppliers like Microsoft, Oracle, IBM, SAP and others have released software at the predictable, plodding cadence of annual or bi-annual releases. IT departments have adapted to the vendor release cycles with mirror-like processes: highly structured in the ITIL fashion with processes in place for working around lots of bugs that don’t get readily fixed by the vendors. Yet this delivery management model is now out of pace in today’s always-connected, Internet-based world in which apps and sites morph on a weekly or even daily basis.

Suddenly, companies and vendors alike must change what they’ve been doing for years. They need to transform operations and products from the legacy-based delivery model to one which is faster, dynamic and proactive. But that’s hard for most companies. It’s like driving a 1979 Chevy pickup on the Autobahn. You can’t keep up with the pace of traffic, no matter how hard you try.

Companies such as Facebook, Twitter and Netflix, born as Internet-age companies, don’t have the same problems adapting to modern times in which consumers expect instantaneous response times. They were built from the ground up to handle the relentless pace of technological change: their infrastructure and applications automatically shift according to network and CPU conditions and workload spikes. Such companies are masters at Web-Scale IT, and rely upon a multitude of open source and web-based technologies designed for distributed computing. More than just technology it also takes radically new processes and mindsets to become a Web-Scale IT company.

Operating in the Web-Scale World

Web-Scale IT consists of technologies and methodologies that enable developers to build, change and scale applications 10 to 100 times faster than in the past. Web-Scale IT relies upon many modern concepts, including cloud computing, peer-to-peer architecture, NoSQL, SOA, Agile development, DevOps, Continuous Delivery and Deployment and more. Gartner describes Web-Scale IT as “a pattern of global-class computing that delivers the capabilities of large cloud service providers within an enterprise IT setting.” The research firm predicts that by 2017, Web Scale IT will be an architectural approach found in 50 percent of global enterprises, up from less than 10 percent in 2013. Below are a few areas of change that IT organizations must navigate to make the shift.

1. Reevaluate IT partners. It’s not just CIOs that need to shift operations for Web-scale computing. The legacy IT vendors are struggling to walk away from profitable business models based on licensing to embrace the on-demand, services-based revenue that just a few years ago, catapulted startups like Salesforce.com into billion-dollar enterprises. The legacy vendors are unable to innovate fast enough and are having to rely on serial acquisitions to keep their technology relevant. Yet rapid acquisition comes at a great price and is not sustainable. IT departments will need to look far and wide to find the best partners that can support their Web-Scale IT efforts moving forward.

2. Acquire diverse, open-source based IT operations tools: CIOs will likely need to constantly review their technology sourcing and have a more critical method of choosing their technology partners. Relying on a few trusted long-term partners will likely not achieve the Web-scale IT goal. Consider that Netflix, which now runs its business almost exclusively on Amazon AWS, only a few years ago relied on its own datacenters full of legacy vendor equipment. Today, the company manages its high-availability video streaming business using a collection of inhouse-developed open source tools (coined The Simian Army) that test the fault tolerance of the company’s operations. Being flexible and open to change allows Web Scale companies to keep ahead of the competition. Standing still, even for a short while, is no longer an option.

3. Foster iterative, collaborative processes: Strong DevOps cultures, where developers and IT operations personnel exchange information frequently to stay in step with changing business requirements, is also pivotal to making this shift. Companies with advanced DevOps practices are usually committed to running their businesses differently: more efficiently, more customer-focused and with the keen ability to respond quickly to change. CIOs and CTOs need people with strong Web technology skills yet also an unflinching commitment to collaboration and complex problem-solving. IT people today need to think on their feet and adapt as they go. Twelve-month planning cycles are out the window.

The building of a storm

Just a few years ago, it was considered evolutionary, if not revolutionary, to purchase CRM or HR as a service. That’s commonplace today. The next area for innovation is within core applications such as a retailer’s pricing application or a bank’s market forecasting tool. Those applications must now also run at Web Scale to be responsive, customer focused and speedy. Don’t count on your Oracle or SAP database to shift so easily or quickly—at least not yet. That’s why Internet leaders are managing IT infrastructure using the newer generation of Web Scale technologies including Puppet, Chef, AppDynamics, New Relic and yours truly, Boundary.

If running at Web Scale sounds overly complex, it’s not. Since most of the latest, open source-based operational tools are highly automated, using them is actually simpler. Crowdsourcing is another strategy that companies are exploring for Agile development projects. The bigger hurdle is in moving from the old-fashioned software development style of over-engineering everything to the more intuitive, iterative style that characterizes modern Web application development. With Web-Scale IT, it’s okay to cut corners and take more risks, because the infrastructure will stand behind you.

There are plenty of people who’ll disagree with me, but I can envision a world in which the Oracles and IBMs no longer run most of the companies in the world. — one that is adaptive, business-responsive and always forward-looking. The future is closer than we think. Will your business evolve?

June 17, 2014