devopsy.kinsta.cloud

Author: contributor

Five Areas Every e-Commerce Business Should Monitor Using Log Data
If you’re responsible for an e-commerce website, I guarantee you’ve been asked about, told about, or wondered about what kind of information and events you should be logging. Important things are happening all the time, and you need to know what’s going on! Right? Are they bad things, good things, or “nobody cares” things?

So what do you do? Log all the things! Right?! If you’ve ever found yourself awake at 3:00am wondering why you decided to become a software engineer instead of [insert any job that lets you sleep during the night here], chances are it’s because you decided to log everything. Having said that, guess who has two thumbs and logs all the things? This guy.

You need to focus. It’s okay to log all the things, but you need to focus your attention on the stuff that matters the most, and a simple tool for aggregating IIS logs isn’t enough. A scalable and easy-to-use service that will alert you and provide an interface for surfacing the events that matter the most is critical to ensuring a constant revenue stream and the best possible experience for your customers. Yes, there are lots of components and potential events that really matter when it comes to developing or hosting an e-commerce website. So where should you start? I’m going to break this down into the top five areas that should be the initial focus of your log data efforts. You should not only log these types of events but also put an alerting system in place so you’re aware about things that require your immediate attention. (We use Loggly for this.)
1. Checkout. We’ll go with the lowest hanging fruit first. If you’re an e-commerce website, your job is to convert customers. Conversion means pushing your customers through the checkout funnel and completing an order. All checkout errors should be logged, and you should instantly be alerted if any part of the checkout process fails.
This seems obvious, but it’s important you focus on each minute piece of the process including:
- Address data collection
- Inventory availability checks
- Payment processing (Even if you use a third-party service, you need to log any events in addition to protecting yourself from service failures – more on this later.)
- Authentication and authorization logic
- Shipping estimation
- Tax calculation
…and the list goes on. Not all checkout failures will prevent your user from physically completing the checkout process if you’re doing it right, but many of them decrease the probability that your customer will complete their order. Any squeak in the cog of the checkout process can result in revenue loss.
1. Add-to-cart. You can’t place an order if you can’t add products to your shopping cart or bag. This is a huge duh(!), but there are cases where shopping cart issues are intermittent and aren’t completely preventing people from adding products to their carts. The bigger danger lies in these occasional or even moderately frequent product adds that fail without your knowledge because orders continue to flow through your website. Causes might include bad inventory availability checks, bad data, or an intermittent bug that’s difficult to reproduce. Protect yourself by logging all add-to-cart failures when they occur, send out an alert, and investigate the problem ASAP. Honorable mention goes out here to logging Wish List and Favorites errors as well. These features are about customer experience, but they’re also proven winners when it comes to converting customers.
1. Online Catalog/Product Page. Your customers can’t add items to their shopping cart if they can’t find your products. Similar to adding to their shopping carts, the danger here is the intermittent problem that doesn’t always rear its head in an obvious manner. The larger your online catalog, the greater the chance you’re going to miss errors with specific product lines, markets, or other logical groups of products. Bad data is a likely suspect here, especially if you’re dealing with very old data and/or integration with legacy software that has forced you to compromise a bit on data storage constraints. In addition to product and category pages themselves, consider logging issues with suggested, related, and recently viewed products. You’re missing out on impressions and potential revenue if any of these processes fail.
1. Email Signup. Any marketing director worth their salt will stress the importance of collecting email addresses at every opportunity. Sometimes an email address is the only thing you know about a potential customer, and sometimes it’s all you need to turn a shopper into a buyer. Any break in this process should be identified and fixed immediately. If your website gets a lot of traffic, it doesn’t take much to lose out on collecting a significant number of email addresses and a significant amount of potential future revenue. The trick here is that much of your business logic for presenting email signup to customers may exist client-side. Don’t forget to catch and log those exceptions as well as server-side issues.
1. Login & Registration. Well, you want to get to know your customers much better, right? Of course you do! All the better to turn wonderful customers into wonderful repeat customers by providing additional features and more relevant content. Failures here can lead to frustration and exits from your website as well as a degradation in your ability to provide a more personalized experience. Pay attention to your authentication and authorization logic as a whole, meaning you need to go beyond evaluating simple form data submission and validation issues. Social media login errors, out-of-sync authentication and authorization cookies, and errors generated by any additional authentication checks should be logged.
I’ll wrap this up with a short discussion about third-party services. They’re not specifically called out in my top five areas listed above, but they can definitely impact the performance and user experience of your website. There’s a good chance you’re using at least one or more third-party services as a solution for different business problems your company needs solved, and it’s likely these third-party services are an integral component of critical website features. Whether you’re using a third-party service for calculating shipping, sending emails, or finding and displaying the cutest cat GIF of the day, the success of your site is depending on something that’s outside your control.

Protect yourself from these services. You should know their unscheduled “maintenance” periods better than they do. Chances are you’re in the position to decide which third-party services your company uses or you have significant influence over that decision. Either way, be informed as to how they perform and have data to illustrate where their services are falling short. Are they reliable and worth the eleventy billion dollars they cost, or should your company be looking elsewhere for that cute animated kitty GIF service? It’s likely you’re already protecting yourself from failures here with some fallback business logic, but make sure you’re logging and alerting yourself when exceptions occur. They can have a serious impact on your customer’s experience and could prevent shoppers from becoming future buyers. That’s especially true if these services play in the five areas I describe above (e.g. shipping, tax, payment, personalization, etc.)

And my last parting thought: The same considerations regarding third-party services apply when your e-commerce platform becomes more of a distributed solution where those distributed services and applications are your own. You will almost certainly have other performance monitoring and metrics in place for your own services, but you still need to log, aggregate, alert, and protect yourself from yourself!

About the author/Aaron Remaklus:

Aaron Remaklus is chief architect and development manager for Speedway Motors, Inc. He leads agile development teams building high availability, extensible e-commerce platforms. A Nebraska Wesleyan alumnus, Remaklus taps into a diverse background in business, marketing, training, and software development in tackling large-scale problems.
March 3, 2015
Webinar: How Hired helps DevOps engineers discover new opportunities

Rails v. Recruiting: How engineering is changing the way you’ll get your next job.

In an ideal world, your job search would be efficient, transparent, and flexible in scope. You would have a good idea of your value on the job market and you’d be able to leverage that to find the best opportunities available. The traditional job search methods—job boards, recruiters, and web forms—have left a lot to be desired.

Hired decided to tackle this problem with technology and innovation. They created a web application that powers the world’s first career marketplace. On Hired, software developers & DevOps engineers are introduced to over 2,000 companies in just one week. And here’s where it gets interesting—in the marketplace, the companies apply to the candidates. They’ll make introductory offers to users that provide compensation numbers upfront. The open marketplace encourages transparency and leverages the market’s demand for talent, meaning candidates get insight into their value and the most competitive offers. And the efficiencies created by this software-supported marketplace allow job seekers get more support, more services, and more opportunities.

Do you have questions about the job market for DevOps engineers or technical jobs in general? Interested in learning more about Hired and how it can help you on your job search? Join us on March 31st for a webinar and Q&A with the Hired team. And in the meantime, check out the all new devopsy.kinsta.cloud job page, powered by our friends at Hired!

Video

Slides

[seoslides embed_id=”6c907fb7a8bf” script_src=”https://devopsy.kinsta.cloud/embed-script/hired-slides/5972/” overview_src=”https://devopsy.kinsta.cloud/slides/hired-slides/” site_src=”https://devopsy.kinsta.cloud” site_title=”devopsy.kinsta.cloud” title=”hired-slides” /]

Scott Bacon, Talent Advocate

When Scott is not fundraising for his stealth salted meat startup (Bacon as a Service), he works as a Talent Advocate at Hired—the leading marketplace for engineering and sales talent. Prior to joining Hired, Scott worked onsite at Google as a technical recruiter and toured as a professional musician. Scott graduated from Columbia University with a degree in Anthropology or, as its informally known, “the gateway to sales and marketing jobs”.

Kyle Brett, Software Engineer

Kyle Brett was born with only five fingers on each hand. Despite this, he has become a reasonably talented software engineer. He has five years of professional XP and his first company, Virtrue, was acquired by Oracle. Today, he ships more bugs than any other engineer at Hired. Kyle prefers mechanical keyboards and emphatically believes that cats are better than dogs.

March 2, 2015
To Be or Not to Be – Shadow IT

Is it being an innovator, or demonizing business decisions to use other vehicles, and cry foul?

Sun Tzu, the famous Chinese strategist and philosopher, once said, “If ignorant both of your enemy and yourself, you are certain to be in peril.” These words ring true to the quandary that traditional IT organizations find themselves in today: on one side, hordes of tech-savvy business users are deploying and engaging technology services at an alarming rate, often without any input or validation by the IT team. On the other side, there’s an aging on-premise infrastructure that requires frequent care and stresses the limited resources and budgets of IT staff. In between, sits a group of technology professionals that want to support their users while maintaining the appropriate level of compliance, risk management and security — fondly known as “Traditional IT”.

Security teams, a part of traditional IT organizations, have begun demonizing any non-sanctioned technology use – dubbing it “Shadow IT – taking a somewhat close-minded view as to how software and services are being procured and utilized in today’s organizations. In the legacy world, IT Security could block or veto projects based on its merit and risk. Business teams had no choice but to work within their constraints, as the security teams controlled the firewalls — the proverbial gates to the kingdom. Nothing entered or left the organization without being blessed through the firewall, which left IT Security in a very advantageous position. Fast-forward to today and we see a drastically different dynamic taking place in organizations: DevOps and business teams are driving buying decisions, or making them without IT approval.

Why is this happening?

If you look at how various teams are structured in most organizations it becomes obvious. IT departments, a legacy construct best suited for organizations with few technology-savvy users, are best suited with a role as a service provider. Meaning, they build infrastructure, deploy software and services, and operate utilities as a service to their user base. This could be users in the form of bankers, call center employees, or even software engineers. There is no minimum or maximum level of technology expertise required on behalf of the user, because the IT department is there to offer support and guidance as necessary. This is very much a “thou shall not use technology not provided by IT” point-of-view, because IT is ultimately responsible for the support and user experience of said technology.

However, IT departments are fundamentally based around annual budget cycles and 3 to 5 year amortizations of capital expenditures (servers, network equipment, etc.). As the quality and variety of software, hardware and SaaS solutions has increased ten-fold over previous decades, we have quickly entered a buyer’s world where multiple satisfactory solutions can be obtained quickly. This is counter to the traditional IT pattern of planning purchases in advance, leaving ample time for procurement haggling. In as little as a year, entire service providers can spring up with scalable, resilient, superior technology solutions — often affectionately referred to as disruptors. These disruptive technology companies move fast, gather feedback from their customers, improve their product/service and deliver new products. If you just spent a million dollars on servers that you plan on keeping for the next three years, it is nearly impossible to justify leveraging a fleet of cloud servers as an alternative just a few months later. The cloud servers could be the right solution for your business — but the pain of wasted capital is too great for most organizations to shake, much less the implied defeat or mistake in judgment by IT management.

DevOps teams, on the other hand – and much to the chagrin of IT and InfoSec teams, thrive in the chaos of moving quickly and breaking from tradition. This is not done maliciously, but rather out of necessity to reach a peculiar and demanding audience. Years of “buy it now” instant gratification have conditioned the end-user to expect fast, high-quality and competitively priced products. This holds true for the Internet generation now responsible for the buying decisions of everything from inexpensive productivity tools to enterprise-grade communications solutions. Decisions are not made solely on who can deliver the highest SLA or who had the best benchmark score. Agile teams now look for the best product fit, defined by a set of conditions ranging from automation integrations to direction and vision of the vendor. This wide range of criteria falls well outside the traditional IT buying spectrum, and thumbs its nose at the legacy buying practices. DevOps teams buy subscriptions to services that they can change or cancel at any time. They procure products that don’t even have user interfaces beyond an API and JSON output. They buy based on “what can we do with this” rather than “what can this do for me”, which is reminiscent of maker culture.

A new breed of tools that enable the DevOps teams to do great things while catering to the IT Security team has emerged. DevOps teams tend not to budget for security, and are often too fast moving to build all the requisite capabilities themselves. IT Security is desperate for a place at the table to ensure the safety and security of organizations and its users. This sets the stage for new, emerging technology companies to capitalize on the widening gap that has occurred as the industry shifts from the traditional datacenter model to one of private, hybrid and public clouds. This means no more 3-5 year investments in a security technology and no 3-5 year guaranteed relationship for security vendors. Emerging companies also have to raise their game to meet the needs of these organizations and prove to be the partner of choice. The uptake to date has been slower as the industry has been slow to detect and understand this shift. Many organizations have legacy investments and don’t want to bring additional complexity to their lives by adding new panes of glass. DevOps teams may directly integrate raw data feeds from new products into their toolkit, but IT teams need solid integrations to existing infrastructure to make the best use of these new products. Without this, they can’t keep pace with the rapid deployments and changes made by DevOps teams, and are left nearly blind to the dynamism of their infrastructure risk and security posture.

So what is the solution?

It’s simple. By integrating security more deeply into the DevOps toolchain, and operationalizing security practices the same way we operationalize infrastructure and code issues. This means there is a space to not only bring cloud-centric data down to the legacy investments through connectors and integrations, but also an opportunity for IT Security to enable DevOps by automating common security practices. If the practices and behaviors of Security teams were operating at or beyond the pace at which DevOps teams deploy, then a collaborative environment can be built with security detection, notifications, and even mitigations act as a “safety net” in agile environments. This makes a lot of sense when you realize that the biggest threat to these agile organizations is not malicious hackers or state-sponsored attacks, but rather human error, policy failure, lack of transparency into security practices, and lack of consistency over time.

About the Author/Tim Prendergast

With well over two decades of experience pushing the limits of technology, Tim set out to create the first “next-generation” security company, Evident.io, focused solely on programmatic infrastructures (cloud). Tim co-founded Evident.io to help others avoid the pain he endured when helping Adobe adopt the cloud at a massive level.

Tim’s prior experience includes leading technology teams at Adobe, Ingenuity, Ticketmaster, and McAfee. He holds over 15 years of security experience; 8 years of AWS security experience, with 3 of them defending the Adobe AWS infrastructure from inception to production as an AWS Certified Solutions Architect.

Connect with Tim: LinkedIN | Twitter | Evident.io

February 25, 2015
Five Key Components of Application Release Automation (ARA)

What are the fundamental building blocks of successful ARA?

Application Release Automation, or ARA, is the consistent, repeatable and auditable process of packaging and deploying an application or update of an application from development, across various environments, and ultimately to production.

Successful ARA eliminates the need to build and maintain custom scripts for application deployments, while simultaneously reducing configuration errors and downtime. By providing a model-based approach to performing critical automation tasks, it effectively increases the speed to market associated with agile development and gives stakeholders the ability to coordinate and automate releases between multiple groups and people.

In this article, we’re going to take a look at the five fundamental aspects that make up application release automation and talk about how each one serves a critical role in ARA’s success.

Packaging

The first tenet of ARA is packaging. Packaging is just what it sounds like – taking a bunch of parts and pieces and bundling them up together in a manageable and functional way. So why is this important and how does it fit within the context of Application Release Automation?

Consider the following example: You and your company decide to switch offices, from your current building to one just down the street. You hire a couple of movers, get them in, and tell them you want to take all the stuff in your office and move it to the new location. Easy enough, right? So the movers start grabbing chairs, desks, computers – everything – and carrying them down the street. They’re throwing things out the door, through the windows, everywhere! Next thing you know, a few chairs are broken. A laptop is missing. Your iPhone is gone! The movers say, “Hey, look, those chairs were already broken. I didn’t see any laptop on the table…” In this scenario, there is no bill of materials – just a bunch of confusion and some missing and broken parts. That is where packaging comes in.

Packaging creates the bill of materials for your deployment. It is your way of knowing exactly what you need to move from point A to point B. You take all the pieces of your release and wrap them up neatly so they are ready to ship. It’s the method by which you can assure that everything that was supposed to make the move, in fact, did. Packaging allows you to confidently say that what you had at the source location is what ended up at the target location – and it is critical to a successful ARA process.

Dependencies

Dependencies are another critical cog in the ARA process. When you are working on your release, you need to understand what other applications or components are involved for its success. For example, your web application uses a database hosted on a different server. If the data in that database is changed, or columns are created or deleted, how will that impact your application’s functionality? This is a dependency – the way your application functions is dependent on the data in another location.

In the case where you are pushing out an update to an existing application, you need to also take a look at how other things might have relied on the old application. Are you doing away with a feature or function that is going to have an impact on something else? This highlights how you need to not only examine what your application relies on to function, but also the mirrored function – what outside processes and/or applications might depend on your existing app.

As you can imagine, the process of charting dependencies can grow exponentially. If application A depends on databases B and C, and C depends on D, E and F and then F… you get the idea. It can get very big, very quickly. This is why it is helpful to chart dependencies as you move, so you have a dependency map that is up-to-date and adaptable. You know where the dependencies are, how and what your updates or releases will affect them, and what you need to keep an eye on post-deployment. If you start charting dependencies at the beginning of your development, it will make ARA much easier down the road.

Promotion

Promotion in the context of Application Release Automation is the process of moving the application or update from one stakeholder to the next so that the release is as close to perfect as possible before it is deployed.

This is a crucial step in the ARA process because it creates an audit trail that assures that the right people along the way gave this deployment the “OK” to move forward. Getting the input from multiple parties is imperative to a deployment’s success because each party – from the developers to release engineers to the Quality Assurance (QA) team – has a different perspective. For example, developers are looking at an application and saying, “How can I make this the best application possible with the most robust set of features?” While someone on the QA Team says, “How can I break this?” They are running your application through tests, wringing out inefficiencies and errors along the way.

By passing (or promoting) your release through the chain of stakeholders, you can find the holes in your release before the customer does.

Deployment

Deployment is the act of launching your application and from point A to point B. Using the word “launch” is no mistake – it’s just like the commander of a space operation pushing the launch key. It’s pressing one (or maybe a few) buttons and releasing your application or update, configuring its operating settings, and letting it out into the wild. Deployment is making sure the release makes it to its destination safely. At its core, this is the fruit of the labors of ARA – letting your deployment free to do what it was designed to do! The core objective here is that ARA moves this from a scripted, expert level process to a more model-driven perspective (where less skilled resources can understand and perform the deployments…). So when applications, components, and dependencies are modeled within an ARA solution, companies now have the means to deploy their critical applications without having to rely on the superhuman resources who designed and built the application.

Compliance

Another key aspect of Application Release Automation is compliance. This is literally, “Are you following the rules of the game?” However, the rules of compliance vary wildly within different environments. At a small startup, with a team of three people making a web app – there are probably not too many rules to follow. The compliance necessary for a game launching birds at disgruntled pigs is minimal.

At the other end of the spectrum are things like banking and government applications. Here, there are laundry lists of rules and regulations that need to be strictly obeyed or you may find your application being tossed – or find yourself in a CIA interrogation room (ok, we haven’t heard about that actually happening…but you can imagine!). With compliance, it can be helpful to start by asking yourself, “What do I really need here?” Once you figure out the requirements that are necessary for your specific application, you can map them out and discover what tools you can use to solve the problem.

This is an essential key to ARA because if you have four out of the five steps above, but miss on compliance, all your work was a waste because it’s not going to play nice in its environment. Adhering to compliance – and documenting that you did – is crucial to successful ARA.

With a strong understanding of these five key building blocks, you will have a solid base to start exploring the wonderful world of Application Release Automation. Go forth…and automate!

About the Author/ Wesley Pullen

Wesley Pullen is the general manager and vice president of deployment solutions at Electric Cloud. Pullen leads the Release and Deployment business at Electric Cloud and is responsible for go-to-market strategy to aid in the company’s growth and expansion efforts around DevOps, Continuous Delivery and Deployment. He brings more than 18 years of experience in software development methodologies and design standards, applying this experience to both commercial and private sectors. Linked in: https://www.linkedin.com/in/wgpullen Twitter: @WesleyGPullen

February 23, 2015
Comparing DevOps to traditional IT: Eight key differences
You can make the argument that in the Software Delivery Life Cycle (SDLC) value is created in two places. First, when the software is created (development) and second, when the software is delivered to the customer and feedback received. The rest is transportation. The faster one can move code from the hands of the developer to the customer and receive feedback, the more value can be realized by both. Don’t get me wrong, there is benefit to various “transportation” activities (testing, QA, staging etc.). But those are not value creating activities, they are risk mitigating processes.

A DevOps organization understands this value equation. They focus on the processes that add value, organize themselves around those actions and do the best to minimize the risk. Traditional IT gets it too – at least conceptually. However they treat all stages in the SDLC with equal importance or worse emphasize the wrong stage(s).

In general, when comparing DevOps organizations to traditional IT, DevOps will have policies and practices that differ from those of traditional IT along the following eight key dimensions,

Planning & Organization
1. Batch Sizes: Go from Big to Mirco
Traditional IT has a bias for going big and for good reason. First, most development shops grew out of the waterfall method which by its inherent nature takes a lot of time. Second, since releases are costly and disruptive, operations allows developers only a few windows a year to release software. And third, its simply not sexy – doing big is how you get promoted. As a result, development organizations maximize productivity, by planning big projects, that involve a lot of code, bundled (hastily at times) into a release, and jammed into production.

A DevOps organization takes the opposite point of view and believe that small is beautiful. They understand that large batch sizes are inherently complex, risky (since there are so many moving parts), and hard to coordinate. Small batches sizes, on the other hand, are simple, easy to understand and test rigorously, and less risky. If things go wrong the impact is minimal and it is much easier and faster to fix. In other words, by going small organizations are able to perform more frequent releases and become more responsive to the customer.
1. Organization: From Skill Centric Silos to Dedicated Cells
Traditional IT is organized around skill centric silos. For the most part silos work, they band like skills together, drive greater utilization, and benefit from economies of scale. However, where these “cost optimized” silos break down is at the hand offs. In a typical IT environment, a new feature has to go through at least 3 – 4 silos before the customer gets it. It is not uncommon for an idea / code to spend 80% of its time waiting or ping-ponging back and forth between silos.

A DevOps organization also operates in a silos but at a different cross section. Here teams are arranged in “cells”, consisting of dedicated cross functional teams, and focused on only one application. By creating this self-sufficient cell that consists of developers, testers, business analysts, and operators, an idea can move from one stage to another without hand offs, promotes cross-training / understanding (key to collaboration), focuses the team on the end goal, and encourages the “shift left” thinking. As for utilization and scale benefits, the rules for response states that for every ¼ reduction in cycle time, productivity will improve 2x, and operating costs by 20%.
1. Scheduling: Centralize to Decentralize & Continuous
Efficient scheduling is at the heart of a Traditional IT organization. Since resources are pooled, projects are usually clamoring for access to SMEs and / or infrastructure. To get around this, enterprises have invested in sophisticated scheduling planning systems. And while these systems are quite sensitive, they are inherently inaccurate, take up to much time to manage, and in some cases become the very bottleneck that they are trying to alleviate.

In a DevOps organization scheduling is pushed to the local cell level. The combination of smaller batch sizes, dedicated teams, and automated processes makes scheduling more simpler to operate. How? First, forecasting is limited to the very near future (2 – 3 weeks) where the teams have better insight. Second, there is no fighting for people’s time since it is a dedicated team. Third, there is no waiting for infrastructure since it has already been defined and automatically provisioned. And fourth, it eliminates time consuming loop backs to management for escalations and decisions.

Performance & Culture
1. Release: Turn a High Risk Event to a Non Event
In a Traditional IT organization, releasing software into production is a high risk proposition. It is fraught with issues, escalations, and constant fire-fighting. The process is tightly managed, governed from the highest levels, and requires participation from all parts of the organization. It is not uncommon for IT to set up war rooms staffed 24×7 for weeks before and after the release.

DevOps organizations, on the other hand, make the release of software into a non-event much as possible. They reduce risk by daily integrating code into the trunk, automating testing, ensuring all environments are in sync, reducing batch sizes (as mentioned above) etc. In other words, they only promote code from one stage to another after they are confident that it will work in production. Thus eliminating all the hoopla about release windows and they are able to move new functionality into production at a much faster clip.
1. Information: Focus is on Dissemination vs. Actionable
Both types of organizations generate and share a ton of data. The difference lies in how the teams uses data. In Traditional IT, information is generated by specialists (e.g. operations team), bundled together with other data into a massive report, goes through management approval, is then shared with other managers, who then send it out their specialist (testers, developers etc.). In most cases the report goes unread, simply because there is too much data, not timely enough, and / or not the right data. In other words, information is shared but poorly consumed, and rarely used to take any actions.

Within a DevOps organization, it’s the team cell that gather and creates the data. Since the data is to be consumed locally they only collect the data (also automated) that the team deems is necessary. And since the data is processed within the team, it eliminates the time lag of creating lengthy reports, manager approvals, and queue time (sitting in some ones email box). As a result, the team is quickly able to read and react to the data – resulting in faster feedback time.
1. Culture: Do Not Fail vs. Fail Early
Traditional IT is fundamentally a risk averse organization. A CIO’s first priority is to do no harm to the business. It is the reason why IT invests in so much red tape, processes, approvals etc. All focused on preventing failure. And yet despite all these investments, IT has a terrible track record – 30% of new projects are delivered late, 50% of all new enhancements are rolled back due to quality issues, and 40% of the delay is caused by infrastructure issues.

A DevOps organization is risk averse too but they also understand that failure is inevitable. So instead of trying to eliminate failure they prefer to choose when and how they fail. They prefer to fail small, fail early, and recover fast. And they have built their structure and process around it. Again the building blocks we have referred to in this article – from test driven development, daily integration, done mean deployable, small batch sizes, cell structure, automation etc. all reinforce this mindset.

Measure
1. Metric: Cost & Capacity to Cost, Capacity, & Flow
Silos work for Traditional IT because they are directly tied to its measurement model – cost and capacity. IT is measured on how much can it get done (capacity) for the least amount of money (cost). It is no wonder that cutting cost, while trying to keep capacity constant, has been all the rage in IT over the last two decades. It’s one of the reasons why outsourcing became so popular.

And while this model works great for stable legacy applications, the newer “system of engagement” applications require an additional metric – the element of time. DevOps organization understand this paradigm shift and have added “flow” as an additional metric. Flow forces an organization to take a look at its end to end cycle time, identify areas of waste, calculate true productive time, quantify quality, and focus on activities that add the most value.
1. Definition of Done: “I did my job” vs. “it’s ready to deploy”
Another source of measurement difference is in how the two types of organizations define done. In Traditional IT, done is defined as the specialist doing “just” their part and handing it off. In essence, their done is more focused on meeting the hand off deadline vs. making sure what is done is deployable. This varying definition leads to some bad habits such as sub-par work, quality loss, finger pointing, and an overall sub-optimized process.

To get way from the finger pointing mentality is another reason why a DevOps organization prefers to create a dedicated cross functional team. With every member of the various parts of IT represented in this “cell” structure and all of them being held accountable for one and only one thing, they all end up having just one definition of done – bring quality software to market.

If you would like to learn more about this topic, please join me at IBM InterConnect 2015. I will be hosting several workshops on DevOps.

About the Author/Mustafa Kapadia

Mustafa Kapadia is the Service Line Leader for IBM’s DevOps practice, a business advisory practice focused on helping large enterprises transform their software & application delivery. He has over 17+ years of experience in the tech. space, both as a service provider and a management consultant.

Prior to joining IBM, Mustafa was a management consultant with Deloitte’s Strategy & Operations practice. He lives in San Francisco Bay Area, is an avid blogger (when time permitting), a speaker, and adviser to start ups.

Connect with Mustafa LinkedIn| Twitter
February 17, 2015
devopsy.kinsta.cloud and InnoTech Conferences Partner on DevOps Connect

We are very happy to announce that devopsy.kinsta.cloud is teaming up with the folks at InnoTech Conferences to bring DevOps Connect to San Antonio and Dallas this April! The press release is below with details. You can find out more at DevOpsConnect.com. Also if you are interested in presenting at DevOps Connect @ InnoTech you can contact to devopsconnect@devops.com. Please include which event you would like to speak at and we will respond with details. If you are interested in sponsoring or exhibiting at DevOps Connect @InnoTech, please contact parker@devops.com.

devopsy.kinsta.cloud and InnoTech Conferences Partner on DevOps Connect

DevOps Conference Comes To Dallas and San Antonio at Popular Innotech Conferences With Full Day of DevOps Learning

Boca Raton, Fl, Febraury 12, 2015 – devopsy.kinsta.cloud, where the world meets DevOps and InnoTech Conferences today announced that they will team up to add DevOps Connect to the InnoTech Conferences in San Antonio and Dallas on April 9th and 16th, 2015 respectively. Both conferences will feature a full track of DevOps sessions in addition to sponsors and exhibits from the DevOps community.

InnoTech is an annual conference and exhibition built specifically for business and technology executives, highlighting the region’s innovation and technology development opportunities. InnoTech is designed to be the most comprehensive and collaborative regional technology event available in a one- or two-day format. Over 1,000 I.T. professionals will attend each InnoTech conference. Featuring over 25 breakout sessions on Security, Big Data, DevOps, Infrastructure, more, InnoTech offers an affordable educational opportunity for the I.T. community. This year, InnoTech is thrilled to partner with devopsy.kinsta.cloud to offer a full day track on DevOps in each of their conferences.

“Our attendees have been telling us they would like DevOps related content more and more over the past few years. Partnering to bring DevOps Connect to InnoTech allows us to provide them with premium DevOps content and opens our conferences to the DevOps community,” said Sean Lowery, CEO of InnoTech. “We look forward to a great lineup of sessions around DevOps at our next conferences in San Antonio and Dallas.”

The InnoTech Conferences are a perfect partner/venue for devopsy.kinsta.cloud to spread the word and increase awareness and knowledge about DevOps, said Alan Shimel, CEO of devopsy.kinsta.cloud. “DevOps Connect events work well in conjunction with a broader IT conference, so we think these will be great events.”

Registration, exhibit and sponsor information are available at http://www.innotechconferences.com/ and http://www.devopsconnect.com. You can also write to devopsconnect@devops.com for specific DevOps

sponsor and exhibitor information.

About devopsy.kinsta.cloud

devopsy.kinsta.cloud is where the world meets DevOps. With over 100 new original content articles per month, it is the largest source of DevOps related news in the world. Additionally its community support and other DevOps information help DevOps community members the world over.

About InnoTech Conference

In one focused day, InnoTech creates an environment where education, innovation, peer-to-peer networking and the latest technology and business solutions are all available specifically for IT professionals. InnoTech assembles the leading business and technology executives in the region for a day of answers to your IT issues, real world case studies and your best chance to see the latest products and solutions to support your business.

InnoTech is your best, local opportunity to meet and interact with those leaders driving the IT industry in the region.

February 12, 2015
Why are we still asking, “Can DevOps work here?”

It’s easy to get tired of buzzwords, especially in the software development and testing world where they’re so prevalent. But while it’s easy to blame the thought leaders and evangelists for creating them, and publishers and powerful marketing teams for inserting them into every content channel imaginable, the buzzwords that truly stick around do so on their own merit.

Agile, big data, and cloud have been written about and philosophized for years—and I hate to break it to you, but they’re not going anywhere. While they may not be the panacea that some make them out to be, their impact on the way we develop, test, deploy, and release software has been documented in every market and industry around the globe. This gives the words their staying power. Not relentless press coverage that’s been covertly sponsored by marketing and PR dollars.

Members of the academy and esteemed buzzwords in the software hall of fame: join me in welcoming our newest member: DevOps.

You may say it’s too early, and that DevOps hasn’t been around long enough to earn such universal acceptance. Many quickly cite John Allspaw and Paul Hammond’s 2009 presentation, “10+ Deploys Per Day: Dev and Ops Cooperation at Flickr” as the real inspiration behind Patrick Debois’ coining of the term, and if you’ve never viewed the session, I can’t recommend it enough.

Skip ahead to today and we’ve got multiple conferences that focus on DevOps exclusively. Devops Days, DevOps Summit, DevOps Connect and DevOps Enterprise feature some the most respected speakers and writers alive today.

But they’re not the only ones speaking there. CTOs, CIOs, chief architects, IT heads, and QA directors from every bank, retailer, hospitality group, government agency, and software development shop are there, too, and with a captivatingly similar story behind their employer’s decision to “do” DevOps.

They were stuck. Their software was late. It didn’t address their customers’ real needs, and it didn’t address their own company’s needs. Their application stacks were growing increasingly complex, the bottlenecks in their SDLC were increasing, and their ability to innovate, accelerate, and stay ahead of their competition was slipping further away from them.

What I find especially interesting in re-watching many of these conference sessions is that a lot of them were also leveraging the other buzzwords mentioned above. They’d been practicing agile for years, and they’d started to look to the cloud to lighten their hardware dependence and provisioning headaches.

What they all needed, and thankfully found, was a disruptor. And no matter what definition you attribute to DevOps (and good grief, there are so many) there’s no denying its disruptive nature.

It disrupts the culture of your dev, QA, and ops teams. It requires constant communication, more deploys, earlier testing, shifting left, an embracing of “failing fast and failing often” and what’s often described as “relentless automation.” Some would even argue that the hardest part of DevOps is getting executive buy-in and sponsorship so that you can even begin your initiative. Though to be fair, your ground level troops should be just as on-board as your commanding officers in order to have the best chance for success.

Joachim Bauernberger at InfoQ recently looked into whether DevOps can work in Telecoms, and it turns out, like in any industry looking to resolve the constraints and bottlenecks listed above, it most certainly can—however, it requires support and commitment from “all links in the delivery chain.”

And this is the only reason you should be asking, “Can DevOps work here?” Do you have the right people, and are you the right person to increase the likelihood of this disruptive shift within your company? Stop wondering if (or worse, doubting) your industry will support DevOps, because each year and each conference that passes proves that it can.

And soon enough, if the people at your organization are exposed as the real constraints and bottlenecks, they’ll be removed. Don’t be one of them. Be disruptive.

About the Author/Noel Wurst

Noel Wurst, Managing Editor at Skytap, Noel is responsible for setting and executing Skytap’s content marketing strategy. That includes overseeing the creation and distribution of high-quality, targeted content, and managing the team and external resources associated with those efforts. By leveraging deep online marketing, project management, and editorial experience, Noel plays a crucial role in helping generate and nurture leads with compelling content. You can follow Noel and Skytap at @NoelWurst and LinkedIn and @Skytap, Skytap’s LinkedIn

February 12, 2015
Webinar: ChatOps: Conversation-driven DevOps
At its heart, the goal of DevOps is better communication between the dev and operations teams at an organization. Some of the tools that help achieve this include version control, chat, and monitoring. By moving the conversations around development and operations out of email and into chat, everyone can see exactly what is going on at every step of the way.

In this session you will learn about the tools and strategies available to:
- Automate system notifications
- Convert conversations to documentation
- Sync teams within organizations
- Collaborate using operational metrics and events
Join Atlassian and Datadog as they discuss a better workflow for bringing all parts of the organization together into one collaborative team.

Video

Slides

[seoslides embed_id=”27e31b714435″ script_src=”https://devopsy.kinsta.cloud/embed-script/5394/5410/” overview_src=”https://devopsy.kinsta.cloud/slides/5394/” site_src=”https://devopsy.kinsta.cloud” site_title=”devopsy.kinsta.cloud” title=” ” /]

Panelists

While Ian Buchanan has broad and deep experience with both Java and .NET, he is best known as a champion of agile methods inlarge enterprises. He has driven organization-wide process improvement with results of greater productivity, higher quality, and improved customer satisfaction. He has built multi-national teams that value self-direction and self-organization. When not speaking or coding, you are likely to find Ian indulging his passions in parsers, meta-programming, and domain-specific languages.

Matt Williams is the DevOps Evangelist at Datadog. This means he speaks, writes, and codes the technology to get everyone to believe in monitoring events and metrics using Datadog. In the past he has been the Evangelist at other startups including Placester and Yottaa, a Linux guy at Microsoft, and all sorts of other roles at Sun, Oracle, OpenText, and more. You can find Matt on Twitter at @Technovangelist.

Date: Tuesday, March 10th, 2015

Time: 2pm eastern standard time
```
REGISTRATION LINK:(EXPIRED) 
```
February 11, 2015
The Internet of APIs
The Internet moves at the speed of light. Not only that, but the terminology used when describing The Internet seems to change with the tides. One of the newest phrases to start making the rounds is “The Internet of APIs.” In an attempt to understand exactly what it is, I went looking for a definition and was unable to find anything at all. As such, it’s time to put a definition out there.

There are two important words in the phrase: “Internet” and “API”. Let’s start with the last word first and work backwards. What exactly is an API? Luckily, the definition of API is pretty settled. For our purposes, I like the definition given to us by API Evangelist:

An API, or Application Programming Interface, at its most fundamental level, is how software applications talk to each other using the Internet. An API allows you to open up data and other digital resources, to public developers, businesses, or even between departments and locations within a company. APIs are increasingly the way in which companies exchange data, services and complex resources, both internally, externally with partners, and openly with the public.

And while we’re at it, let’s look at the popular definition of “Internet” given to us by Webopedia:

The Internet is a global network connecting millions of computers. More than 100 countries are linked into exchanges of data, news and opinions.

So we’re narrowing down on a definition of The Internet of APIs. But we’re not quite there yet. Let’s look back at recent history to see if we can clarify things a little bit. The modern-day web API has been around since Salesforce introduced it’s API at the turn of the century. In the year 2000, they introduced an XML-based API that kicked off the web services API era. And while there were a few other web services APIs that came out afterwards (including eBay and AWS); but things didn’t really start getting serious until Flickr launched their API in 2004. The social network revolution, sparked by companies like Friendster, MySpace and Twitter, brought about a real need to securely exchange information between multiple applications. So when Facebook and Twitter announced their own APIs in 2006, that was when the usage of APIs grew to include identity management in addition to the exchange of data it had become known for.

And then came smartphones. APIs exploded in popularity once the smartphone industry took over the continent thanks to Palm and Blackberry; then came iOS and Android. Why the popularity explosion? Well, applications needed to be made for these new (popular) form factors. However, these new form factors had very specific requirements around data usage, drastically different screen sizes than application developers were traditionally used to, along with previously unseen device capabilities. Add to that the fact that mobile applications needed to figure out a way to share data with each other securely and a perfect storm of needs presented itself.

While all of this was going on, the wireless capabilities available to the consumer also changed. In the year 1999, one year before Salesforce released their web services API, 802.11a and 802.11b hit the market. WiFi was still a growing technology, but the introduction of 802.11a and 802.11b changed everything. The prior iteration of 802.11 had a transfer rate of 2 Mbps; while a allowed for short range speed of 54 Mbps and b allowed for longer range speed of 11 Mbps. The cost of b was also a lot lower than a.

Also in 1999, the first full Internet service available on a phone was launched in Japan by NTT Docomo. This was during the days of 2G networks. In the US, around the same time, cell phones became affordable to everyday consumers. It was a time of incredible growth in the wireless industry and over the following years, cellular networks were stretched to their limits.

This is the environment that led to the growth of the web service API. There were a few things that were extremely important to the upcoming boom:
1. A speed boost in wireless networking was required; and
2. Wireless networking hardware needed to get into the hands of more people; and
3. The cost of wireless networking hardware needed to drop; and
4. Cell phones needed to get in the hands of more consumers; and
5. Internet service needed to hit the cell phone.
Add all of that to the growth of social networking and identity management via API, and a perfect storm of circumstances presented themselves. More and more APIs were built utilizing a person’s identity for personalization and security.

Thus the Internet of APIs was born.

So, based off of the existing definitions as well as recent history, what exactly is “The Internet of APIs”? The Internet of APIs is:

A global network of structured data, digital resources and commands accessible via a growing interface that allows applications and devices to exchange information using any and all communications protocols available.

Since the creation of the ARPANET (arguably, the Internet’s predecessor) packet-switching networks have been utilized as an efficient and reliable way to exchange data. So what’s different about the Internet of APIs since at its core it’s about exchanging data? The main difference lies in the fact that the data provided by the Internet of APIs is data with structure. It’s data that is being shifted around and meant to be consumed in ways that may not possibly have even been foreseen by the creator(s) of the data. The Internet of APIs allows data that may have been designed for one thing to be used for something completely different; thus facilitating the connection of two completely different applications in ways sometimes unimagined.

The Internet of APIs is the underlying foundation to the Internet of Things. Without the capability for connected devices to personalize their usage according to the individual, as well as share data, the Internet of Things could not exist. Without the improvements in specifications in both wireless and cellular hardware, the Internet of Things could not exist.

The Internet of Things is powered by the Internet of APIs; and as the Internet of APIs grows in scope and capability, the same will happen to the Internet of Things. Without regularly updated data and personalization, a hardware device is dumb. A thermostat is just a thermostat without the underlying API. But with the API, a thermostat becomes a Nest. Without an API, a light bulb is just a light bulb. But with the API, a light bulb becomes a Philips Hue light bulb. Without the Internet of APIs, a person walks in to their home, walks in to their living room, flicks a light switch and turns their heat up. With the Internet of APIs, a person walks in to a warm, well lit home because their thermostat turned on five minutes before they got home and their lights turned on one minute before they walked through the door.

Data and personalization: that is the power of the Internet of APIs.

About the Author/Kurt Collins

Kurt Collins is Director of Evangelism at built.io (link). Before that, he started his engineering career as an intern at Silicon Graphics (SGI) where he worked on debugging different parts of their UNIX-flavored operating system. Two years ago, Kurt co-founded The Hidden Genius Project along with eight other individuals after they all saw the screaming need to increase the amount of Black men in the technology industry. Mentoring young people interested in the technology industry is a personal passion of his. Today, he continues his work with tech startups and non-profits alike.
February 9, 2015
The Last Frontier in the Software-Defined Data Center

Three years ago, technology pioneer and venture capitalist Marc Andreesen famously proclaimed, “Software is eating the world,” and, more recently, industry research firm Gartner – in its top predictions for 2014 – declared, “The business is IT.” Combined, what do these statements mean for today’s enterprises? Simply put, with the continued migration toward a software-based world, a modernized IT infrastructure means the difference between business success and failure.

Over the past 10 years, technology companies have seized this opportunity, delivering massive innovation in IT to help organizations transform business models, processes and infrastructure to succeed in an increasingly online world. Key among them has been the use of software to break the 1:1 relationships between application architecture and physical hardware infrastructure, enabling major technology movements such as cloud, SaaS, and big data.

As each part of the infrastructure gains this abstraction process, commoditized, templatized services emerge, improving availability and performance, simplifying operations, and driving further innovation.

The Last Frontier Ahead
The data center is undergoing a revolution in migrating to a software-defined model. As a result, the move to the abstraction layer approach – the software-defined approach – is underway for all five fundamental infrastructure components of the data center:

Compute
Storage
Networking
Security
Data access

Technology vendors have evolved most of these components to operate in this software-defined model. But data access has lagged – it’s the last frontier in successfully building the software-defined data center. Until the benefits of the abstraction approach extend to the data access layer, data center operations will remain compromised.

The Common Thread: Breaking the 1:1 Chain
The software-defined data center breaks the 1:1 relationship that has existed previously for IT functions. This approach holds true whether that 1:1 relationship tied hardware switches that provide the fabric to the vLAN domain or applications to database servers. By allowing the software to abstract the data from the physical device, and treat the device as just a resource rather than a physically bound container for the data itself, software creates high-performing services that can be infinitely scalable and infinitely available.

What innovation in each part of the software-defined data center enabled, and how can we achieve that benefit in the last frontier of data access? Here’s a look.

Making Computing Capacity Elastic – Virtual Compute
Compute resources were among the first to undergo the migration to virtualization. VM has enabled the fast and easy provisioning of compute resources. With the ability to easily add or remove virtual machines (VMs) based on load requirements, access to computing resources has become magnitudes more simplified. Running many VMs simultaneously, organizations benefit through improved system availability, performance and resource utilization.

Slashing Storage Costs, Automating Provisioning – Software-defined Storage (SDS)
Software-defined storage (SDS) decouples the programming requirements for storage-related tasks from the physical storage hardware. By managing and automating storage through software, SDS has allowed organizations to simplify storage functions, including configuration and provisioning, as administrators no longer have to consider hardware attributes. The end result is that storage resources can be more efficiently and easily allocated as needed within an enterprise.

Simplifying Network Operations – Software-defined Networking (SDN)
As VMs rapidly multiplied, they became deployed in large and disparate environments. With networks scaling across thousands of typical machines and hundreds of thousands of VMs, organizations needed a solution that wasn’t tied to the switch.

The OpenFlow protocol delivered an important stepping-stone in the development of software-defined networking (SDN). Separating the programming of routers and switches from the underlying hardware, OpenFlow lets switches and controllers talk with each other. Other technology vendors have also contributed with solutions to separate the control plane from the physical network, enabling VMs to easily move from one physical server to another without limitations.

With the ability to program the network free of hardware, SDN has simplified network operations and vastly improved the provisioning of large, complex networks.

Simplifying Data Asset Protection – Software-defined Security
Software-defined security abstracts security resources away from dedicated appliances, such as physical firewalls or wire sniffers. This model lets organizations establish security parameters without changing the physical hardware, so administrators can easily extend security policies across a cluster or data center as needed. Software-defined security can make it easier for administrators to spot susceptible behaviors, for example. Unlike traditional security, which depends heavily on manual detection and administration, software-defined security leverages automated processes to alert organizations to possible security breaches.

Delivering Continuous Application Availability – Software-defined Data Access
Companies are just now starting to address the data tier, the key communications layer between the database elements and applications through software-defined data access. Some larger companies, such as Google and Facebook, have built their own proprietary data abstraction layer, while other companies are bringing these capabilities to the mass market.

One growing approach is to leverage SQL load balancing software to create a transparent, intermediary layer between the applications and databases. With database load balancing software, organizations can separate the application from the database, removing application complexity by avoiding having to program database structure into the applications. This approach provides for flexibility in how the database components are deployed and scaled, enabling automatic failover and other load balancing functionality with no application engineering.

Without changing a single thing at the database or application layer, software-defined data access provides dramatic benefits. It enables zero downtime for applications, boosts application performance, and reduces both hardware and license costs.

Conquering the Last Frontier

A software-defined data center ushers in new growth opportunities. It eliminates the technical obstacles that can consume operational energies and streamlines IT to improve customer interactions, driving new revenue streams and innovative social opportunities.

With so much business value wrapped up in easily and always accessing data, getting to the software-defined data access layer is crucial to business agility and success.

About the Author/Michelle McLean

As Vice President of Marketing at ScaleArc (link), Michelle is responsible for overseeing all of the company’s marketing strategy and initiatives. She has more than 20 years of networking and market positioning experience. Prior to ScaleArc, she held director of product marketing positions at Silver Spring Networks, ConSentry Networks, Peribit Networks, and Trapeze Networks, and prior to that, she was director of strategic marketing at Pluris. She previously served as program director at the research firm META Group, providing technology and strategy direction to global 2000 enterprise clients. Before that, she tracked technical developments, networking trends, and vendor strategies as a journalist for two leading networking publications, LAN Times and LAN Magazine. Michelle earned her BA in English from the University of California at Berkeley. ScaleArc social links: Facebook, Linkedin McLean social links: Twitter

February 5, 2015