devopsy.kinsta.cloud

Tag: automation

VictorOps Announces the Incident Automation Engine, Further Advancing Innovation in Real-Time Incident Management
VictorOps helps companies bypass traditional Network Operations Centers by proactively routing alerts, transforming data, and solving problems using end-to-end automation

BOULDER, Colo. – June 22, 2016 – VictorOps, the real-time incident management company for DevOps teams, announced the Incident Automation Engine, a set of cutting-edge features for on-call teams to automate their way to faster mean time to resolution(MTTR) through use of intelligent alerts, sophisticated routing, and smarter system outputs.

Unlike other incident management tools that mainly focus on triaging alerts and notifications, the VictorOps Incident Automation Engine provides unsurpassed control and insight throughout the entire incident lifecycle via a combination of intelligent system inputs and actionable outputs. The Incident Automation Engine automatically quiets the noise of unnecessary alerts, delivers remediation solutions alongside alerts, and provides visibility into and documentation of all communications and actions that occurred during an incident. Additionally, through the use of outbound webhooks, the entire VictorOps solution can now be configured to send data to other tools in real-time, enabling users to do everything from automatically fixing standard issues to customizing dashboards.

“More and more often, the best way to speed organizational innovation is by enabling developers to move quickly,” said Todd Vernon, VictorOps CEO and co-founder. “When it comes to a company’s uptime, this same concept applies. Today valuable time is lost with archaic alerting systems and manual processes. The [Incident Automation] Engine now becomes a foundation to automate out these inefficiencies, gain clearer access to actionable information and continuously evaluate and improve the inputs driving on-call processes.”

Key features of the Incident Automation Engine include:
- Alert Transformations– Quiet unactionable alerts and automatically route specific alerts to the correct team members.
- Alert Annotations– Automatically append alerts with runbooks, monitoring graphs and other relevant information, giving the alert recipient immediate remediation steps.
- Outbound Webhooks– Extend VictorOps platform data into other systems and integrate VictorOps incidents into other service dashboards.
- API– Leverage the VictorOps API to extend VictorOps to existing legacy tools and reporting systems.
- Post-Mortem Report Generator– Automatically pull a snapshot of all activities (alerts, conversations and remediation actions) associated with an outage. Easily use this data to report on SLAs and facilitate team retrospectives.
VictorOps was one of five companies Gartner featured as a 2016 “Cool Vendor” in DevOps. The Incident Automation Engine is one of the many reasons for this recognition. Its purpose is to support on-call teams as they learn from each incident, providing them with the means to continuously improve and evolve their processes.

“Visibility at velocity is very important to us so engineers know what needs to be addressed from problems identified in production,” said Mike Perttula, DevOps Specialist, Signiant. “[The] VictorOps Incident Automation Engine is allowing us to inform more team members in a timely fashion. We are lowering our MTTR and increasing the reach of our incident alerting audience without waking up everyone for the wrong reasons.”

“With the Incident Automation Engine, our customers are now better empowered to resolve and even prevent future incidents, thereby protecting revenue loss, driving customer satisfaction, and maintaining brand reputation,” said Vernon.

The functionality delivered as part of the Incident Automation Engine is now available in the Enterprise edition of the VictorOps product. For more information about the Incident Automation Engine, visit VictorOps.

About VictorOps

VictorOps is the real-time and agile incident management platform focusing on incident lifecycle management and collaboration for IT and DevOps teams. The solution combines the power of people and data to solve IT problems in real-time. The VictorOps platform seamlessly orchestrates team situational awareness, incident creation, escalation, notification, and remediation with team members regardless of physical location or time of day. Privately held, VictorOps is backed by Costanoa Venture Capital and Foundry Group. For more information, visit www.victorops.com and follow the company on Twitter @VictorOps.
June 22, 2016
DevOps Matchmaking: APM + CI = A Perfect Marriage

As anyone who has ever been set up on blind dates by friends can attest, finding the right match in today’s world is a tricky feat.

There’s a lot of evidence suggesting that the frantic pace of life makes such matchmaking more difficult than ever. That said, when the right set of complimentary traits exist and there’s a willingness to find common ground, things can work out.

Spike Jonzian-notions of artificial intelligence aside, certain technological partnerships are increasingly being perceived as “a perfect marriage” within the context of DevOps automation.

Register today for the upcoming webcast: “Using APM to Turbocharge Continuous Integration”

These technological matrimonies closely support the underlying tenets of DevOps methodology—collaboration, experimentation and increased velocity. In fact, it’s easy to wonder why no one ever conjoined them before. But we were using traditional waterfall development and operations and so couldn’t conceive of such a thing. On the other hand, emerging availability of tooling supporting new integrated workflows represents a significant catalyst.

Finding the Perfect Fit

One such DevOps confederation getting a lot of attention is the matchmaking of continuous integration (CI) and application performance management (APM) technologies. Tying together these once-disparate systems makes a lot of sense, despite the fact that few tools would seemingly appeal to more (traditionally) polarized constituencies.

Continuous integration has been a staple of the development community for many years. It emerged as teams evolved the practice of long build times into more frequent check-ins to deliver code and look for potential errors more quickly. Today, with the rise of CI tools such as the Cloudbees Jenkins platform, some development teams are touting tens of thousands of builds per day.

On the other end of the spectrum, squarely on the turf of IT operations, APM has been evolving rapidly over the past few years as organizations have sought to improve use experience and applications reliability via dedicated monitoring. Enabling this ability to identify and address emerging issues has resulted in APM becoming the fastest-growing segment in the IT ops management market, according to Gartner.

DevOps matchmaking factors in when you bring these tools together to create an automated workflow that spans their respective use cases. A cross-functional team can leverage APM-level monitoring and KPIs within the domain of CI and testing to identify potential performance obstacles introduced in new code. Doing so then rather than waiting to respond later in the applications life cycle benefits everyone.

Reaping the Benefits

For example, say an e-commerce applications developer has implemented a new inventory search functionality—something considered a key differentiator by business management. The developer knows this feature could rely heavily on some back-end database system and wants to ensure it’s tested properly, even as the code is being introduced.

Integrating APM and CI allows the entire team to monitor the process and test various conditions related to the update, tracking multiple builds and looking out for related performance issues. Such integration also allows for centralized tracking of other key attributes such as build number, status and build date, to further ease management.

Most importantly, by tying these two critical aspects of the applications delivery life cycle together, the collaboration and feedback loops required to truly embrace DevOps methodology become ever more valuable as these core Dev and Ops functions become intrinsically linked.

If this type of DevOps advancement gets your heart pumping, you’re not alone. Later this month, we’ll host a webcast here on devopsy.kinsta.cloud featuring experts from CA Technologies and Cloudbees. They’ll be discussing the benefits of matchmaking with APM + CI integration. They’ll also detail real-world benefits organizations leveraging integration of our respective technologies (CA APM and Cloudbees Jenkins) enjoy.

We obviously feel that this is a DevOps tooling marriage that will mature, change and stand the test of time. We’d love to garner your questions and feedback on the topic.

Which is all good, because thankfully there’s no such thing as Tinder for DevOps … yet.

June 21, 2016
Do You Continuously Deliver Your CD Pipeline?
If your job is to build and maintain the continuous delivery (CD) pipeline for your organization, do you feel you can’t keep up with your development counterparts in innovation? Everyone is raving about the new cool features in your company’s product. Who deserves the credit? People who wrote the code, right? It’s not that simple. Churning out new features requires a sound process and solid infrastructure. A well-designed CD pipeline is crucial to the developers’ success. It mitigates the inherent risks in a fast-paced, frequent iteration regime now prevalent in agile software shops. On the flip side, as software gets more and more complex and the appetite for a faster time to market grows, opportunities for innovation are in abundance for DevOps teams.

Multi-speed IT

So you’ve got two tactical teams with significantly different risk-reward equations, working at a different pace due to distinct objectives but marching toward the same strategic goal. You’re finding yourself in a multispeed IT environment. Development is focused on innovation and quick frequent iteration cycles. DevOps’ primary focus is to ensure the stability of the CD pipeline and keep a precise track record of what takes place inside the pipeline every step of the way. How do you make sure to stay ahead of the technological curve while minimizing risk?

Based on my experience on both sides of the DevOps isle, I recommend the following four steps to successfully build and maintain a continuous delivery pipeline:

1. Automate, Automate, Automate!

My personal motivations for rolling up my sleeves and scripting a task are:
- It became an annoying, mindless routine;
- It’s no longer a one-time thing and isn’t going away;
- My time can be spent on more important things; or
- I make mistakes and then waste time fixing them.
As with any type of code, organize the scripts into a sound logical structure easy to navigate and version-control it. You can always revise the code or completely rewrite it as the needs evolve.

2. Treat Automation Code as a First-Class Citizen

That’s right! As a practitioner, I learned the hard way that automation starts innocently small and simple. Then the requirements for more automated functionality begin to pile up rather quickly and things quickly become really messy. So write high-quality code from the get-go and reassess the design and implementation every step of the way. Remember that the visibility of your code is very high.

Apply the same standards to your code as your peer development teams do to theirs. Write tests—unit, integration, functional, end-to-end—as appropriate. Use the same CD pipeline to automatically build, package, deploy and test your automation code. This will give your pipeline another round of validation, your code will be of a much better quality, and your team’s credibility and reputation will soar.

3. Containerize the CD Pipeline Components

In the past, because the CD pipeline components such as build servers were manually provisioned, they often became the single points of failure, which made it hard to plan and roll out the modifications to homegrown, open source and commercial tools. It wasn’t unusual for some components to fall behind the latest release by a full year or more. In the meantime, a year’s worth of updates and patches could address a litany of issues, but the need for a maintenance window, usually on a weekend, and the uncertainty of the outcome often would outweigh the sought-after benefits. Containerizing the CD pipeline component can be the solution. Running a build server in a container, for example, will make it a lot easier to create an exact copy of the production version in a test environment and iron out the upgrade process more thoroughly ahead of the upgrade.

4. Test Commercial Tools

Commercial and open source projects test their software, of course. However, it could make a whole lot of sense for your organization to create your own integration, end-to-end and performance tests for the software you don’t write—especially if the software has a plug-in architecture like Jenkins or UrbanCode Deploy and you use community or homegrown plugins. Commercial tools typically don’t provide any guarantees that those plugins will work after the upgrade. Running your own automated tests that exercise the functionality of the tool in your environment and on your specific data gives the opportunity to validate updates as they become available and identify any regressions proactively.

The objectives of DevOps are not only to identify and eliminate the existing bottleneck in the organization’s continuous delivery process, but also to look out for any future potential bottlenecks and proactively address them before they become real issues.

So your team is DevOps. Your product is a CD pipeline. How do you continuously improve the CD pipeline for your product?

If want to learn more about how to effectively build and maintain your continuous delivery pipeline, please check out the newly released 2nd edition of the “Application Deployment and Release for Dummies” eBook.

About the author / Alex Seriy

Alex is an accomplished technologist and DevOps evangelist with over 25 years of experience under the belt. He is an automation enthusiast and usability guru, passionate about uncovering new use cases and innovating applications for platforms, frameworks and tools. At IBM, Alex is a subject matter expert in Continuous Integration, Delivery and Testing. He currently focuses on integration testing in the API Economy through service virtualization. Prior to joining IBM, Alex served as the Senior DevOps Architect at TIAA-CREF for 4 years. Before that, he was a consultant specializing in SCM, Release Engineering, QA and DevOps for over 10 years.

Connect with Alex on LinkedIn / Twitter
June 14, 2016
Lean Security: How Better Development Can Protect Your Business

If companies are to reach their strategic goals—reducing time to market, boosting sales, improving product market fit and brand image, and cutting cybersecurity costs—then it’s time for a new outlook on software security.

Today’s business leaders must learn to see security for what it is: A differentiating factor. Companies with reputations for secure developmental processes and infrastructure will rise above those known for data breaches.

Security as a Business Strategy

The road to a more secure company—and realizing the perks associated with it—begins with lean security. This is an approach to information security similar to the Toyota principles of management and production that calls for environmentally aware engineering, simplified coding, automation of security checks and constant incorporation of feedback.

Although some business leaders believe organizational security is inherently expensive, lean security doesn’t mean massive costs. Much like DevOps, lean security champions process improvements and cultural changes above purchasing new tools.

Lean security does, however, require engineers to keep its principles in mind throughout the development process. In this way, security is much like sales: A sales representatives’ best strategy is to consider his goals from the beginning. Just as the salesperson isn’t trying to make any sale, the engineer isn’t trying to create just any product. For the salesperson, this means tweaking the sales process—perhaps by vetting leads before engaging them—to make profitable sales. For the software engineer, it means structuring the development process through lean security to create secure products.

Done right, lean security results in a self-defending, simple system created by people who are aware of a product’s real-world security risks and the company’s business goals. Lean security recognizes your company’s data is always at risk, no matter the size of the company or team, and it works to create value through systemic protection. Here’s how to start:

Improve Environmental Awareness

Lean security approaches risk management as a team effort. Think about a hospital. Everybody working there — from the cooks to the nurses to the executives — has a responsibility to take care of patients.

Software firms must function the same way: Everyone from engineers to salespeople to the CIO must keep data privacy, software security and business needs in mind throughout product development and a product’s life cycle.

Environmental awareness means addressing security with the same sensibilities that you would your own life. Just as you don’t hand your credit card to random people, neither should you hand the keys to a software engineer who doesn’t know the risks of insecure development.

More secure coding begins with a change of mindset. Once team members understand a product’s security risks, they start thinking about how to create a more secure environment. If a developer is creating a mobile app for a car, for instance, then they must recognize the app will be used in conjunction with a device that carries human life.

Simplify Engineering

Complexity is the enemy of security. Simplification, however, cannot just be limited to lines of code: It means organizing your teams—and thus, your entire system—in a way that improves time to market and tightens feedback loops.

DevOps is a great way that business leaders can improve security through sensible reorganization. While the integration of development and operations teams has been lauded as a security boosting measure by software leaders at companies including Dell, CA Technologies recently studied DevOps’ business benefits. It found companies that have implemented DevOps techniques are 2.5 times more likely to improve customer retention, twice as likely to grow their revenues and 3.4 times more likely to improve their market share.

In short, keep teams compact and agile and design with the end in mind. The more lines of code you add, the more complex teams you create and the more times you reinvent the wheel—and, ultimately, the less secure and less profitable your company’s products become.

Automate or Die

Automation is an essential component of lean security, and DevOps engineers are no strangers to automation. Threats are constantly increasing, and no engineer—or team of engineers—can discover and remedy them all manually.

Look to Netflix for a great example of automated security. Once upon a time, Netflix addressed malware alerts by manually creating a help desk ticket and assigning an engineer to investigate the problem. The time from alert generation to eventual resolution, according to Netflix, often spanned more than a week. As Netflix’s security challenges increased in both diversity and complexity, it found itself spending exponentially more time and money combating these threats.

Then, Robert Fry, Netflix’s senior information security architect, came up with a brilliant solution: He engineered FIDO, an orchestration layer that automatically evaluates, assesses and responds to security threats. FIDO detects threats, then scores them based on the attack’s intended target and other factors. FIDO then attempts to mitigate the event by closing a network port, ending a VPN session or disabling the account.

Tools such as Netflix’s FIDO automatically detect risks in seconds, while it could take a human months to stumble across them. Think of security automation like a home security system: ongoing, vigilant security without the need for continued manpower.

Measure Everything

Although its origins are disputed, many credit English mathematician Karl Pearson with quipping, “That which is measured improves. That which is measured and reported improves exponentially.”

Development teams should take Pearson’s wisdom to heart, and measurement must begin with identification of goals. Is your goal to reduce application downtime or decrease event response time? Then it’s essential to measure mean time to repair. Do you want to reduce code defect density? Begin by measuring the number of issues per thousand (or million) lines of code.

Once you’ve identified goals, use software to track progress over time. Etsy’s engineers built a tool they call StatsD. The software helps Etsy monitor everything from login failures to coffee availability. The data is then displayed in handy graphs to help the team make sense of the information.

Once you’ve taken baseline measurements, be sure to review your metrics on a regular basis. For instance, you might realize an application’s deployment times aren’t where you’d like them to be. Keep measuring results—and keep trying new things—to nudge those values into better territory. Just remember: You cannot improve that which you do not measure.

The Business of Lean Security

The value of a secure, simple system cannot be understated: It means getting products to market faster; it means better public perception and press; it means more satisfied customers who feel secure using your product; and it means fewer disruptions in engineers’ work schedules. At its core, lean security lowers a company’s overall exposure to risk and reduces its expenses.

Companies are already staking their futures—and consumers’ safety—on code that facilitates transactions, drives vehicles and manages power plants. The Internet of Things will soon run everything from toasters to jet engines, and security will grow even more important. Protect your company from within through lean security.

June 3, 2016
Embedding Predictive Analytics

For an enterprise with a deep DevOps maturity model, the initial “wow” factor of automation has worn off. At this point, the sexier concepts involve proving continuous improvement. As an example, our first baseline of metrics might have taken the pre-DevOps timings of a typical build and deployment of a given application. We can measure this time against our post-implementation of DevOps automation and show a significant savings in time. This is opportunity time, meaning an employee must be ready to turn around the decision cycle faster to take advantage of the time saved (not be playing solitaire, for example). But once DevOps automation is old hat, the difference between our initial post-DevOps typical build and deployment time and our current one still using DevOps may not be all that significant. But is there more hidden there than meets the eye?

Establishing Context & Baselines

At this point context may be a more meaningful measurement. As an example, if my first version of DevOps automation for a deployment contained 35 steps and my current one contains 160 steps it would stand to reason I am “doing more” now in my deployments than I used to do. These additional steps may involve executing more tests than I originally thought to do. They may be interacting with monitoring systems (turning them on and off as appropriate), which reduces “noise” in the alerts my monitoring teams have to respond to. In short, over time, my deployment processes may become “longer” because I am increasing the capabilities or the sophistication of what I do during these deployments. While from a pure timing perspective this may make them longer, it also makes them more powerful and, so, is worth measuring. To know the delta, however, I must have retained older versions of automation to compare against, not just throw away the old every time I update with new steps.

So let us assume the context of my automation is now fairly stable. I do the same things, more or less, the same way every time I deploy my application. At this point, measuring the functionality of what I do becomes more meaningful. As an example, my deployment automation may complete four main functional tasks to complete, I deploy the code (the app), I deploy the dataset, I deploy the config files and, finally, I run a series of tests on the above. While I may not ever incorporate too many new kinds of things to do in this automation, understanding how long each function takes to complete is relevant to proving continuous improvement. For instance, if my code takes roughly the same amount of time each deployment, as does my config files, but all of the sudden my dataset begins to grow in each deployment (from normal usage), my overall deployments may be getting slower. This is not due to hardware inefficiency or network inefficiency; it is simply due to a larger dataset that keeps growing.

If on the other hand, it is my test set that is getting progressively longer to execute, that is an indicator of test script inefficiency (given same code, same config and same dataset). If the number of tests being executed has not increased, but the length of time it takes to run them has, there likely is inefficiency in “how” the testing is being constructed or executed. Understanding the context for what is going on in a deployment or similar kinds of automation must occur before introducing predictive analytics can occur.

Historical Trends Extended

Predictively evaluating automation is nothing more than taking the data from a historical perspective and extending the trend lines into the immediate future. If for example, my deployment automation is getting four seconds longer each month for the last 12 months, I can reasonably infer the trend will continue, and I can predict the length of time it takes to execute a deployment in the coming months. Understanding “why” this occurs requires a contextual investigation, but predicting its occurrence does not. This may seem inconsequential as we talk about the length of deployment processes of a given app, particularly if my portfolio of apps is small. But if the trend holds true for a portfolio of thousands of applications, the impacts become more consequential. Or if the phenomenon is directly connected to a particular cloud provider and not seen elsewhere, again, the consequences become meaningful.

Where having “ground-up” predictive analytics (build and/or deployment automation) becomes much more useful is in the construction of release events. Often, release managers are asked to supply an estimated time duration for a given event to factor into the construction of a new release. If a vendor were to integrate more predictive analytical tooling into the release orchestration (RO) toolsets, “random” or “best guess” timing estimates could be replaced by extending historical data into the timing estimation fields. The RO tooling could provide a default timing estimation based on examining historical trends for a period defined by the user. Estimation defaults could always be replaced by a user (assuming discreet knowledge of other impacting events), but at least the new overall lengths of releases then could become significantly more accurate.

This has special significance for manual tasks. Let’s say our database admin team traditionally takes 40 minutes to perform a given release event update, but now is running significantly longer than that. Having the RO software aware of the historical trends could alert the release manager, as well as the database team manager, during the event to check out why this is occurring. For manual tasks in particular it may be hard for anyone besides a predictive analytics module to determine what timings “should” be, as compared with what they are in a given release.

Impacts to the Bottom Line

The real questions are, Why should a customer care about this kind of feature set? Why should we be looking at the vendors of our tooling to incorporate forward-thinking architectures such as predictive analytics into the tooling they sell us? Because the efficiency it can bring to our DevOps set of services lowers the costs of administration and increases the speed at which we can deliver innovation in general. Over time, trending combined with context can statistically prove continuous improvement of our DevOps services, which is the promise we made to the executive teams that sponsored and made the investment in DevOps to begin with.

Metrics analysis can occur (usually with a dedicated investment on our part) without a predictive view of the impacts. But this changes the perspective of our analysis to hindsight thinking instead of forward thinking. In essence, we then are simply “reacting” to what has occurred instead of tailoring our environment to alter the picture of future predicted events to be better than the software predicts it to be. We start tweaking the variables to instantly impact future predictions, instead of only examining them in a post mortem.

This perspective is yet another cultural catalyst for change your business needs. When I talk about changing how an organization “thinks” about innovation, this is one of the key objectives. We take an enterprise from knowing little about the costs of innovation, to having an extensive knowledge, to having a forward view of what is—or what should be, possible. All of this is possible despite the loss of waterfall controls and in the midst of an agile construction model. All it takes is the integration of forward-thinking architectures into the products we select. If our vendor does not offer them, we begin to enlighten our vendors on the value they would bring to us in cost reduction and to them in market capture. If you need help in these areas, feel free to contact me, I am looking at the moment 🙂 .

To continue the conversation, feel free to contact me.

June 1, 2016
Brocade Advances Automation Leadership with Open Network Automation Platform
Brocade Workflow Composer Transforms IT Operations with DevOps-Inspired, Cross-Domain Network Automation to Increase Business Agility
SYDNEY, Australia — May 26, 2016 — Brocade (NASDAQ: BRCD) today announced Brocade Workflow Composer, a new server-based network automation platform that enables enterprises and cloud service providers to improve their IT operations and drive greater business agility. Brocade Workflow Composerprovides DevOps-style network automation for provisioning, validation, troubleshooting and remediation of multivendor networks — while integrating with workflows across multiple IT domains for end-to-end automation.

As companies move to digitise their business, they need an underlying network architecture, including automation, which supports business agility. This New IP architecture enables the network to become a platform for innovation and for developing, delivering and securing new applications. According to a global CIO survey [1], 75 percent of respondents stated that the network is impacting their organisations’ ability to achieve business goals. The lack of network automation and integration of the network with other IT operations and tool chains is the single biggest inhibitor to capitalising on digitisation.

Automation has been a long-standing hallmark of Brocade solutions, starting with storage networking in 1995. In 2009, Brocade extended its automation leadership with the industry’s first Ethernet fabric comprised of Brocade® VDX switches featuring Brocade VCS® Fabric technology. Earlier this year, Brocade expanded its data centre fabric portfolio with Brocade IP Fabrics, an open, standards-based design that leverages cloud-proven BGP-based architectures, with automation delivered by the new Brocade Workflow Composer. More recently, Brocade acquired StackStorm, an innovator in event-driven automation software, to help bring DevOps-style automation capabilities to the network. StackStorm technology is now a foundational element of Brocade Workflow Composer.

“Embarking on the digital transformation journey can be overwhelming for most organisations requiring changes to the network infrastructure, organisational and operational models and new skill sets,” said Phillip Coates, systems engineer manager for ANZ, Brocade. “However, when broken down into digestible and evolutionary steps, organisations can navigate the journey at their own pace. One of the first places to start is to bring a new approach to network automation. With Brocade Workflow Composer, Brocade is bridging IT silos and enabling end-to-end automation of cross-functional IT workflows.”

A Workflow-Centric Approach to Automation Increases Business Agility
To enable cross-domain automation, organisations must think in terms of workflows. Workflows have emerged as a fundamental part of IT operations within hyperscale cloud providers because they are proven mechanisms for converting business rules and policies into IT services delivered at scale. Since a workflow is a collection of tasks that are often executed manually and routinely, it serves as a strong foundation for developing automation.

Workflows can be single-domain, such as provisioning a network device, or cross-domain, such as detecting a network device issue and automatically creating and assigning a help desk ticket to a network operator. With cross-domain workflow automation, the manual steps of detecting an issue, identifying the potential cause, performing remediation actions and even creating a trouble ticket or paging a senior engineer can all be eliminated. As a result, IT can minimise human-induced errors, improve operational efficiency and greatly reduce time-to-resolution. According to ZK Research, human error accounts for 35 percent of network downtime. Network automation can reduce that downtime to zero.

Key Features of Brocade Workflow Composer
Unlike other network automation solutions that are often monolithic, proprietary and only address the network, Brocade Workflow Composer features:

Software-driven lifecycle management. Brocade Workflow Composer automates the entire network lifecycle ranging from initial provisioning and validation to troubleshooting and auto-remediation.

Cross-domain integration. Brocade Workflow Composer leverages customisable integration points to recognise events from any network device, cross-domain platform or application, and then invoke action by other tools and applications to execute an automated workflow. Brocade Workflow Composer provides more than 1,900 customisable integration points for popular platforms and applications, such as Linux, Windows, vSphere, AWS, Azure, CloudFoundry, OpenStack, Docker, Kubernetes, CoreOS, FireEye, New Relic, Sensu, Splunk, ChatOps, PagerDuty and VictorOps.

Flexible workflow automation for multivendor networks. Brocade Workflow Composer enables IT agility and choice with turnkey, customisable or do-it-yourself network workflow automation—all supported in multivendor network environments.

DevOps-inspired methodologies. Workflow execution is provided by the Brocade Workflow Composer’s open, extensible, microservices-based framework. This framework leverages StackStorm, a variety of other popular, open source DevOps technologies, as well as a thriving community for peer collaboration and innovation.

Organisations that need help in transforming their network engineering and operations team can utilise Brocade NetDev Professional Services. Staffed with experienced network software developers, Brocade NetDev leverages solutions from Brocade and other vendors to help customers accelerate their automation journey.

Packaging and Availability
Acquired by Brocade in March 2016, StackStorm built a thriving open source project and community which Brocade will continue to support and contribute to under the StackStorm name. StackStorm cross-domain integrations are available today on GitHub.

Brocade Workflow Composer leverages StackStorm open source technology as well as other organic and open source technologies, while adding enterprise-class features and support. A trial version of Brocade Workflow Composer is available today. The product is currently scheduled to be released in the third quarter of 2016.

Additional Resources

Blog: Network Operators: Evolve or Get Left Behind

ESG Report: Leveraging Network Automation for Greater IT Agility and Operational Efficiency

Brocade Workflow Composer At-A-Glance

About Brocade
Brocade (NASDAQ: BRCD) networking solutions help the world’s leading organisations turn their networks into platforms for business innovation. With solutions spanning public and private data centres to the network edge, Brocade is leading the industry in its transition to the New IP network infrastructures required for today’s era of digital business. (www.brocade.com)
May 27, 2016
Five Harsh Truths About DevOps

DevOps can provide tremendous productivity improvements, but it’s not a panacea. As much as we’d want it to be the solution for all of our software development problems, here are five truths you should consider.

1. DevOps may lead to juggling job roles and coming out of your comfort zone

With DevOps, the development team is responsible for writing the automated scripts that deploy the application. That used to be manual work done by someone in operations. Just as in manufacturing, automation will eliminate jobs. Typically, that system administrator can be retrained to work in development writing those deployment scripts, but programming is a different skill from restoring databases. If you can’t adapt, it’s time to update that resume.

2. Not everything should be automated

Yes, DevOps can be made to work on any platform, including the mainframe. No, it’s not confined to projects using agile. You don’t have to use Ruby or PHP; it’ll work for Fortran applications too. But be realistic. Transforming an application to use DevOps is expensive. You should not invest in something that is slated for sunset, or only has one release per year to fix some minor bugs. Legacy systems are considered legacy for a reason. Focus on the top 20 percent of your portfolio instead.

3. Developers will need to get their hands dirty

Some developers like to do one thing, and one thing only: write code. Grudgingly, they’ll test; they’ll attend team meetings if they have to; and, if tortured, they’ll write documentation. But forget about getting them to create the installation scripts.

That mindset is not going to fly in a DevOps world. Developers need to think about how the application will run as a nonfunctional requirement, and treat it with the same level of importance as security, performance and data privacy. Unless they feel ownership of the installation process, monitoring the environment and collecting end-user feedback, you have not really implemented DevOps.

4. Operations no longer can be ticket-driven

Usually, the operations team doesn’t get involved until someone from the development side opens a ticket for deployment. That’s too late for any kind of meaningful integration. The answer is not opening a ticket three months earlier; tickets are meant to be assigned, worked and closed within a short period of time. The type of collaboration needed between development and operations is far more complicated, and should be treated like it’s a project. That’s really difficult for most operations organizations. It’s not part of their culture, and they’re not staffed for that.

5. Your process is really, really bad

As soon as you start automating your software delivery process, you’ll find out exactly how inefficient it really is: duplicate reviews, inconsistent forms, competing standards … When you take away the people layer, you can’t hide the crazy anymore. You don’t want to take what you have today and force it into an automation tool. You will need to standardize before you optimize, and optimize before you automate. That means a lot of different towers will need to cooperate—something that won’t happen unless there is an outside party driving the cooperation with a big stick.

To learn more, check out the hangout with industry experts: What will DevOps look like in three to five years?

What aspect of DevOps do you think people are uncomfortable to talk about? Leave a comment below or contact me @baspluim

About the author / Jan S (Bas) Pluim

Bas Pluim is a Cloud/ DevOps solution architect in IBM’s innovation practice. He works with clients on infrastructure solutions, process re-engineering, automation tools and transforming the overall software delivery process. Bas is also a member of the IT Specialist certification board, a gaming enthusiast and an advocate of social media tools

May 24, 2016
Continuous Delivery and Deployment: A Tale of Two Models

EnerNOC (Energy Network Operations Center) is an energy intelligence software provider with more than 1,300 employees in its Boston headquarters and international locations. The public firm helps global customers to efficiently consume energy. “Our mission is to change how the world uses energy, helping customers to improve profitability and visibility for better decision-making and sustainability by focusing on how they buy energy and when and how they use it,” says James Nichols, principal engineer and cloud architect at EnerNOC.

A regular user of Splunk (the company uses other DevOps tools and platforms as well, including AWS and Oracle), EnerNOC has fully automated monitoring in its perpetual drive to move past continuous delivery and on to continuous deployment.

Embedded in EnerNOC’s story of why it uses continuous development and continuous deployment models is a chapter on its move to fully automated monitoring.

EnerNOC DevOps Models

EnerNOC’s DevOps model is in constant transition toward full automation. Currently, it has fully automated deployment, fully automated monitoring and semi-automated quality assurance (QA), which pull the company’s continuous development approaches.

A change to “builds on commit” of smaller applications within the legacy software monolith at EnerNOC came hand in hand with further automation, targeting what the company calls “trustworthy continuous deployment,” Nichols says. That was the crux of EnerNOC extending itself into deployment automation.

Wherever the model is not fully automated and continuous in deployment, it remains continuous delivery. “Most of the variance between the approaches is due to aspects of the delivery pipeline that still have manual processes; these parts are usually comprised of the remaining manual testing procedures,” says Nichols.

Change Control Necessitates Two Models

EnerNOC develops and maintains mission-critical software that must pass through a change management approval process. This process includes communications within and across the company and customers, as well as user acceptance testing. This is where the manual testing comes in. Only at the end of this approval pipeline can EnerNOC pass the software baton into the hands of production.

Due to EnerNOC’s tightly yoked systems and the manual testing, change control remains, along with a mix of continuous development and semi-automation with fully automated continuous deployment. “DevOps is a cultural movement as well as a technical one, and so the existing model could evolve with the organization’s quality and maturity evolutions,” says Nichols. If the manual testing and the relationships between systems give way to more fully automated testing and more amenable relationships, QA could more closely embrace full automation.

Monitoring Challenges/Achievements

EnerNOC’s application diversity includes hybrid software with a foothold in the Amazon cloud, in EnerNOC’s data centers and on-premises. “We use three tiers on top of Oracle in the data center. In AWS, we use a combination of big data, cloud-friendly and serverless technologies,” says Nichols. Business transaction data traverses many machines that belong to service groups’ stakeholders. “The distributed, decentralized architecture makes troubleshooting, performance management, security operations and production operations challenging,” notes Nichols. Better monitoring was the answer.

EnerNOC wanted to offer nontechnical users easy access to systems-level monitoring data with transparency into the many systems through a single pane-of-glass dashboard with readily available system notices and troubleshooting abilities. “We would use Splunk to do this and then feedback everything we learned from using Splunk into driving our DevOps and engineering processes into the future,” he says.

The EnerNOC DevOps performance team built the dashboard that accompanied the Splunk service, ensuring it would register the performance metrics that are of value to the organization. The performance, product management and development teams maintain an awareness of the metrics and receive the related alerts. “The development team incorporates the ongoing metrics early in the development life cycle to improve application performance,” says Nichols.

EnerNOC first applied Splunk to web logs for operational capacity planning, then used Splunk to give the Ops team transparency into the function of its new real-time data product that was working in production. As new products came on board, the organization added them to Splunk to provide Ops teams with dashboards, at the urgent request of Ops team members.

Current Status: All Systems Good

EnerNOC applies Splunk to test and production systems for views into consistency and acceleration. “Splunk is our primary alerting infrastructure for low-level system metrics like CPU utilization as well as for complex business transactions that span multiple systems,” says Nichols. Splunk has eyes on application health, cluster state across machines, expected systems tasks and suspect or known-bad system behavior, as well as the presence or lack of any significant event.

May 20, 2016
Best Practices for Cloud Deployment

When it comes to deploying apps in the cloud, the conventional wisdom is that DevOps is not required if you’re a small or midsized company. The cloud is designed to enable virtually anyone to set up the infrastructure they need, quickly and easily, without a lot of experience in the intricacies of managing data center operations or hardware. But then, by the time the company realizes it needs DevOps, it’s almost too late—the cloud infrastructure has become complex, costs have exceeded expectations and precious time is being wasted.

To ensure you’re able to optimize cloud, or hybrid cloud, deployments—and avoid many of the problems that other companies have experienced—there are a number of best practices you must employ within your company.

Best Practice No. 1: Realize You Need DevOps from the Start

When you manage your own hardware, you know exactly what the deployment costs are and what resources are available, and have a clear understanding of how they are interrelated. The abstraction of the cloud makes the initial setup a snap. But typically there is no formal structure involved: Individual developers can initiate new instances whenever they need them, without considering the bigger picture of the resources already in use, how they’re used or what is being planned by the organization.

Spinning up unnecessary cloud resources not only costs a company more in the long run, it also leads to a cloud sprawl that requires time-consuming, manual processes to maintain. Think about the challenges your company would face if it had thousands of machines created by different processes but you didn’t know who created them, what they were being used for or even if they were still needed.

By employing DevOps from the start, you will be able to balance the needs of developers with the availability of resources in the cloud to ensure expenses do not run out of control and you understand exactly how the cloud infrastructure is being used now and into the future.

Best Practice No. 2: Have a Process and Purpose

As you create your cloud infrastructure, you need to have a clearly defined purpose, as well as a process in place for deployment and ongoing management. You will need to keep track of what resources are being used and their designated purposes. Some cloud vendors, such as AWS, offer tagging capabilities that help you keep track of what services each machine was created for and whether they are being used for production, backup, load balancing or testing.

Regardless of whether you use tagging capabilities or maintain a separate database of cloud resources, knowing what you have purchased and how it’s being used will provide needed visibility, ease of management and greater control of costs.

Best Practice No. 3: Leverage Automation

Automation is important—both in the cloud and on premises—to streamline routine practices, make sure they’re applied consistently and eliminate potential errors that are common with manual processes.

When deploying on multiple clouds or in a hybrid environment, it is important to understand that tags used for one cloud provider won’t work on another. The process is typically broken into three steps: inventory creation, system preparation and automatic deployment. Many cloud systems allow you to create a dynamic inventory directly from the servers, eliminating the need to create or maintain it manually. However, deployment over multiple clouds require an abstraction of this inventory; it is common to create a static inventory in a unified format from the various dynamic inventories.

Best Practice No. 4: Design the Network with Security in Mind

Poorly designed networks are almost impossible to secure. Problems often arise when companies allow their networks to grow organically rather than being planned out in advance. A number of companies have employed the best practice of segregating the services network and the management network. The services network includes all the components required to provide services to your customers and users, while the management network is comprised of what you need internally to manage the network. Separating these networks provides greater assurance that unauthorized users cannot gain access to your systems via the Internet.

Another part of security is key management. Depending on the cloud provider, keys can be widely available or have a root password that is open to the world. In fact, many hackers routinely scan Github for keys developers accidentally check in. With having to manage so many keys, you may struggle to ensure that they’re not exposed. DevOps plays an important role in keeping the keys locked away, then providing them to a select group of individuals who can use them only in production.

Best Practice No. 5: Be Strategic About Monitoring

When you’re deploying hardware on-premises, it’s a common practice to define in advance what to monitor. But monitoring in the cloud is quite different, with basic defaults that produce vast quantities of data—much of which may not be relevant to your organization.

Taking a cue from how you handle monitoring on-prem, you’ll want to design cloud monitoring to fit your company’s unique needs and filter out the information that isn’t pertinent to your network or operations.

Best Practices Go a Long Way

Keeping in mind these best practices and employing a DevOps model from the start will enable your company to get the most benefits from its cloud deployment regardless of its size, and avoid the costly and time-consuming challenges that arise without proper advanced planning and a strong focus on network design and processes.

About the Author/Roger Fulton

Roger Fulton is Iron.io’s director of Infrastructure and Operations. His background includes broad experience in software architecture and operational planning. He manages international teams to deliver highly available products for end users and highly performing in-house infrastructure. Prior to joining Iron.io, Roger served as head of application and infrastructure engineering at Kaybus. He has also held technical leadership roles at JRV Consulting Group, Cybrata, Invisible IT and Clarus Systems. Roger holds a Masters from the Queens University of Belfast, having graduated with highest honors.

May 20, 2016
Data Protection for Operational Reporting Platforms
Making real-time business decisions based on data from social, mobile and cloud platforms is a requirement in today’s world. That’s why an ecosystem of next-generation operational reporting platforms such as Spark, Apache Cassandra, Kafka, Docker, Mesos, Marathon and more have come to life. Enterprises are using these technologies to re-architect their data storage and processing frameworks to achieve lower costs, scalability and faster response times. However, as enterprises onboard business-critical applications on this next-generation technology stack, they need to prepare for failures or errors that may disrupt customer-facing applications.

One of the key benefits that operational reporting platforms provide is real-time visibility, which allows businesses to make strategic decisions based on actionable insights. However, the new applications generate a large amount of data and archaic operational reporting platforms (those based on traditional data warehouses, data marts and ETL logic) are unable to catch up and, as a result:
- Business decisions are based on stale data
- Fragmented data increases the risk of errors
- Multiple copies of data increase storage costs
These issues have driven many progressive, data-centric enterprises to re-architect their operational reporting platforms. In this new architecture, Kafka serves as a persistent message bus that ingests data from relational stores and provides failure resiliency and message buffering. Spark or Spark Streaming may be used for transformation, aggregation and other lightweight stream processing. These processing nodes may be deployed in Docker containers. Finally, Apache Cassandra serves as a distributed data storage layer that provides linear scalability and high-availability. Kafka also enables the ability to add multiple data consumers such as Hadoop/HBase through Flafka.

However, as enterprises adopt operational reporting platforms, they want to ensure that bad data feeds and human error do not result in data loss. Apache Cassandra natively replicates data but that does not provide the capability to go back in time to recover data efficiently and at scale.

As enterprises onboard their critical applications to achieve development agility, scalability and lower operating costs, they must also explore data protection solutions that are specifically dedicated for scale-out databases.
May 18, 2016