devopsy.kinsta.cloud

Author: contributor

DevOps Resolutions For The New Year

As we welcome in the New Year, it’s natural for us to set both personal and professional goals. While many are adding the obligatory eat healthier, spend more time with family, and exercise more to their personal list, some of our client are adding DevOps resolutions to their professional goals.

During a recent customer roundtable discussion, we asked our clients what are their DevOps goals / resolutions for the New Year. Here are a few that stood out from the crowd.

“Focus on fewer problem areas (bottlenecks)”
-Ben S., VP, Major European Auto Manufacturer

This is my favorite. It’s counterintuitive but makes perfect sense. With a topic like DevOps, it can feel over whelming at times. It’s for easy organizations to create a litany of things to do. Many enterprises hesitate to invest in DevOps not because they don’t know where to start, but rather they have too many choices on where to start.
So this year, listen to Ben’s advice: pick the top 2- 3 areas that can make a positive impact to your organization. That may be continuous integration, or automated testing, or automated environment configuration. Whatever the area, just focus on a few areas, rally your team, and stick with it.
If you are struggling with what to pick, use time, pain, and / or cost as your prioritization criteria. Even if you get it wrong, the impact on one part of your Software Delivery Lifecycle (SDLC) will highlight the bottlenecks in other areas.

“Spend time talking to the people in trenches”
-Jay S., Senior Director, Leading HealthCare Provider

Jay hit the nail on the head. Too often we only focus on executive messaging and alignment. While that is important, talking to the people that are actually living this day in and day out is just as important. The insights you gain from them will be invaluable as you create your DevOps strategy and will go a long way when you try to get their buy in. Also, don’t just stick to the people within your silo. Talk to business, development, and operations – to get a holistic point of view
If you are going to start a DevOps cultural revolution, you might as well get both top down and grass roots support. It’s the own way it is going to work.

“Connect with others who have been there done that”
-Ann H., VP, Major Telecommunications Company

DevOps is a paradigm shift that challenges many of our long-held beliefs. It is a mind-set change and the only way to break the old way of thinking is to learn something new. So as the New Year unfolds, consider replicating Ann’s strategy: plan to connect with others who have been there and done that.
We are at a stage in the market where many organizations have jumped in with both feet into DevOps. Some have been successful and continue to invest, and others not so much. Talk to both, you can learn a lot on what to do / not to do.

“Stick To Your Guns”
– John B., Snr. Director, Leading Technology Firm

DevOps fails in an enterprise, not because of a flawed strategy, or mis matched tools, or even a bad process, but due to death by a thousand cuts. For example, breaking the continuous integration protocols and letting bad code pass through, increasing the batch size due to scope creep, not checking in code daily, and / or using non-standard pattern for deployment. Each cut, though small in nature, undermines the entire process. If done once in a while, it might be ok. But do it too often and you will literally see your DevOps investment vaporize into thin air.

So if you were to add just one resolution to the list, I would recommend you follow John and add “sticking to their guns”. Whatever process that you and your team have decided on, stick to it. The more you stick to it, the easier and more natural it will become. Do it often enough and you will be setting the right tone for yourself, your team, and towards DevOps success.

What are your resolutions for 2015?

About the Author

Mustafa Kapadia is Service Line Leader for IBM’s DevOps practice, a business advisory practice focused on helping large enterprises transform their software & application delivery. Through DevOps, organizations can get software to market faster, with better quality, less defects, reduced risk and shortened lead times.

Prior to joining IBM, Mustafa was a management consultant with Deloitte’s Strategy & Operations practice. He lives in San Francisco Bay Area, is an avid blogger (when time permitting), a speaker, and and advisor to start ups.

You can follow Mustafa on LinkedIn: https://www.linkedin.com/in/kapadiamustafa and Twitter: https://twitter.com/mkapadiatweets

January 5, 2015
Top 5 DevOps and Operations Challenges in 2015

DevOps and operations teams are under increasing pressure from tech-savvy, app-centric business users to collaboratively solve complex business problems with IT. Adding to the battle, the pursuit for the perfect synchrony between software development and IT operations is still ongoing, and striking the balance won’t happen any time soon. New tools, technologies and processes change and grow at a problematic pace – leaving us with no option, but to accept and address all upcoming challenges.

Teams must keep up the pace to succeed, but operations teams cannot handle it solo and must work closely with DevOps. The operations group is responsible to drive flawless, organization-wide execution – this was a straightforward request when the systems were configured and maintained by operations. However, recently, development became more collaborative, with involved users demanding more, using the newest tools and making the process harder to manage. Additionally, the product environment of business users has rapidly moved to portable devices, which are self-contained and hungry for enterprise-wide cloud services. This shift has created five principle challenges. These are my top 5 DevOps and Operations challenges for 2015:

Challenge 1: Building the DevOps Culture

Culture is an abstract group of ideas and behaviors, which makes it hard to identify a standard collection of requirements. It is much easier to decide which tools to use and how to use them in the most efficient way. Tools are material objects, but the people who use these tools are much more complex and difficult to change. As DevOps is all about continuous change – including reconfiguring a company’s core culture – there is a steep hill to climb.

Replacing traditional development and operations silos with cooperative teams of people from both camps can only get us so far. While DevOps is supposed to intentionally foster collaboration and results-driven effort, it needs the support of operations. Until operations can identify how to embrace the new mentality of culture-fostering activities into its daily regimen, there is no way to help this aspect of DevOps grow.

Challenge 2: Meeting the Quick-Fix Tool/App Mentality

While most end users now rely on a wide palette of tools to get their job done, they often circumvent the IT department to get the tools they prefer. As many are open source or offer initial trial versions, unsanctioned tools can be adopted without any internal oversight – leading to major issues down the road.

Making matters worse, business users believe everything can be solved with an app, but that is often untrue. While it is the overwhelming consensus that every tool should have an easy-to-use app, making the app a useful, secure one is much more complex. Even if there is an “app for that,” it must be properly integrated to back-end systems as part of the cohesive enterprise infrastructure. DevOps must balance tool delegation and management early on.

Challenge 3: From Legacy to Cloud

Technology-savvy business users may want to use the latest tools, but that is a tall order for operations. When it comes to DevOps tooling and configuration management, there is still a lot of confusion as to which tools do what, and how they play with each other – if it all. Tools need to be compatible with existing solutions and technologies used by the company, and that is becoming increasingly complex with the rise of hybrid environments.

New hybrid environments include on-premise tools, IaaS tools like Amazon, Google or Azure, PaaS compute services, and SaaS business applications. Somehow, DevOps must ensure that they all work well together and deliver a seamless experience for the business user.

Challenge 4: Networking

Network engineers – if they are applying automation at all – are either doing ad-hoc scripting or tap a dedicated software engineering team to build homegrown systems for their use. Inflexible networks run rampant in today’s IT environments.

Looking for an example? Even vLANs have a spider web of interdependent IPs, ports, routers, routing tables and perimeter networks. Changing, breaking, moving or replicating any part of that intricate web often seems impossible.

Challenge 5: Marginalized and Changing Budget

Taking another perspective: Will customers pay more for network automation tools from the equipment vendor to make the networking user experience better? If a networking customer is spending a lot of money on the physical equipment, they often do not want additional software tools from the vendor, even if the network engineers do. The gap between the network engineering teams that need the tools and the decision makers who pay for them is widening.

While capital expenses may be separate from operational ones, CAPEX spends are now often mirrored by OPEX ones. Trying to quantify spend on an on-premise project management tool with a SaaS application is a major headache that can no longer be left solely in the hands of the CFO. This shift impacts the structure of IT budgets – it could even take away IT’s control if OPEX spend becomes more distributed.

What can be done to overcome these growing problems? Taking the following five steps should help you start off 2015 in the right direction.

Collaborate Wisely:

It might seem to get a little tense at times, but open forum discussion is well worth it. Collaboration allows teams to learn from each other and suggest possible solutions. It also encourages critical thinking and creativity. When you have different people working together on a project, you get more expansive creative input. To manage the potential leadership glut (i.e., the excess of people trying to lead the group and not that many willing to slow down and focus on a specific task), break the traditional decision-making process chain and turn it into a continuous flow of fresh thinking and ideas.

Empower Users:

Let your business users pick their own tools, but make sure that they report them to you. It’s a good idea to consider a library of vetted tools available for them to pick from. Encourage business users to suggest new tools so they have a sense of decision power. However, keep the tools current and make sure they actually address user needs.

Speak Accounting:

Try not to let OPEX issues become obstacles in tool adoption. If IT can help finance understand what is going on in the world of technology licensing vs. cloud service, it can fast track the inevitable shift, and ease budget reporting and bottlenecks.

Break the Immutable Network Infrastructure:

With immutable network infrastructure, you run the risk of being locked into vLAN and physical devices. What is the solution then? Leverage advanced virtualization tools (i.e. converting legacy systems to virtual machines) and take Software-Defined Networking (SDN) on board. With a centralized, programmable network that can automatically and dynamically address changing requirements, you can reduce both CAPEX and OPEX by enabling algorithm control of the network and use ASIC-based networking hardware supporting pay-as-you-go models. Deliver agility and flexibility by helping organizations rapidly deploy new applications.

Prioritize Analytics:

Analytics should make IT’s life easier. Build analytics and monitoring into everything from the start and think about it as “making sense” of the data or logs being collected. This information helps IT determine the true business impact of a performance issue to better prioritize where and what to troubleshoot. A strong analytics platform, where all application and tools data can be stored, gives operations the systems visibility it requires. Become a facilitator while maintaining all needed operations practices.

There will always be challenges ahead as we work to get smarter/ faster and develop better systems and applications. However, the right analytics tools and collaborative processes designed for cloud-enabled, distributed environments can make 2015 much easier than years past.

About the Author

Trevor Parsons, as chief scientist and co-founder at Logentries, Trevor Parsons is responsible for product strategy and direction. He works closely with customers and partners to continuously understand what they need, and to validate product market fit. Trevor also leads the product management and UX teams and assures the best possible user experience. Trevor enjoys speaking at local dev ops meet-ups and events, and is always looking for how log data and analytics can be applied in more and more powerful use cases. Trevor was a post doctoral researcher and member of the Performance Engineering Lab at the School of Computer Science and Informatics in University College Dublin, Ireland. He received a PhD from University College Dublin for his thesis titled “Automatic Detection of Performance Design and Deployment Antipatterns in Component Based Enterprise Systems.” You can follow Trevor on Linkedin and Twitter at https://www.linkedin.com/in/trevparsons and https://twitter.com/trevparsons.

December 19, 2014
The Top Metrics You Need to Track to Improve Operational Performance
Living in a data-rich world is a blessing and a curse. Flexible monitoring systems, open APIs, and easy data visualization resources make it simple to graph anything you want, but too much data quickly becomes noisy and un-actionable. At PagerDuty we’ve thought hard about what you should monitor and why from a systems perspective, but what about monitoring data on your operations performance? We’d like to share some specific metrics and guidelines that help teams measure and improve their operational performance.
1. Raw Incident Count
A spike or continuous upward trend in the number of incidents a team receives tells you two things: either that team’s infrastructure has a serious problem, or their monitoring tools are misconfigured and need adjustment.

Incident counts may rise as an organization grows, but real incidents per responder should stay constant or move downward as the organization identifies and fixes low-quality alerts, builds runbooks, automates common fixes, and becomes more operationally mature.

When looking at incidents, it’s important to break them down by team or service, and then drill into the underlying incidents to understand what is causing problems. Was that spike on Wednesday due to a failed deploy that caused issues across multiple teams, or just a flapping monitoring system on a low-severity service? Comparing incident counts across services and teams also helps to put your numbers in context, so you understand whether a particular incident load is better or worse than the organization average.
1. Mean Time to Resolution (MTTR)
Time to resolution is the gold standard for operational readiness. When an incident occurs, how long does it take your team to fix it? Downtime not only hurts your revenue but also customer loyalty, so it’s critical to make sure your team can react quickly to all incidents.

While resolution time is important to track, it’s often hard to norm, and companies will see variances in TTR based on the complexity of their environment, the way teams and infrastructure responsibility are organized, industry, and other factors. However, standardized runbooks, infrastructure automation, reliable alerting and escalation policies will all help drive this number down.
1. Time to Acknowledgement / Time to Response
This is the metric most teams forget about– the time it takes a team to acknowledge and start work on an incident.

While an incident responder may not always have control over the root cause of a particular incident, one factor they are 100% responsible for is their time to acknowledgement and response. Operationally mature teams have high expectations for their team members’ time to respond, and hold themselves accountable with internal targets on response time.

If you’re using an incident management system like PagerDuty, an escalation timeout is a great way of enforcing a response time target. For example, if you decide that all incidents should be responded to within 5 minutes, then set your timeout to 5 minutes to make sure the next person in line is alerted. To gauge the team’s performance, and determine whether your target needs to be adjusted, you can track the number of incidents that are escalated.
1. Escalations
For most organizations using an incident management tool, an escalation is an exception – a sign that either a responder wasn’t able to get to an incident in time, or that he or she didn’t have the tools or skills to work on it. While escalation policies are a necessary and valuable part of incident management, teams should generally be trying to drive the number of escalations down over time.

There are some situations in which an escalation will be part of standard operating practice. For example, you might have a NOC, first-tier support team or even auto-remediation tool that triages or escalates incoming incidents based on their content. In this case, you’ll want to track what types of alerts should be escalated, and what normal numbers should look like for those alerts.

About the Author

David Shackelford is a product manager at PagerDuty, the leader in operations performance management. David works with teams across the company to plan, build, and ship features that improve operation teams’ quality of life, decrease time to incident resolution, and ultimately improve uptime for PagerDuty customers. Prior to PagerDuty, David worked in education technology, creating integrations between school information systems and digital content, and as a Teach for America corps member, teaching Mathematics in San Francisco public schools. You can follow David on LinkedIn: https://www.linkedin.com/in/dshackelford and Twitter: https://twitter.com/dshack
December 17, 2014
DevOps in Action: MYOB

This is the first in a new series called DevOps in Action. A look at organizations who are putting DevOps into practice. Does your organization have experience putting DevOps into action that you think our readers might find useful or interesting? If so contact us at editor@devops.com and perhaps we can feature your story here.

Australia-based MYOB is a long-established developer of accounting systems and related software. During the last several years its original focus on desktop software has changed, and it now offers cloud-enabled and pure SaaS products. This change had two significant impacts on the company.

Firstly, it meant that instead of simply delivering packaged software to customers, MYOB now has to manage the infrastructure that its SaaS products run on.

Secondly, the traditional annual update cycle primarily driven by tax and compliance issues goes out of the window in a SaaS world, where the norm is continuous improvement based on customer feedback.

“Traditional operations management just doesn’t keep pace with that,” said MYOB business division product delivery manager John Sullivan. “The pace of change has changed dramatically.”

Not only are there more ideas for improvements and new features, but the changes are increasingly “externally fuelled,” he said. This makes it difficult to plan infrastructure requirements very far ahead.

MYOB joined the DevOps movement as it was seen as a way of releasing better software more quickly, and of ensuring that the hardware could be scaled to meet the demands of new software releases.

At some companies, DevOps is about building infrastructure for developers to use, or having operations build infrastructure in parallel with software development, he said.

But MYOB now has operations staff that literally work alongside developers to translate features requirements into infrastructure requirements, and then use repeatable methods to build that infrastructure.

“We scale our infrastructure at the time features are being built,” said Sullivan.

The company uses multiple cloud providers, reflecting the varying needs of its 40-plus products. AccountRight Live (a desktop program that stores its data in the cloud) was written using .NET, so it made sense to use Azure for the cloud element.

And it has adopted a microservices architecture so a single implementation of a function can be used by multiple products (read more about this in ‘MYOB moving ahead with microservices’ http://www.itwire.com/business-it-news/accounting-software/65878-myob-moving-ahead-with-microservices). These microservices mostly run on Amazon Web Services.

Everything else runs on virtual cloud infrastructure provided by Macquarie Telecom.

Using these larger providers means MYOB can be confident that the physical resources are available whenever they are needed, without having to invest large amounts of money in private infrastructure. Having the infrastructure “come alive” at the moment any particular piece of software goes live provides significant savings.

MYOB had an important advantage when starting its DevOps journey: the need for operations staff was new, as it was going from being a traditional software vendor to also offering SaaS products. Sullivan said that made it easier to put development and operations staff together than it would be for an organisation that has a history of managing its own infrastructure, especially with a steady release cadence as opposed to the rapid tempo associated with cloud services.

“We were very lucky that we needed to manage infrastructure for the first time,” he said.

But there were some cultural issues, particularly around making the delivery teams understand they had to take infrastructure into account when building features, as it is important that the cost of delivering the service is kept within reasonable limits. So developers need to tap into operational expertise to ensure that the patterns they use will scale adequately, and that the hooks for monitoring and managing the software are included.

MYOB started with an independent DevOps team with an oversight role, but this was less than optimal, Sullivan said. “If you say you have a ‘DevOps team,’ you probably haven’t” because the closer that development and operations staff are, the better the systems become – getting the right design means looking at application features from several different angles.

Those on the operations side need to be able to advise developers about the software patterns and tools that will be able to handle increasing load. Not only does that potentially avoid the need to build complex failover arrangements, it also saves you running into a situation where you have to re-architect the whole system when demand reaches a certain level. “That’s the advantage of having operations and development people close together.”

MYOB uses the Agile methodology, and infrastructure concerns and requirements are included on feature cards so the software and infrastructure can be built together. As the company gets better at DevOps, developers become increasingly aware of infrastructure requirements and so they build as they go – “we’re just about there with one of our teams.”

“You need operations people with the mindset of a developer … or you’re going to struggle,” he said. For example, you want systems that can automatically scale on demand, and that requires scriptable deployment processes: “a difficult thing to do at times.” Fortunately for MYOB, the company was able to find the right people. It can be hard for development staff to adapt to DevOps, Sullivan suggested, but it’s even harder for traditional operations staff.

One of the ‘tricks’ used by MYOB to build the DevOps culture was to rotate developers into operations roles after the software they had been working on entered production in order to learn about operational requirements. “Then they actually feel the pain of phone calls in the middle of the night if something goes wrong.”

Another is that the company favours ‘self-reflective teams’ (regardless of function) that review what has or has not worked for them during the last two weeks, and adapt their practices accordingly. DevOps is just one area where this practice is used. Sullivan notes that because the environment is so dynamic, ideas are not rejected permanently. So teams can decide that just because a certain technique or idea did not work on a previous project, it might be successful this time because the situation is different in a relevant way.

Early in the transition MYOB filled some key roles with DevOps practitioners that had Azure and AWS experience. But Sullivan warned “there are very few conferences or training courses on DevOps … so most companies don’t do DevOps correctly.” Consequently, special care is needed when hiring people that claim experience in the field.

“The term ‘DevOps’ is overused and sometimes misunderstood.” So some people say they have DevOps experience when all they really know about is doing scripted builds, and do not have the skills and practices to be embedded in a DevOps team.

“There are few people that have the right mindset,” said Sullivan, so it is very important to check a candidate’s skill set against your requirements – and on the flip side, to take the steps necessary to retain your good DevOps staff.

So did MYOB get what it wanted from DevOps?

One initial goal was to go from the traditional software-only annual release to a six-week ‘software and infrastructure’ cycle. “We met that pretty quickly,” said Sullivan, and the company is now targeting ‘any time’ releases for online services with minimal or no interruption. This will allow the company to pick and choose which new features are to be released and when according to marketing priorities, rather than being hamstrung by arbitrary release cycles.

The other was to achieve “one click deployment,” that is to be able to release code and infrastructure to customers with no manual tasks beyond initiating the process. The automated process should either run to completion, or back itself out if something goes wrong. This has been achieved for most products, the main exception being those for mobile devices because the requirements of the various app stores get in the way.

“Other companies may have different metrics, but ours were about release capability and infrastructure efficiency,” said Sullivan.

Asked if MYOB had any regrets about going down the DevOps route, the answer was succinct: “None whatsoever.”

And his advice for others considering following this path sounds sensible for any part of working life. “Embrace the idea that you have to grow and learn, and keep making the best decisions you can at every step.”

About the Author:

Stephen Withers is one of Australia¹s most experienced IT journalists, covering everything from gadgets to enterprise systems. In previous lives he has been an academic, a systems programmer, an IT support manager, and an online services manager. Stephen holds an honours degree in Management Sciences, a PhD in Industrial and Business Studies, and is a senior member of the Australian Computer Society. You can see his Linkedin profile here: http://au.linkedin.com/pub/stephen-withers/0/222/687/

December 10, 2014
DevOps: As critical for the Systems of Record as for the Systems of Engagement

There is a lot of talk about DevOps, and how it improves the process of delivering business value. Much of the discussion of DevOps has been focused on web based companies such as Facebook, or the mobile and distributed development teams. But DevOps principles apply just as well to the traditional Systems of Record, z/OS applications.   Many of these applications have been around for a long time and have very mature development and deployment processes. These processes include the requirements for separation of duties and audit tracking. Over the years these processes have developed into lengthy procedures, to be comprehensive, the net effect of this has been to reduce the number and frequency of the deployments to reduce risk. However, there is a better way.

In the recent blog posting by Ashok Reddy he talked about the 8 critical DevOps Practices: innovate, deliver, repeat. All of these are as applicable to z/OS applications as any other, for this blog I want to focus on just a few of the practices, Loosely Coupled Architecture, Automate Testing using APIs, Transparency and Minimize hand-offs, Maximize flow.

DevOps is about bringing the entire organization together, including Development and Operations, to more efficiently deliver the business value. The practices of “Transparency” and “Minimize hand-off, Maximize flow” go to the core concepts of the people aspects of DevOps i.e., integrating the organization.   By breaking down the barriers between the organizations, people can work together to accomplish the business goals more quickly. This does not necessarily mean actually changing the organizational structure, but it does mean having the organizational structures work together as cross functional teams.

One example would be the monitoring and optimization practices. These are generally done by Operations after the applications are deployed into production. However, if instead Operations and Development worked together in the early part of development cycle to monitor and optimize the application using the same capabilities that will be used in production any issues can be addressed much earlier in the cycle. This also helps both teams learn what is required to monitor and manage the application long before it is ever deployed to production. In many z/OS environments today, the same infrastructure is already available in the test environments, but it is seldom used.

A second example is to have Operations define the deployment automation such that it can be done by Development in early stages and turned over in later stages. This ensures continued testing of the deployment capability and a validation that by the time the deployment is done in production; the deployment process itself will be well tested. Through this integration of Development and Operations you remove the hand off that currently happens only at the final stage just before production deployment.

The practice of loosely coupled architecture is key when building multi-platform applications. Having clear interface definitions between the different parts of the application is essential of a loosely coupled architecture. It is easier to allow the capability to build independently, tested at the interface level and then brought together when the parts are ready. By defining clear interfaces between application components and properly handling forward and backward compatibility, there is less of a requirement for coordinated simultaneous deployments. This allows teams to be less dependent on other parts of the application during the development process, and allows for easier automated testing.

The practice of Automate testing is critical to any DevOps transformation. It provides the capability to improve the speed of delivery while reducing risk. Moving to agile development practices while still requiring people to perform manual tests, will not allow the quick iteration on the capability. Even for existing applications you can generate interface tests to begin to build up automated regression test cases. Automated tests can be generated based on interface definitions such as a JSON schema or a COBOL copybook for new development. The increased focus on testing the interfaces early helps reduce the integration problems generally found late in the development cycle.

These are just a few of the key practices of DevOps but hopefully they have started to demonstrate how DevOps is as applicable to the Systems of Record on z/OS as it is to Systems of Engagement.   To learn more read the best practices for a DevOps approach with IBM System z whitepaper or watch the replay of the November 20, IBM DevOps Symposium webcast.

About the Author:

Rosalind Radcliffe is a Distinguished Engineer within the IBM Rational organization. She is Chief Architect for CLM and DevOps. She is responsible for driving the DevOps for multi-platform architecture. This includes System z and Power system. In addition she is responsible for the architecture for the Collaborative Management capability for Enterprise solutions. This includes UrbanCode Deploy and Rational Team Concert’s support for standard Mainframe development activities.

She is a member of the IBM Academy of Technology and a Master Inventor. Prior to Rational, she was in Tivoli responsible for the SOA Management Strategy for IBM.

December 2, 2014
Closing the Gap Between Database Continuous Delivery and Code Continuous Delivery

The most important feedback I get about the current state of database delivery is directly from the DBAs I meet with. Their DevOps teams are stuck in manual database delivery purgatory. Development managers just want the database delivery to work like their software delivery already does. However, databases are a different beast altogether. Unfortunately, DevOps for Database has taken a back seat to source code, as we see the tools and methodologies perfected, leaving database automation to play a game of catch up.

Another issue is that DBAs don’t trust deployment automation like code developers do. All DBAs have been burned by breakdowns or conflicts at one point or another, and most of them just feel they need to take matters into their own hands to ensure the release is done right. We recently surveyed over 200 DBAs, and the largest percentage of them noted mistrust of automation as the biggest barrier to adopting continuous delivery for their databases. In the same survey, only half of the companies doing continuous delivery of their code said they are also doing DevOps for Database. This needs to change soon or database delivery will be stuck in the past and slow down all company releases.

Databases hold the most vital company information and they need to start being treated like they matter as much, if not more, that the code being written for a company’s software. So, the question is, how do we close that gap between source code automation and database automation?

The answer is simpler than you might think. We need to bring the same processes for source code continuous delivery to DevOps for the Database. What a novel concept!

DevOps for Database requires best practices just like source code. Deployment practices need to be enforced. Version control needs to be enforced. Safe automation processes need to be created. You need to include the database in the continuous process and feedback loops. The most important thing is to make sure you have a way to control these processes, and if something needs to be handled there will be automated notifications and red flags to point out the issue.

That being said, ensuring safe continuous delivery of databases is not so simple. Databases are not a collection of files; they hold client information, transaction data, application content, etc. To make automation really work there needs to be a solid transition code created to handle database schema structure, database code, and content such as metadata.

Furthermore, there are organizational-level changes that will be required in order to ensure proper continuous database delivery. Both DBAa and C-level management will be required to adopt these development methods for databases. It can’t be and isn’t a one-person job. Both sides need to advocate for database automation.

Everyone recognizes the importance of the database to the organization. That is the reason DBAs are so afraid to automate and management is not forcing it. It should be just the opposite! Their sentiments on database control and automation need to be more positive. The question they are asking is – what could go wrong if we bring continuous delivery to our database? The question should be – what could go wrong if we don’t bring continuous delivery to our database?

Yes, database deployment automation is not a simple process, but the fact is that ensuring continuous delivery means increased productivity, faster time to market, reduced risk for new releases, and higher quality with fewer bugs. Otherwise your database will keep lagging and eventually you can find yourself in a difficult situation down the road.

About the Author

Yaniv Yehuda, CTO, DBmaestro (http://www.dbmaestro.com/)

Yaniv is the Co-Founder and CTO of DBmaestro, the leading provider of DevOps for Database solutions which enable control of database development and deployment. Yaniv is also the Co-Founder and the head of development for the Extreme group, a leading IT services solutions provider group.

November 19, 2014
XebiaLabs Offers Free Community Edition of XL Deploy & XL Release

BOSTON – November 18, 2014 — To introduce the DevOps community to the powerful benefits of launching applications to the market via automation, XebiaLabs today announced it will offer free community editions of both XL Deploy and XL Release. The company, a leading provider of enterprise software that enables DevOps and Continuous Delivery, is committed to empowering software developers to apply agility to their application release process with delivery automation solutions that add visibility and speed to the process. Fully-functional community editions of XL Deploy and XL Release can be accessed by visiting XebiaLabs.com and will allow 30 days of full technical support.

International corporations such as General Electric, Expedia, Xerox, KLM, and Société Générale use XL Deploy and XL Release to get applications into customer hands while reducing costly errors and downtime. CIOs, development managers and professionals in quality assurance, testing and operations at these organizations view the fast conversion of good ideas into marketable software as a critical imperative that drives business success.

About XL Deploy – Agentless Deployment Automation

XL Deploy is the leading Application Release Automation software that helps organizations deliver higher quality software faster. XL Deploy is agentless, model-based and fully automates the last mile of the Continuous Delivery process by eliminating the need for scripting.

XL Deploy also features out-of-the-box integration with all leading build, provisioning, middleware and cloud solutions. The solution combines intelligent deployment automation, a rich content set covering everything from Docker to WebSphere, and seamless integration into the application delivery ecosystem.

About XL Release – Continuous Delivery Pipeline Orchestration

XL Release allows users to orchestrate their Continuous Delivery pipelines end-to-end. Users can analyze and progressively automate their release process while gaining enterprise security and control, handling pipeline dependencies, risk, and impact. XL Release is built for the enterprise with the security, access control, LDAP integration, and calendar integration one would expect delivered right out of the box. The solution offers a complete audit trail for the entire release process.

XL Release can track and measure release progress and identify challenges early on as a situation evolves, repeat or skip tasks or phases as needed. Teams can evolve towards Continuous Delivery and finally integrate release management and execution while providing a path to fully automated delivery pipelines, resulting in the faster delivery of software.

Tech Support for Community Editions of XL Deploy, XL Release

Unlike most community editions, XebiaLabs will offer free technical support for up to 30 days. Users can be up and running in 15 minutes. For more information visit http://xebialabs.com/products/ or call (866) 656-4408 or write to sales-us@xebialabs.com.

“Current software development processes cannot keep pace with the demand for the faster delivery of software,” said Coert Baart, XebiaLabs CEO. “While application development times have accelerated of late due to Agile, scrum and other methodologies, deployment best practices have not kept pace, resulting in a significant bottleneck at the point of application release.”

November 18, 2014
What is DevOps and Why Should CIOs Care?

Like the buzz-phrase: “cloud” or “big data”, DevOps is the new flavour of the month within the IT industry. You’ve probably heard of it by now – but many CIOs and heads of IT have not got to grips with what it is and are therefore reluctant to let developers take ownership and run their own operations.

A recent survey found that 23 per cent of respondents were not familiar with the term DevOps, and another 17 per cent thought it was all hype. DevOps is a method for improving the performance of the entire IT group. It is aimed at creating a better set of joined up IT outcomes for the business. As a methodology, DevOps allows all IT functions and activities, from cradle to grave, to be integrated into a seamless whole. It is an IT engineering philosophy and discipline that sees each part of the IT activity and value chain as being important to the delivery of IT products and services.

So why is it important? Well, DevOps is about culture, organisation, people, processes and tools. It helps directly address the historic silos that have existed between ‘development’ and ‘operations’. It also deals with the unhelpful and counter-productive behaviours that exist within different parts of the IT organisation that interfere with the smooth running of IT and impedes the delivery of high quality and cost-efficient products and services to the business.

DevOps provides a model for a more efficient and effective IT group; one that delivers more business value for less IT cost. By changing the way that different parts of the overall IT group operate together as a unified whole, DevOps improves the eventual outcomes for the business by making the ‘business of IT’ more efficient and more effective: “Better IT means better business”. Well-implemented DevOps initiatives will deliver rewards in terms of IT efficiency and quality improvements.

Whilst it can be a difficult sell to the business, the commercial benefits of a DevOps methodology applied to the ‘business of IT’ can bring significant improvements in productivity, consistency, flexibility and stability and must be championed by the CIO in order to get business buy in.

As promising as the DevOps movement is, especially for global organisations, some clear lessons have already emerged. Understanding these in advance could be the difference between success and failure for your DevOps initiative, and ultimately, your entire IT strategy. The alignment and collaboration of development and operations is key to a successful DevOps programme. Both groups must strive to achieve business agility through enabling change whilst preserving the stability of the existing IT infrastructure.

To achieve this the CIO must foster an environment where there is no longer two groups with opposing interests but one team that is pulling in the same direction and a culture where the individuals are cogniscent of each other’s responsibilities.

Get this right, and it will undoubtedly lead to better business.

About the Author

Paul Greer is Chief Software Architect at RedPixie. Paul works alongside the product and services teams to ensure products are high quality and enterprise ready. Paul also maintains a close relationship with RedPixie’s client base to ensure software services are delivered to a professional standard and exceed expectations. Prior to co-founding RedPixie, Paul spent 14 years in the financial services sector, holding technical leadership roles in business application development, innovation and architecture.

@paul_greer

November 18, 2014
Build security into your app development
Integrate security testing into your dev process now or else face cyber-attacks later

The way we develop software has been radically transformed in the last few years. Agility and speed are vital components for any company that wants to compete in the market. In order to achieve that it has proven necessary to break down barriers. The idea of separate silos with developers, operations, testers, and management working in isolation, sometimes even in opposition, is dated and flawed.

We’ve accepted the logic of bringing development and operations together with the DevOps movement. The recognition that testing is required earlier in the process has come. Now it’s time to apply the same logic to security. It’s time to bring InfoSec into the fold.

The security threat

Trends like cloud computing, big data, SaaS, and the BYOD end of the mobile revolution have created new opportunities for increased efficiency and productivity, but they also represent unique challenges for security. Sensitive data, financial information, and intellectual property are all exposed to risk when security is a secondary concern.

The issues that splashed back from the waterfall model for testing apply to security as well. Only passing applications through the QA process at the end of production, when time pressures to release were the greatest and it was too late to make significant changes, resulted in fixes being more expensive and software quality suffered.

If we don’t integrate security testing into the development process and make it part of the software development lifecycle now, then we run the risk of encountering exactly the same problem. We waste time retro-fitting functionality that should have been there in the first place, and we all know the pain of securing a hybrid system with legacy software that wasn’t designed with modern security threats in mind.

How to address security in software development

Automation for testing has enabled developers to move towards a continuous delivery system where new features can be rolled into live software as they are created. How do we ensure that security is maintained?
- Secure programming education is required to make sure developers are limiting and testing inputs, storing minimum data and encrypting, compartmentalizing the system, questioning requirements that may introduce security risks for little gain, limiting privileges, and analyzing and auditing the code.
- IAST (Interactive Application Security Testing) allows you to combine elements of static and dynamic techniques to run automated tests continuously on the software under development and see how it copes with malicious traffic. Since IAST monitors data inside the application it can pinpoint issues that might arise from real-world attacks, enable a useful assessment of the impact, and make it easier to remediate.
- Security analysts are needed to properly configure your tools and interpret the results. You can buy the best security tools in the world, but you have to know how to leverage them and act on the data. An external analysis can provide real insights that will boost application security.
- OWASP (The Open Web Application Security Project) is a great community where you can find innovative solutions to modern software security challenges. It can help you understand secure development standards and it’s packed with invaluable resources and advice from experts around the globe.
Bringing InfoSec on-board from the outset will help you build security considerations like this into your development pipeline. It will save time and money in the long term.

Ongoing research and modelling

It’s important to model potential threats and test for them, but you must be aware that new threats evolve and emerge all the time. Dedicated InfoSec employees will continually research and explore the new trends and risks in the security industry. Opening a direct line between DevOps and InfoSec enables them to pass along that wisdom and fold it into the mix when it’s relatively cheap and easy to do.

The security testing in your development pipeline is no more static than any other element. It has to be continually reviewed and modernized to ensure it continues to deliver results. Continuous real-time monitoring will deliver the oversight you need.

Building for the future

There’s no question that cyber-attacks will come, but if you prepare properly you can detect and nullify them with minimal effort. When considering the investment now, you must factor in the cost of lost confidence, post-mortem forensic investigation, and significant redevelopment to close any gaps in your defenses in the event of a future breach.

Once you have built solid foundations for security in your application development they will benefit every project going forward. Taking a long term view makes financial sense and results in better quality software.

About the Author

Michelle Drolet is founder of Towerwall, a data security services provider in Framingham, MA with clients such as Smith & Wesson, Middlesex Savings Bank, Brown University and SMBs. You may reach her at michelled@towerwall.com.
November 17, 2014
Expanding DevOps Beyond Dev and Ops
I’ve been involved in software delivery for longer than I care to admit, and since DevOps has come onto the scene I’ve witnessed the deconstruction of silos that were once actively built to contain separate projects and team functions. Central to this deconstruction is collaboration between Dev and Ops, ushering in new innovation paradigms. What’s exciting is that in just the past five years since DevOps has started permeating organizations, other areas within the enterprise—including technical and marketing teams—have started to recognize the high-value that comes from embracing collaborative tools and processes. DevOps has quickly evolved from sideline projects to a state of mind rapidly adopted by leading companies across various business units.

DevOps and Culture

Accompanying this notion of greater collaboration and cross-silo participation is the need for new tools to help facilitate the pace of change and collaboration. Although there are no silver bullets, you can take steps and implement technologies to help make the odds of success more favorable.

Before DevOps, hostility between Dev and Ops teams was not unusual. Dev and Ops had conflicting goals: Development to introduce change and Operations to maintain stability. The common bond was a love of technology rather than delivering quality products to customers. In my experience, technology employees care more about the technology they can experiment with than the value something they’re working on brings to customers. However, in the five years that DevOps has been on the scene, those of us who follow DevOps closely have noticed staff on both sides becoming much more customer-focused.

Common Goals

Many organizations publish their core values and say that they expect employees to live up to those values. But are those values something that employees are measured against? In my experience, they are not. Organizational values tend to end up being words to make people feel good. But what if every manager in an organization had a common goal of delivering quality products or services to customers, with the goal defined in a measurable way? Then everyone in the organization contributes to delivering the products to customers. It’s a goal is not and should not be department specific.

The departments that often see considerable delays due to red tape are:
- Human Resources
- Budgeting
- Procurement
- Marketing
Any one of these departments can result in projects getting stuck in the mud, usually due to large amounts of lead time required to fulfill a request. Delivering quality products and services frequently obviously doesn’t mean having no process, but when things need to be done quickly, procedures should be flexible enough to accommodate such requirements. The need for agility applies to all parts of an organization. It’s not just a technology problem.

Consider trying to hire someone new to work on a project. Adding an employee could involve HR, Finance and Procurement, each with their own processes and requests for lead time. The multitude of steps in their respective processes frequently take weeks, if not months. This applies to almost any department of the organization and it’s highly likely that there are processes that add significant delays to reacting to business needs.

Setting common goals and measurements is a sure way to cut red tape. Employees don’t want to be the reason something could not be done. Processes will become streamlined and departments will collaborate out of necessity, if not for the good of the organization, then at least as an act of self-preservation.

While this high-level overview makes it sound simple, most organizations outside of Silicon Valley don’t set common goals for employees. It’s up to the Unicorns to spread the word.

Using Technology to Facilitate Collaboration

The DevOps movement resulted in advances in tools and technologies that aided in Dev and Ops collaboration, and took advantage of the opportunities created by this frictionless new environment.

In order for all departments in an organization to be able to take advantage of similar opportunities, other teams will need to look at ways to optimize their workflows and tool chains.

One area where tools have reduced friction and increased collaboration is code review. Code review has a lot in common with collaborating on any other kind of content. Code review used to be done with people sitting in a room with printouts of source code receiving feedback that often resulted in holy wars about coding styles. It wasn’t effective and that was even within the same department.

Tools now allow documents to be reviewed asynchronously with in-line comments, no matter where in the world the reviewers are located. It’s one of the few examples where communicating via text rather than voice lowers tensions, which is surprising given how easy it is to accidentally upset someone via email!

A great example of this is how my working style changed once I started working with Gene Kim, the award-winning author of The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win and The Visible Ops Handbook. Rather than email documents back and forth, we’d collaborate via Google Docs. No matter what our schedules were or where we were in the world, we worked together effectively. The first time I worked like this with Gene changed the way I think about collaborating on non-technical content.

A key element of collaboration on a single set of artifacts is not having multiple copies in different systems in an organization. In fact, the 2014 DevOps Survey by PuppetLabs found that versioning production artifacts in one place was a key indicator of high-performing organizations. Imagine being able to store all artifacts involved in a project–Including requirements docs, marketing campaigns, and spreadsheets–In a single location accessible via tools that fit your needs. Collaboration, passing audits, applying for tax credits, and disaster recovery would be so much easier.

We’re not there yet. As mentioned, tools such as code review, Google Docs, wikis, version control systems, and email all help with collaboration but they are often still silos of information. Just as breaking down silos between teams has been revolutionary, breaking down the walls of data will one day have an equally profound impact.

About the Author

Before joining Perforce as a Technical Marketing Manager focused on Continuous Delivery and DevOps, Jonathan Thorpe held technical marketing roles at CFEngine, Serena, and Electric Cloud, where he specialized in DevOps culture and automation technologies. In his spare time you can find him reading, playing with all of the current video game consoles, dabbling with mobile software development, and playing with the Raspberry Pi.
November 13, 2014