devopsy.kinsta.cloud

Author: contributor

  • Tailoring Enterprise Collaboration: Strategies for Success

    Tailoring Enterprise Collaboration: Strategies for Success

    In the age of SCRUM, Agile and DevOps, developers collaborate with their peers and colleagues more than ever before. Many developers already use a variety of collaboration tools on a daily basis, such as a team messaging application, an online discussion forum, or an open code repository.

    But while developers love to collaborate, it can be easy for them to forget that not everyone collaborates the same way. By focusing solely on collaboration products and solutions that work well for their development teams, developers are missing the opportunity to provide tailor-made collaboration capabilities to the applications they’re building, or to their entire company or organization. When it comes to developing collaboration capabilities for their organization, developers often fall short of business expectations, as their own personal needs differ from their constituents.

    Tailor-Made Collaboration

    Here are three recommendations on how to tailor a collaboration solution that will fit well for an entire organization:

    Beware of the “One Size Fits All” Approach

    Today’s array of collaboration tools is more sophisticated than many developers realize. In fact, incorrect assumptions about which platforms are “best of breed” result in rolling out collaboration technology that may have technical but not business, appeal. As many developers hold the keys to the platforms that are eventually put into practice within their organization, this leaves executives at the hands of those who often see collaboration platforms as group chat spaces and wikis for sharing code.

    However, other lines of business don’t only collaborate with chats or wikis. A business with thousands of employees located around the world cannot only use general correspondence streams to acquire the information they need to successfully do their jobs. It is unrealistic to expect that messaging platforms alone can support groups of people across departments who do not necessarily understand one another. For instance, IT and Marketing departments’ daily operations are far from the same—so then why would a “one size fits all” approach for workplace teamwork be successful?

    Developers must look beyond topline technologies and examine a full range of collaboration solutions that can also provide centralized repositories for information, customized landing pages and unified communication capabilities.

    Identify and Understand Communication Gaps in Business Processes

    As departments differ, mindsets across organizations vary as well. The developer creating the back end of the platform operates in a unique workflow and framework from those utilizing the platform. It is, therefore, critical to demonstrate a clear integration between different levels of the organization to create a unified understanding of how people work.

    Individuals who rely on each other to accomplish tasks need to interact in a variety of ways, including through a centrally located platform to converse on ideas and join forces on project development. To create useful online spaces, developers should attend meetings with other teams to truly understand their needs. By doing so, they will gain perspective on how their peers operate in the workplace, and develop a platform that is increasingly applicable and effective.

    Build a Plan for Scale and Growth

    By better understanding business needs, developers can predict the appropriate scaling requirements for their collaboration solution. Scaling a collaboration solution may involve setting up workspaces and areas for mass access to thousands or tens of thousands of people, but many of the leading tools cannot scale past several thousand users. For global businesses with multiple constituents and varying workflows, messaging-based approaches may not suffice on their own, as they are not scalable methods to distribute content to a high volume of people.

    Ultimately, enterprise collaboration work patterns are shifting toward more speed, and developers are at the heart of the transition. However, being overly reliant on the technical appeal of specific collaboration tools will not lead to the best outcomes for the business.  Developers need to design and build collaboration solutions that not only assist their colleagues to communicate, but also help drive collaboration that impacts business outcomes at scale. By understanding the nature of the way an organization needs to collaborate, developers can provide solutions that directly contribute to increased revenue, empowering them as strategic partners in organizational change.

    About the Author / Stephen Hamrick

    Stephen Hamrick is VP Product Management, Collaboration Software at SAP. Based in Palo Alto, California, Steve leads the product management, user experience, and documentation teams for enterprise social software products at SAP. He has spent the last 20 years of his career in the enterprise software technology industry, focused primarily on collaboration and social software. Connect with him on LinkedIn and Twitter.

  • Spurring the DevOps Adoption Journey

    Spurring the DevOps Adoption Journey

    Every few years, the IT canvas is splashed with a new initiative that sparks enthusiasm among businesses and technologists alike. DevOps is one such philosophy gaining significant momentum in today’s digital transformation landscape. At the same time, the concept remains misunderstood and adoption is slow.

    DevOps embodies Development plus Operations within software development. Traditionally, these two parts of an organization have operated siloed from one another with marked differences in both ethos and approach. The result: delays in launching products and releases to the market. The fundamental principles of DevOps are to reduce barriers between Development and Operations; increase Development autonomy; and to automate Development and Operations activities to reduce wait time (and waste from a customer’s perspective), thus enabling faster time to market—all while keeping the end customer at the center.

    Planning the Adoption Journey

    Here are four different aspects an organization should consider before starting the DevOps adoption journey:

    Conduct an as-is continuous delivery assessment on product development practices and processes within the organization

    This will best determine an organization’s current maturity level across several different parameters, including:

    • Culture and organization
    • Build management and continuous integration
    • Environments and deployments
    • Release management
    • Testing and verification
    • Data management
    • Design and architecture
    • Information and reporting

    This process also includes identifying current automation levels and opportunities to automate, inclusiveness of tools and environments supporting continuous delivery, and an assessment of existing product design to determine if it lends itself to frequent and fast deployments.

    Map the journey of a piece of code from a developer’s desk all the way to when the consumers use it

    Utilizing a development pipeline assessment to study the life cycle of a piece of code through its journey from a developer’s desk to production will best identify “waste” during the life cycle. This assessment should bring out critical information, including the number of environment hops, test pyramid maturity and level of continuous testing, level of provisioning and deployment autonomy and its automation.

    Establish a culture of inclusiveness, autonomy and a sense of a ‘common goal’ within the organization

    Product management, development and operations teams should have a common mission and goal for the product, sprint or release they are working on. The definition here should be the value delivered to the customer as opposed to the value of code, test results, or individual deployment steps. Including operations roles within a close-working scrum team benefits this cause by:

    • Bringing in common views on what a product, release or sprint is intended to deliver.
    • Creating an opportunity to bring forward and address operations needs during the development cycle.
    • Creating a better understanding of mutual expectations and empathy for delivering outcomes that meet customer expectations.

    Measure and track

    Finally, determine parameters the organization would like to establish to track progress towards a successful adoption. The key here is to have a few, simple metrics that truly reflect “value to the end customer.” Here are a few commonly considered metrics:

    • Production deployment frequency: The number of product releases within a specified time.
    • Average lead time for a production change: The typical amount of time it takes for a piece of code to traverse from a developer’s desk through production.
    • Average production recover time: The time taken to bring back production systems after an incident involving a release, release rollback or system outage.
    • Change failure rate in production: The number of releases that did not stick in the live environment and needed either a rollback or an immediate emergency fix.

    The goal and a true measure of successful adoption would be an improvement of all these parameters over time. Internally, they would require strong engineering rigor and cultural realignment to be able to improve these customer-facing metrics.

    Navigating DevOps adoption successfully is neither easy nor fast for a number of reasons. DevOps encompasses different parts of the organization, which historically may have operated in silos and with different ethos. DevOps also questions the status quo regarding engineering roles and practices, and focuses on automation and reskilling, all of which need both upfront time and effort investment. Finally, DevOps believes in and propagates continuous learning and improvement, whereas many organizations may have more stepwise-based programs in place.

    For a midsized IT organization, it could take anywhere from 18 to 24 months or more to be able to reach a reasonable steady state. That’s why it is important to carefully craft a charter that encompasses the above parameters before embarking on this DevOps adoption journey.

    About the Author / Amit Gupta

    Amit Gupta is leader of Ness Digital Engineering’s DevOps practice. He has several years of software delivery background. He has keen interest and has developed deep expertise in establishing practical software development processes especially for globally distributed teams. Amit currently leads Ness Digital Engineering’s DevOps practice, and engages with several customers on such areas as improving engineering effectiveness, engineering tools and process transformation. Connect with him on LinkedIn.

  • DevOps 2.0 for Digital Transformation

    DevOps 2.0 for Digital Transformation

    We already know that DevOps is a culture, a movement, a philosophy of software development and delivery. It focuses on the communication, collaboration and continuous integration of the entire organization—a new conception of software development whose adoption is not easy. It aims to streamline software delivery by removing barriers between developers and IT operations teams. It also aims to reduce the inefficiencies of processes, adopting new tools, and even proposing changes in business architecture.

    Agile practices allow development teams to deliver iteratively more frequently, among other things. In DevOps, the entire process cycle is automated to achieve continuous integration and continuous deployments (CI/CD). The test and safety teams are involved from start to finish following an end-to-end strategy. At the same time, with the principles of Lean, we have removed from the whole process what does not add value. Thus, it is getting more productivity, maximum value and improvement in the time to market.

    Talking about DevOps is thinking about techniques, patterns and good practices. It will help us to provide our organization with an efficient software architecture—an architecture based on microservices, with everything-as-a-service mentality (XaaS). We are talking about container, automation, elasticity, self-scaling, infrastructure such as code, monitoring, configuration management, self-sustaining and adaptable systems and ecosystem, all of which allow us to deliver value with fast, reliable and frequent developments, not to mention zero downtime and, of course, the ability to undo the changes.

    How Do We Prepare to Meet User Expectations?

    DevOps aims to help an organization to produce software products and services quickly. But end users are becoming more and more demanding. We must use techniques that allow us to make faster, functional deliveries to the production environment, mitigating the risk involved.

    At present, we face users with greater technological knowledge and greater expectations in the dynamic and interactive experience than we used to some years ago. This has evolved and given rise new one called DevOps 2.0 or BizDevOps (business, development and operations).

    DevOps 1.0, as the origin of the movement, was focused on operations teams giving continuous feedback to development teams as well as to SQA teams. However, DevOps 2.0 is now focused on extending the benefits of feedback to the entire organization (marketing, sales, product, etc.). This is what will allow a greater adaptation to the IT boom and the digital transformation (IoT, intelligent platforms, big data, etc.). Additional benefit of DevOps 2.0 is faster continuous software deliveries, without compromising the stability of systems in production.

    Toggling Features or Feature Flags

    At this point, the key is to use techniques that helps to decouple the software delivery of new functionalities. In other words, it is about making these new functionalities available to the end user based on business rules instead of the operation teams. The idea would be that, for instance, marketing and sales department can decide, using “flags,” when to release or enable a new functionality to the end user. It will allow us to deploy new functionalities to the production environment more frequently without enabling them to the end user. This removes the “fear” generated by the daily deployment processes in production environments. This is what is known, in DevOps 2.0, as the rollout.

    Using this simple best practice for development, called Flag Driven Development, we do not make available the new functionality to 100 percent of users. It will be delivery in stages, starting with 1 percent of users, then with 10 percent, then with 30 percent, etc. with the ability to quickly enable or disable the functionality when something does not work as expected. In addition, it allows us to make available different versions of a functionality for different types of users (customer segmentation). There are several SaaS models and open-source solutions we can use to implement this type of techniques like Feature flags, toggles or controls.

    What are the Benefits of DevOps 2.0 Techniques?

    The use of this type of techniques during the development process provides these benefits:

    • Quick scaling
    • Stability of the system
    • Integrity of the applications is not compromised.

    This way we succeed in reducing the risk associated to each new delivery of functionality, in any productive environment. Business stakeholders can control when new features are available to end users, who continuously demand new changes and improvements.

    For organizations, DevOps 2.0 will bring DevOps power to non-technical teams such as marketing, design, sales and business. Now all teams will have the skills and responsibilities that are required in a DevOps culture. It enhances the confidence, communication and feedback that is needed from all teams in a DevOps strategy.

    Therefore, one of the major pillars of the DevOps 2.0 approach is the ability to control, through a control panel interface, the launch of new features of applications in production environments. This process would be launched in a controlled way by both technical and non-technical people. Also, the process will be separate from the development and continuous deployment.

    At the same time, this type of coding techniques will allow to perform real-time analytics, making changes to the functionalities of a system that may also impact in the application performance monitoring (APM) tools. This makes it possible to determine when a particular functionality is degrading the global system performance and needs to be disabled quickly, as well as identify which functionalities are improving the end user experience (UI/UX).

    There are a lot of articles about these “Fast Feedback” practices. I hope this article help you better understand the potential impact of DevOps on your organization.

    About the Author / Jesús Pérez Franco

    Jesús Pérez Franco is an Operations Manager at Gfi Spain. His big passions are Agile Development and DevOps Continuos Delivery (Continuous Integration, Deployment, QA Automation and Infrastructure as Code, etc.). Connect with him on Twitter and LinkedIn.

  • 5 Ways to Keep DBAs Happy, Alive and Efficient

    5 Ways to Keep DBAs Happy, Alive and Efficient

    As DevOps adoption continues to grow, organizations are competing to find the best and brightest application developers available. Meanwhile, database administrators (DBAs) are hard at work in the background to keep up with the burgeoning number of databases platforms developers deploy.

    DBAs have it rough. If developers are the James Bonds of the IT world, then DBAs are the Qs who quietly toil away to create the next big toy that Bond will destroy.

    The average DBA in a large enterprise can be tasked to manage 40 or more databases spread across various development stages. These disparate and sometimes rogue databases compete for the same compute and storage resources. Nearly all come with their own reporting and management tools. And when they break or fail, any number or combination of factors can be blamed.

    Database growth is especially difficult to control when companies such Amazon Web Services make it easy for any developer to create one. The AWS site, for example, provides just five easy steps to create a MongoDB database. Compare that to the time and work that goes into hiring, training and retaining more DBAs.

    Overloaded DBAs can overlook issues that lead to data loss or missed insights that affect revenue and productivity. If you can’t establish policies that control database growth, then think about ways to help your DBAs keep up and stay ahead of the game. Here are five possible tips:

    Focus and eliminate distractions

    DBAs can sometimes be lumped into general, overarching IT roles including help desk, backup and recovery or system upgrades. You probably wouldn’t want your developer to multitask, so why ask the DBA on the back end to do that? Give DBAs the opportunity to move beyond the firefighting role to focus more on proactively improving application performance. In the long run, this approach will eliminate fires before they occur.

    Give DBAs time to train on new databases

    By eliminating outside distractions, DBAs have more time to spread their knowledge across traditional databases including MySQL, Oracle or Microsoft SQL, and develop their expertise on more modern open-source databases such as Couchbase, Cassandra or PostgreSQL.

    Centralize monitoring tools

    The management tools that come with most databases provide a limited view of their broader environment. Centralized tools can provide visibility across the entire application stack. They show how databases interact with shared resources and unveil interdependent factors that lead to failures and performance issues. A centralized tool allows DBAs that “Monday morning coffee” view of all databases on one screen to determine which one needs the greatest or more immediate attention.

    Deputize your DBAs to enforce database policies

    Give your DBAs the power to evaluate and regulate new databases. This role is difficult to maintain if you confine DBAs to a traditional operations team. If databases can cross boundaries, so should your DBA. They should be empowered to seek out and evaluate databases that might be redundant, resource-intensive or unused. This means giving them the ability to say no to new databases unless they have a clear and compelling argument to be created. DBAs also deserve the authority to create a common set of goals, metrics and SLAs that must be met by all databases.

    Hire and treat your DBAs like rock stars

    It is becoming increasing more challenging to find and hire talented DBAs—especially the ones who can multitask across multiple database platforms. According to a survey in 2015, “The Vanishing Database Administrator,” more than 41 percent of database professionals surveyed said they intended to leave the data management field within the next 10 years. Possibly because of the impending shortage, the DBA profession is now seen as one of the hottest IT careers over the next several years. CNN Money ranks it as No. 9 in its 2017 Best Jobs in America. Employers should give particular consideration to these candidates and do everything they can to retain them.

    If you want your data to continue playing a central role in your business, give your DBAs as much recognition and credit as your developers. Behind every  developer, there’s a DBA keeping applications and data running at top speed. Remember Bond could never be successful with his Q.

    About the Author / Cameron Jones

    Cameron Jones is the Principal Product Manager at Blue Medora. Based in Grand Rapids, MI, he has been with Blue Medora for over 5 years working with partners and customers to ensure the highest value of Blue Medora products. This has involved initial investigation, feature planning and ensuring customer satisfaction. Connect with him on LinkedIn and Twitter.

  • DevOps Needs a New Cloud Solution for Best Practices

    DevOps Needs a New Cloud Solution for Best Practices

    Imagine this scenario: You are a company building modern software which is a key driver of your business. To support this, you have grown your infrastructure organically over time by adding servers, storage, networks, security, virtualization software, monitoring, alerting and analytics tools. You have also added a plethora of other middleware, databases, messaging and operating systems that your developers need to build, test and deliver their applications.

    Because of this unplanned organic growth, there is no consistency in the environment. You manage, maintain and operate much of this environment manually. Your various specialist teams (storage, networking, security) begin to form fiefdoms around their respective functional areas. Change becomes extremely difficult and automating across silos turns into a maintenance nightmare. You over-purchase to meet the demand and risk under-utilizing resources. Your costs spiral out of control. You are now finding it a hopeless challenge to meet the needs of your developers—which was the primary purpose of doing all of this in the first place.

    Moving to a Modern Platform

    A modern DevOps platform should address the requirements of both your development and IT/operations teams. It should give software development teams an easy way to consume on-demand compute, storage and networking as well as provide easy access to continuous integration and continuous deployment (CI/CD) tooling and software services that help increase their development throughput and shorten product delivery. In addition, the platform should also empower operation teams to manage, maintain and operate the entire infrastructure environment with very few people using smart software that drives automation and intelligence into the entire stack.

    In short, it should have the ability to break the silos within your IT environment and between development, QA and operations teams. By doing this, you can increase the velocity of software development, improve infrastructure utilization, increase overall operational efficiency and reduce costs.

    Requirements for a Cloud Infrastructure That Streamlines DevOps

    There are several requirements for a modern, cloud-based DevOps platform. Let’s look at the key essentials of such a solution.

    Self-service API-driven infrastructure is a fundamental requirement to enable infrastructure as code, a key attribute to enable DevOps best practice. It allows developers to write code (which can be done using their favorite programming language) and make RESTful API calls to the underlying programmable infrastructure to manage initial deployments and configurations. It also enables them to manage ongoing automated dynamic provisioning of infrastructure, auto-scaling, monitoring and alerting.

    All this automation removes the confusion and error-prone manual steps for the entire application delivery process, including development, testing, staging and production deployments. This, in turn, accelerates software delivery and increases quality.

    However, building such a self-service API-driven infrastructure by yourself is not easy. The key component that drives up the cost is the operational complexity and the cost of experts needed to build and operate it.

    Smart software that drives intelligence into the stack. Very few solutions in the market automatically handle the full life cycle (deployment, upgrades, monitoring, alerting and ongoing resource management) of infrastructure and also allow seamless migration of workloads between private environments and public clouds. Additionally, if self-service is available as a SaaS-based delivery model, it is easy to add more features and workflows very quickly without having IT perform a major upgrade. If your solution comes with an on-premises install, ask how much work IT has to do to maintain this environment and how often new features will get added.

    Hyper-converged Infrastructure (HCI): Eliminating silos in the infrastructure. As mentioned earlier, silos impede velocity because they lead to complexity of operations, lack of consistency in the environment and lack of automation. A hyper-converged cloud design with a software-centric architecture tightly integrates compute, storage, networking and virtualization resources and other technologies from scratch in a commodity hardware box supported by a single vendor. This approach eliminates silos and lowers costs and complexity. It also makes it easy to start small and grow on demand while staying tightly right-sized on capacity and cost.

    A built-in browse-able categorized application store. DevOps engineers need readily accessible CI/CD tools such as Jenkins, Git, Maven or JUnit to automate the development and test pipelines. Additionally, tools such as Ansible, Puppet and Chef help with automated configuration and lifecycle management of workloads. Application developers need to quickly integrate and deploy middleware services such as RabbitMQ, Redis or storage back-ends including MySQL, Postgres, Cassandra or MongoDB. Single-click deployment of such services and other complex multitier network tools and applications greatly automates, accelerates and simplifies the development process.

    Seamless migration between private and public clouds. Developers and testers often need much greater capacity than is available on-premises for scale testing, and they should be able to move their application to a public cloud for such purposes. Alternatively, they may already have some applications running on a public cloud and they may want to bring them back to their private cloud for cost savings or better performance. For these use cases, a private cloud solution should offer seamless bi-directional migration between the public cloud and the private cloud.

    Dashboards and visibility for troubleshooting. One of the key advantages for a private cloud is that you can get complete visibility across the infrastructure and applications. In a public cloud, you can only get VM-level stats but have no control or visibility below that. Make sure that your private cloud solution can provide visibility to developers directly. Many vendors only provide access to IT, thereby controlling what developers can do and making IT a bottleneck in the application development and deployment process.

    Better insights to improve efficiency and capacity management. A cloud requires capacity planning, utilization monitoring, right-sizing of workloads and detecting zombie VMs and unused resources. Look for a solution that comes with built-in analytics, management tools and insights. An even better approach would be having a solution where you don’t have to install the management tool on-premises so that it can deliver its value as a service.

    To meet the needs of application developers and development frameworks and to create an efficient DevOps platform, companies should look at consistent converged architectures managed via smart software. This empowers application developers and operations teams to leverage a consumption model they want, while providing control to IT and making them relevant to your business success.

    Integrated intelligence and insights delivered as a service ensure that you never have to use spreadsheets to decide what to purchase and when. Cloud solutions with smart software ensure that you never have to make a Level 1 support call again; your management software can do that for you.

    About the Author / Kamesh Pemmaraju

    Kamesh Pemmaraju leads product management, product marketing, and technology partnerships at ZeroStack. Kamesh is a seasoned product management/marketing leader with a strong engineering and consulting background. In his most recent product roles at Mirantis and Dell, he focused on delivering private and hybrid cloud solutions to enterprises. He holds a Master of Science in Computer Science from Indian Institute of Science. Connect with him on LinkedIn and Twitter.

     

  • Improve IT Security by Improving Company Culture

    Improve IT Security by Improving Company Culture

    All around the world, cybercrime rates are rising, and it’s clear that traditional criminals are constantly developing new ways to attack. These days, wreaking havoc requires nothing more than internet connectivity and a pronounced lack of scruples. With a lowering barrier for entry causing an increase in the pool of malicious actors, many enterprises are struggling to protect themselves.

    Buzzwords such as “cybercrime” and “black hat” are now commonplace, and global spending to combat cybercrime reached $80 billion in 2016. Despite these efforts, only 38 percent of organizations surveyed for ISACA’s “2015 Global Cybersecurity Status Report” were confident in their ability to defend against cybercrime attacks.

    Why Does Security Still Suck?

    With high levels of cybercrime awareness and direct steps being taken to address these challenges, it begs the question: Why does IT security still suck?

    One of the primary reasons many enterprises struggle with cybersecurity is a lack of cultural support in the workplace. C-suite policies often hinder success more than they help by pushing employees to the brink and creating an environment that leaves their businesses susceptible to human error and attacks.

    While Brooks’ Law can serve well as a guide, it’s often a harbinger of bigger problems in many enterprises: When a software project is running late, leadership often brings another employee onto the project and, as a result, slows things down even more. Between the security system add-ons and services and the human capital involved in taking time to understand the issues as they arise, productivity takes a major blow—without any measurable benefit to security. This traditional tactic has attempted to ignore the crucial fact that 77 percent of professionals said that their information security policies and teams are slowing IT down.

    Compounding these issues is the fact that traditional security tools require an immense amount of time and manual labor. Because these legacy platforms do not have the ability to automate even basic functions, IT professionals are often required to monitor security tasks and patch vulnerabilities around the clock. This is a major responsibility, as the costs of an attack or other breach can be devastating to an enterprise.

    Further, IT security professionals are rarely rewarded for successful efforts. This is challenging for two reasons. First, no one is congratulated for creating a security-sound app, as the consumer expects nothing less. Second, it can be difficult to convince corporate leadership of the need for improved, automated security tools before an attack occurs. Painting a realistic depiction of a hypothetical situation (in this case, the damage caused by a cyberattack) is hard to do, and professionals often struggle to demonstrate their value without resorting to scare tactics. Scare tactics, independent of statistical supportability, have a shelf life and are subject to diminishing returns. Corporate leadership teams don’t regularly identify or prioritize security problems until it’s too late, so they don’t consider the cost to their employees who are fighting these battles on a daily basis. Together, all of these factors add up to fatigue, frustration and burnout for IT security professionals.

    It’s safe to assume that we all want to do our best work. When we don’t have the tools and support necessary to do quality work, attitudes and culture suffer.

    To stay ahead of security concerns, C-suite policies must not tackle IT issues or adjust to new regulations as they arise, but ensure that efficient, effective security practices are baked into agile DevOps processes rather than bolted on. It is essential for leadership to be proactive, rather than reactive—once an enterprise falls behind the curve, its software applications and valuable data are at increased risk until vulnerabilities can be patched.

    Perhaps not surprisingly, this proactive attitude does not always come naturally. Management often wants to do things the way they’ve always been done, but they need to become more agile as well. Security within an organization will benefit from pushing awareness across the enterprise, rather than keeping security professionals siloed within their own department. There’s a fine line between educating employees and using scare tactics, but leadership must be informed of vulnerabilities and necessary steps for shoring up weaknesses.

    However, education can only go so far; automation is the real key to taking the risk out of the hands of security professionals. Modern tools will allow your team to rest easy at night while proactively focusing on your enterprise’s rapidly evolving applications. This makes your employees happier, your security stronger and your business better.

    About the Author / Ash Wilson

    Ash Wilson is a strategic engineering specialist at CloudPassage. He has been a paid tech worker since March 2000, and a hobbyist long before that. He came to security via network engineering and systems administration. Ash spent the last five years in post-sales engineering and strategic engineering for security product companies. Connect with him on LinkedIn and Twitter.

  • DevOps Doesn’t Stop at the Database

    DevOps Doesn’t Stop at the Database

    DevOps has moved into the mainstream and practices such as continuous delivery now are being widely adopted. But what about the database? Applications often have a back-end database that needs to be developed and maintained at the same time.

    At first glance, there doesn’t seem to be an issue. The recent “State of Database DevOps Survey” from Redgate showed that 75 percent of companies have developers in their team who work across both applications and databases.

    Investigate a little deeper, however, and the cracks start to appear. The survey also showed that while version control is used by 81 percent of those in application development, it falls to 58 percent for the database. The same is true for continuous integration and automated deployments—39 percent and 33 percent for the application, 20 percent and 21 percent for the database.

    What’s going on? If developers are increasingly responsible for the database as well the application, why can’t the principles of DevOps be applied across both areas? That way, rather than the database being the problem at the end of the pipeline that can (and many times does) cause releases to fail, it becomes part of the pipeline.

    Especially when you consider that the biggest downside of traditional siloed database development is the downtime often experienced when introducing changes. That’s not to mention slowing down releases and preventing companies responding rapidly to changing business requirements.

    A typical DevOps workflow looks something like this:

    Code is version-controlled so different developers can work on multiple branches at the same time. When it’s checked into the version control system (VCS), often at the end of each day, it triggers a continuous integration process that builds and validates the changes. Once the changes are complete, a release management tool then pushes them through the testing, QA and staging environments and there’s a final review before the changes are deployed to production.

    Not every company does all of this, but many do different parts of it and plan to extend DevOps practices into other areas. They might start with version control, for example, get the team accustomed to checking in changes, and then move on to continuous integration.

    It works too. The “2016 State of DevOps Report” from Puppet found that organizations practicing DevOps deploy 200 times more frequently, recover from failures 24 times faster, and have a 3 times lower change failure rate. DevOps has moved from a developer-led approach to one that businesses are actively encouraging.

    Thing is, database development can be part of the same picture as application development. Let’s take another look at that DevOps workflow:

    The same workflow, but this time with database development included. Note that there are no extra boxes or arrows, there are no asterisks with caveats, and the only difference is that database development has been added to application development and a DBA has stepped in to the frame.

    But how can database development be added to the DevOps mix when it requires new tools, separate processes, a different mindset? The answer is to think about integration not separation, and extending the current infrastructure not replacing it.

    Application development teams, for example, will use a VCS like Team Foundation Server (TFS) or Git, a continuous integration system like TeamCity or Jenkins, and maybe a deployment automation tool like Octopus Deploy. They’ll also have a favored IDE for application development, probably Visual Studio, and if they do database development, they may well work in SQL Server Management Studio as well.

    Ask them to add to their workload with unfamiliar tools and development environments, or enforced policies and processes, and the shine of DevOps starts to lose its luster. What was freeing them to perform better is now tying them down to an unwelcome way of working.

    Ask them, on the other hand, to use the same tools and processes but extend their capability to database development and it’s a different story. If they already version application code using TFS, for example, there are database version control tools out there that plug into it so that database development becomes an additional part of the process, not an extra process.

    The same is true for continuous integration, release management and deployment. At every step of the DevOps workflow, database development can join application development so that it’s a natural partner rather than an unwelcome guest.

    But just how realistic is this approach? When development teams are already facing the challenge of introducing DevOps, is it a pipe dream to think the database will even be considered as part of the picture?

    According to Microsoft, no. When Visual Studio 2017 was launched in March, the Enterprise edition included Redgate Data Tools in the installer to make it easier for users to implement DevOps practices when building, deploying and maintaining their SQL Server databases.

    As Shawn Nandi, senior director for Cloud App Development and Data product marketing at Microsoft said at the time: “We think this is an important step forward to help companies extend DevOps practices to all parts of their applications.”

    So if you’re considering adopting DevOps, remember the database can be part of it, too.

    About the Author/Matt Hilbert

    Matt Hilbert is a technology writer at Redgate Software with 20 years’ experience working for lots of the world’s biggest tech companies – and many of the smallest. He has a particular fascination for emerging technologies and is on a continuing mission to decompile, untangle, and explain techspeak to a wider audience, and excite people about the endless possibilities technology offers.

     

  • Database Bottlenecks: The Hidden Cause of App Slow Downs?

    Database Bottlenecks: The Hidden Cause of App Slow Downs?

    Performance problems can cripple mission-critical business apps. If you are lucky enough, your solution could be as easy as adding more WAN bandwidth, a few extra servers or a content delivery network, but the real problem arises when your database becomes your biggest bottleneck. When performance issues adversely impact the applications that keep your business operations running and revenue pumping in, the challenge is more difficult to overcome.

    It can be painstaking, tedious work to examine the architecture for flaws, dive into code for clues on sluggish performance and scour log files looking for correlation events. Among all the areas to examine, the following three problem spots should be high on the list of troubleshooting priorities to remove performance issues:

    1. Connection Pooling Problems
    2. Below Par Query Performance
    3. Insufficient Concurrent Capacity for Handling Heavy Traffic

    When database administrators (DBAs) fail to find and fix bottlenecks in time, businesses lose eyeballs and revenue—sometimes losing customers for life. Every transactional app accesses a relational database, and their biggest problem is the challenge of scaling horizontally. While modern databases support horizontal scale-out, today’s apps aren’t written to leverage this additional capacity.

    An intermediate software layer, database load balancing software, solves this challenge; it makes the apps able to use additional servers without any code changes. It often includes caching as well, which can substantially increase performance—again, with no code changes. Database load balancing not only makes it easy to scale horizontally but also helps you identify other bottlenecks and mitigate them in a matter of minutes.

    Database Load Balancing: The Silver Bullet?

    Database load balancing software comes with analytics tools that precisely pinpoint database issues in real time. It monitors everything from every read and write query to connections, server performance and overall database load. Detailed analytics offer insights into what you need to fix and where to focus your attention to ensure better performance. A load balancer has all the necessary tools needed to resolve performance problems in a timely manner. It efficiently addresses the three major performance problems previously mentioned and significantly improves app performance and availability through:

    1. Connection Pooling and Multiplexing: Database load balancing software can drastically decrease the number of connections to your servers by applying connection pooling and multiplexing. Unlike client-based connection pooling, this approach ensures greater operational efficiency and facilitates easy handling of additional connection spikes.
    1. In-Memory Query Response Caching: The analytics of the software identifies potential queries to cache and lets you cache them with no application changes. For example, you can cache an article in a content management system (CMS) for a day, having the comments section pull from the database to stay fresh. You can cache a zip code table for months. Look for the ability to set time-to-live expiration or automatic cache invalidation so you can ensure atomicity, consistency, isolation, durability (ACID) compliance and transaction safety.
    1. Dynamic Load Balancing: Database load balancing software routes queries from the app to multiple servers in a safe and consistent manner. Its automated read/write split ensures high performance by diverting all the read queries to available read replicas and writes to the master server. This load distribution increases capacity and facilitates an instant scale-out. A load balancer acts like a single point of control that offers deep visibility into performance needs. With real-time insights into the root cause of problems, the entire find and fix process becomes fast, efficient and effective.

    Saddled with the most difficult task of identifying database bottlenecks, DBAs can make their life easy and work efficient by leveraging a feature-rich database load balancer that automates the diagnosis of bottlenecks and gets the data flowing seamlessly.

    About the Author / Tony Branson

    Tony Branson is Database Load Balancing Senior Analyst at ScaleArc. Tony has a passion for dissecting tech topics such as transparent failover, centralized control, ACID compliance, database scalability and downtime effects. On his days off, he can be found watching sci-fi movies, rock climbing or volunteering. Follow him on Twitter.

  • Achieving True Automation is a Marathon, Not a Sprint

    Achieving True Automation is a Marathon, Not a Sprint

    But It Starts in the Cloud

    Arguably, the most important goals for companies seeking to get into cloud today are agility and flexibility to innovate at speed and scale. Adding cost savings to the mix makes the possibilities all the more attractive. And these three pillars can only be achieved with automation.

    But while many companies may believe they’re automating (and thus attaining these three goals), they’re really not. In the rush to cloud adoption, they fail to see that this type of investment takes time and work—a lot of work. Think of it as a marathon, not a sprint.

    While it’s a significant step toward achieving scale and speed, simply moving an application to cloud is not enough to attain true automation. Adding genuine automation to your server means enabling it to spin up or down as needed, react to scaling requests and shut down when not in use. Only when these processes are in place can you then start seeing the cost savings and agility promised by cloud.

    While there is no simple road map to achieving true, seamless automation (near-Nirvana), there are clear and well-worn steps that organizations seeking these goals can take. Looking at it from a broad perspective, automation in the cloud depends on two major initiatives.

    Long-Term Value is in App Automation

    The first step is to figure out where automation will provide the most value (from an application perspective). In most cases, at least as far as what we’re seeing across the marketplace, the answer to this will likely revolve around infrastructure.

    The second step gets a bit trickier. You essentially want to connect automation to the applications themselves (replacing the manual task of writing a script). The challenge then becomes pulling the application back into the larger architectural context so that it can respond to an event without human intervention. This phase involves making the application cloud-ready—that is, furnishing it with the ability to trigger automation without someone sitting at the controls.

    This second phase bears closer scrutiny—it’s here that the people on the ground start to lose faith, because with some applications automation may seem all but impossible. Let’s face it: A fair majority of applications were not designed with automation in mind. Legacy applications in particular (which typically constitute the bulk of most companies’ IT estate) are fairly fussy when it comes to automation. This is when organizations start debating the value of opening code and making changes—whether the investment of time and capital is really worth it.

    Which brings us to the two biggest barriers to automation—costs and culture. Yes, it can be expensive to implement the processes and skills to achieve game-changing automation in the short-term. It requires moving application development specialists to new roles, a massive shift in culture and skill sets within IT. This is particularly true for code development, which has been manually focused for so long. Many companies fail because they don’t see automation as an ongoing true software development program (which, of course, it is).

    Attaining DevOps

    The Holy Grail of automation is achieving a seamless Development-Operations environment (DevOps), one in which software developers and the IT folks are continuously collaborating and communicating to automate software delivery and infrastructure changes. One of the chief goals of DevOps is to create a culture and environment where building, testing and releasing software happen frequently, rapidly and reliably.

    Achieving a true DevOps environment means that you are a functioning automated shop. Automation does not end with the first deployment. In fact, it is only the beginning. You will continually face new upgrades and changes in service, requiring near-constant tweaking. But automation smoothes the process, loosening the friction of life cycle management. Automation becomes embedded into the very fabric of your entire IT estate.

    For some companies (mostly startups), automation is not a luxury but a necessity. Look at Netflix (which, in some respects is less of a startup and more of an established player). The only way the company could scale and stabilize to meet customer demand with the success it has over the past few years was to automate. It would be physically impossible for Netflix to serve its customers any other way.

    Same with Amazon. Jeff Bezos figured out early on the value of automation in the cloud and built the flexibility into Amazon’s infrastructure to grow and scale according to exponential customer expansion.

    We’re seeing this happen increasingly across industries—from the financial sector to the healthcare industry.

    One large global financial institution is changing its very culture by putting automation at the very core of everything it does, from loan applications to payroll. It has taken the bold step of only adopting applications that can be automated. The simple rule of thumb here is, “If we can’t automate it, we’re not going to do it.” This has resulted in the financial institution being able to provide an improved customer experience, providing features for customization of services via their mobile banking app.

    Another is Avalon Healthcare Solutions, a laboratory benefit management company. The organization wanted to accelerate service delivery with a 100 percent cloud-based claims editor. So it developed a portal for providers, patients and health plans to offer appropriate access to data, claims and prior authorization. Avalon can now edit from 28,000 to 38,500 per hour in a single environment with 99.99 percent uptime and 99.95 percent accuracy,

    The lessons here are clear. By pacing their race with an eye for the long-term goal, Avalon, Amazon and Netflix—among others—trailblazers across industries, are reaping the benefits cloud-automation offers. Namely, they are scaling, becoming more agile, innovating and delighting their customers all at the same time.

    About the Author / Chris Wegmann

    Chris Wegmann is Managing director for Amazon Web Services Practice & Relationship lead, responsible for helping Accenture cross-industry enterprise clients leverage the power of Amazon Web Services including through the Accenture Cloud Platform. He has 20 years of experience in technology consulting and specialization in technical architecture, infrastructure design and engineering, legacy system migrations, cloud application architectures and systems operation. Connect with him on LinkedIn and Twitter.

  • Is DevOps Making Life Better for Data Scientists?

    Is DevOps Making Life Better for Data Scientists?

    The path to success and happiness of the data science team working with big data project is not always clear from the beginning. It depends on the maturity of the underlying platform, their cross skills and DevOps processes around their day-to-day operations.

    The past few years have been like a dream come true for those who work in analytics and big data. There is a new career path for platform engineers to learn Hadoop, Scala and Spark, while Java and Python programmers have a chance to move to the big data world. There they find higher salaries and new challenges and get to scale up to distributed systems.

    But recently, I have started hearing some complaints and dashed hopes from engineers who have spent time working there.

    Their expectations are not always met. Big data engineers spend most of their time in extract, transform and load (ETL), preparing data sets for analysts and coding models developed by data scientists into scripts. They have become cogs between data scientists and big data.

    I see the same feedback from data scientists. Real big data projects differ from Kaggle competitions. Even minor tasks require Python and Scala skills as well as multiple interactions and design sessions with big data engineers and operation teams.

    DevOps engineers get to go no further than just using the basic tools and running through tutorials. They grow weary of being given nothing more to do than deploying the next Hadoop cluster through the Cloudera wizard and semi-automated Ansible scripts.

    In this environment, the big data engineer will never get to work on scalability and storage optimization. Data scientists will never invent a new algorithm. And the DevOps engineer will never put the entire platform on autopilot. The risk is the team’s ideas might not make it beyond proof of concept.

    How do we change this situation? There are three key points:

    1. Tools evolution—the Apache Spark/Hadoop ecosystem is great. But it is not stable and user-friendly enough to just run and forget. Engineers and data scientists should contribute to existing open-source projects and create new tools to fill the gaps in day-to-day operations.
    2. Education and cross skills—when data scientists write code they need to think not just about abstractions but also consider the practical issues of what is possible and what is reasonable. For example, they need to think how long their query will run and whether the data they extract will fit into the storage mechanism they are using.
    3. Improve the process—DevOps might be a solution. Here DevOps does not just mean writing Ansible scripts and installing Jenkins—we need DevOps working in optimal fashion to reduce hand-off and invent new tools to give everyone self-service to make them as productive as possible.

    The Continuous Analytics Environment

    With continuous analytics and self-service, data scientists own the data project from the original idea all the way to production. The more autonomous they are, the more time they can dedicate to producing actual insights.

    Data scientists start from the original business idea and work through data exploration and data preparation. Then they move to model development. Next comes deployment and validating the environment. Finally, there is the push to production. Given the proper tools, they could run this complete iteration multiple times a day without relying on the big data engineer.

    Big Data engineers, in turn, work on scalability and storage optimization, developing and contributing to tools such as Spark, enabling streaming architectures and so on. They provide an API and a DSL for the data scientist.

    Product engineers should be given analytical models developed by data scientists as neatly bundled services. Then they can build smart applications for business users to allow them to use those in self-service.

    Working like this, no one is stuck waiting on the next person. Each person is working with their own abstractions.

    This might sound like a utopia. But I believe this could motivate big data teams to improve their existing big data platforms and processes, with the added benefit of making everyone happy.

    About the Author / Stepan Pushkarev

    Stepan Pushkarev is the Head of DevOps Practice at Squadex.com and CTO of Hydrosphere.io. Co-founded and managed engineering teams for eCommerce, IoT and Ad-tech companies. He has been responsible for the full products stack: math models, infrastructure & operations, enterprise applications as well as hiring, establishing engineering culture and delivery process. Stepan combines strong technology, management skills and entrepreneurial spirit. Connect with him on LinkedIn.