devopsy.kinsta.cloud

Category: DevOps Practice

  • Transition to DevOps Increasing Across IT Firms

    Transition to DevOps Increasing Across IT Firms

    Demand for faster product development and quick market release created a need for increased collaboration between Development (Dev) and Operations (Ops) teams, which eventually led to “DevOps.” With this, transition to DevOps has become one of the key points of discussion across many IT firms. This article gives you an overview of the demand and trend for organizations to transition to DevOps.

    Rise in IT innovation and burgeoning digital revolution have led to the demand for faster development and quicker releases to stand out in the IT competitive scenario. This transition eventually led to a new culture that saw collaboration between two key IT teams, development and operations teams, which the industry calls DevOps.

    DevOps established a new style of IT functioning, easing different phases of software process from product development to testing and final market release, all in a rapid and reliable way.

    Why DevOps Transition?

    Rapid delivery of independent software components and continuous integration of the same with production systems forms the key principle behind DevOps functioning.

    DevOps is often confused with Agile methodology. Though they share similarities in providing improved IT performance through rapid development, DevOps stands apart from Agile because of its efficacy in the software deployment phase, ensuring a constant communication between the two teams. Thus, DevOps is often termed as a successor to Agile.

    A collaborative approach, key for information exchange across the line, is one of the major factors driving firms to undergo DevOps implementation.

    Though DevOps is well-known as a merger of development and operations teams, an effective DevOps model extends beyond that to other key functional areas of organizations.

    Breaking the old trend of limiting individual teams to their respective work domains, DevOps brought in a scope for knowledge-sharing among them, wherein developers can extend themselves to testing and deployment and operations team members can upgrade their skill set, learning new languages, architectures and tools.

    Increased Demand for DevOps Resource

    Transition to DevOps holds a key demand in the fast-growing IT market, wherein managing with the existing infrastructure or resource plays an almost equal role as coming up with something new.

    Greenfield projects aside, DevOps deployment is most often seen in brownfield projects that encourage building on the existing resources.

    With this situation in place, there is also a lot of demand for DevOps resources across firms. The “2017 Open Source Jobs Report” from the Linux Foundation shows 60 percent demand for DevOps human resources among more than 2,000 IT open source professionals and IT hiring managers.

    This clearly shows the level of demand that transition to DevOps is holding in the IT industry.

    “If there isn’t an existing DevOps role within your company, software developers should spend time sitting with the operations team or vice versa to understand how their processes work and how they can be improved through collaboration,” said Mike Durney, CEO of the job-hunting site Dice.

    The report found:

    • Nearly 50 percent of hiring managers are keen about those DevOps resources with professional credentials such as relevant certifications
    • More than 47 percent of the companies are ready to fund certification programs for their existing personnel
    • 42 percent of companies are keen on adding DevOps skills to their hiring portfolio
    • “DevOps Engineer” profile ranks second in Glassdoor’s Best 50 jobs in America

    Despite its advantages, however, DevOps has a few challenges to overcome related to application integration, test automation, budget and skill set, among others.

    — Veritis

  • A Saner Way to Build a DevOps Cloud

    A Saner Way to Build a DevOps Cloud

    There’s a popular definition of insanity: doing the same thing repeatedly but expecting different results. Yet, too many enterprises implementing digital services and apps in the cloud fail to heed this warning as they turn to point-to-point integration of their cloud components. In the process, they’re introducing tremendous complexity that looks eerily similar to the point-to-point application integration headaches companies faced before the concept of centralized mediation emerged some 15 years ago.

    Consider the scope of the challenge. A cloud deployment can contain 30-plus product components that need to be integrated to run together smoothly. At any time, an update to one of these products can break processes that rely on the other components. DevOps compounds the complexity, since teams need to ensure consistency across development, test and production environments. It may be fine for a dedicated DevOps team in the group to handle these environments when there’s just one project. However, as the number of applications utilizing cloud technology grows, it is expensive to have DevOps teams build each environment as a wild west project.

    Standardizing on Stacks

    A saner approach is to implement a cloud architecture that standardizes on stack-level operations. Implemented effectively, a stack serves as the reference architecture for all the components and automation needed to create a replica of your production environment. This “full stack” is in essence much like the configuration management database (CMDB) once thought to be the holy grail of enterprises.

    With a meta-description of your full stack for a service or application in place, it becomes possible to deploy identical copies for development, test and production, as well as automate DevOps across your application. This ability to duplicate stacks removes the barriers to, for example, deploying multiple test environments or multiple copies of the stack for security compliance or disaster recovery.

    The stack is an important concept for understanding how you are building and running your software. If something goes wrong, it also provides visibility into what is running, how it was deployed and what the problem might be, facilitating the ability to systematically correct the problem and deploy the fix to all needed replicas. Developing a stack concept also lets you track applications—plus the associated tools, development practices, cloud practices and vulnerabilities—across your organization, providing a critical step in the maturity of your enterprise’s cloud development.

    Standardizing on stacks enables development teams to focus on building new applications and services rather than spending six to 18 months implementing the cloud architecture. At the same time, different groups may require different capabilities, making a one-stack-fits all-approach impractical. The middle ground is to provide a set of “sanctioned stacks” or a portal for configuring stacks, which limits options to products that have been pretested to confirm they work together.

    Structuring DevOps Cloud Stacks

    A DevOps cloud architecture will consist of at least two main stacks: the core infrastructure stack and the DevOps stack. Increasingly, these stacks run in containers, which provide two key advantages: a greater agility through the ability to spin up apps faster and more cost-effective use of resources through operating system-level virtualization.

    The core infrastructure stack supports a service or application with services that ensure security, reliability and scalability. It includes cloud infrastructure, such as networking and storage, which may be handled by a public cloud platform or a private cloud, along with container-specific functionality, such as a Docker runtime with orchestration provided by Docker Swarm or Kubernetes. The stack also has a layer for handling performance and management, including networking, load balancing, service directory, log management, monitoring, backup and recovery and vulnerability scanning, which are supported by a range of different products.

    At the top of the core infrastructure stack is an application layer or stack that includes development tools, such as Java Stack or Node.js Stack. More advanced application stacks will go beyond tools to provide the blueprints for migrating from monolithic apps to containers and microservices.

    The DevOps stack supports continuous integration/continuous deployment (CI/CD). It runs across multiple infrastructure stacks and codifies the tools, processes and procedures to test and deploy or rollback changes to the infrastructure stacks. It typically incorporates DevOps instructions using Chef or Terraform, Ansible, Puppet, Swarm, Helm and half a dozen other tools. Because automated testing is critical, DevOps stacks also should include tools that help with development, building, integration, automated testing. Popular options include WireMock and Selenium for testing web apps and services, Artillery for performance testing and Blackduck or Clare for security testing.

    Additionally, there are key functions within the DevOps cloud platform that extend across development, test and production stacks. These include security, a catalog of stack templates, a registry for certified containers and usage data collection to apply predictive analytics for performance and cost optimization.

    Introducing Stack Automation

    Enterprises implementing a DevOps-enabled cloud need to consider automating as many processes as possible, particularly for large installations. For example, a bank may have 150 to 200 groups developing applications. With each app requiring environments for development, test and production, that means anywhere from 450 to 600 deployed stacks to maintain.

    Significantly, updating stack components needs to be a coordinated process. It’s not enough to rely on automated upgrades and security patching provided by individual product vendors. That’s because if the upgrade or patch changes something that affects a component in the enterprise’s stack, the vendor probably doesn’t know that and this may break the stack.

    Automating stack functions in a coordinated way is best accomplished using an intelligent, centralized “command center” portal that incorporates a central orchestration hub. This eliminates much of the complexity associated with manually running multiple scripts by performing whole operations on a stack. Whether your organization builds it or you use a third-party solution, this hub for automating the execution of scripts should incorporate an orchestration engine, visual map, domain model, proactive monitoring and cost analysis.

    Importantly, a command center incorporating an automation hub lets you take advantage of improved visibility provided by stacks to track and report on cloud costs. Moreover, you can capitalize on “spot” instances—costing five to 10 times less than persistent instances—by using the hub to automatically handle periodic interruptions and redeploy the components taken away.

    Through automated stacks, your organization now has a sane way to cut the complexity of implementing a DevOps cloud while providing an agile platform to support your development team’s innovations.

    — John Mathon

  • The 2 Best Ways to Automate Your TechOps

    The 2 Best Ways to Automate Your TechOps

    The DevOps community is rapidly building tools and processes to automate more of the DevOps workflow. One key is to put engineers on the front line who can find root causes of problems and automate more of the common tasks. Here are the two key areas to focus on when automating operations processes.

    Automate Your Most Expensive Tasks First

    The first step is to do some analysis on where the team spends most of their time. You will want to know if they’ve done a good job around the tickets and how many requests of what type have they gotten.

    After analysis, one company found it had more than 500 tickets a day for user access requests from users who were locked out of their account. This is a common problem, especially in midsize to large companies that are doing user management or account management and have very manual processes for onboarding and offboarding.

    There’s a ton of money that can be saved by automating these processes. And there’s a whole host of tools for everything from full orchestration systems to pipeline stripping to make it possible. But the first step is to analyze where time is being spent on the repetitive, easily duplicated tasks and automating those.

    Automating Monitoring and Self-Healing

    There are a lot of different tools out there, and the amount of data that’s coming together is enormous. With the emergence of big data and machine learning, we have the ability to put all the different points of monitoring into one system and bring all the data together to provide a single picture of how everything is performing.

    Data sources can include anything from systems monitoring tools to application performance monitoring tools to social media feeds such as Twitter and Instagram. In addition, it can and should include data from your IT service management (ITSM) tools. Once you’ve got that you can correlate your performance data with your incident data to actually predict when a problem is more likely to occur so you can fix the root cause or the underlying issue before it impacts your users.

    There’s some really neat stuff that can happen with automation, whether that’s just a simple script that can save you time and money or advanced machine learning to actually predict issues and address them in real time. Take the first step to make your organization more efficient and effective.

    — Sean Mack

  • The Role of DevOps in Building an AI-Ready Infrastructure

    The Role of DevOps in Building an AI-Ready Infrastructure

    The successful application of machine learning for artificial intelligence (AI) requires several contexts to be addressed, primarily the technical, data and business considerations.

    The Technical

    From a technical standpoint, in building an AI-ready framework there are choices to be made such as which machine learning framework to select, the type and structure of learning function and the style of training algorithm to use. These decisions are normally based on the skill, experience and judgment of the computer scientist. A practitioner should be able to coordinate these aspects and focus on business outcomes without overly burdening the line of business with technical detail. The exception to this is when core design decisions in some way will impact business outcome.

    The Data

    Equally as critical to the efficacy of machine learning is the data available for use in development and training. A deployed algorithm is a function of the data inputs it received during training and testing; these inputs should be consistent and representative of the scenarios that will be subject to inference by the algorithm. It is difficult to accommodate so-called black swan events, as typically these require a level of adaptiveness beyond present machine learning technology.

    However, we can still hope to achieve an effective and enriched algorithm that is appropriately responsive to real-world events by collecting and building data sets and training libraries that are truly representative of the environment in which the algorithm will operate. Ideally, machine learning data should include as many data outliers as may occur naturally within the real world—we should not be improving or self-selecting data, though granted most applications will use some form of normalization.

    Speaking to data scientists and machine learning specialists, it is clear as individuals they have the technical skills and access to the core technology they need to do their jobs. However, more often than not it is access to and availability of real-world, suitably random data that is one of the biggest and most time-consuming challenges. Addressing this requires good engagement with the line of business, as access to sensitive data requires negotiation. But as matter of course, any business that utilizes data analysts to help inform critical business decisions and wants to benefit from machine learning should take the necessary steps to ensure it offers accessible and open data services to their machine learning teams. Having an open, accessible and agile data architecture is an essential prerequisite of being able to deliver an AI/machine learning-ready infrastructure.

    The Business

    On the business side, rather than address the how and what questions of technology and data, we also need to ask why. Certainly, as with any IT project, business does not want to engage machine learning or AI for its own sake. It is important to understand the business goals and seek new technology that will align with those objectives. This is where the application of DevOps techniques can greatly improve business outcome.

    By aligning software development with software operations, more emphasis can be placed on what is important to the business while also helping to support the effectiveness and usability of new machine learning functions. In their deployment and use, these new applications should also exhibit many of efficiency features of modern IT technology. Thus, we expect new deployments to be able to make effective and efficient use of container technology. And where a particular function is infrequently required, how much more efficient is it to use a serverless function, perhaps with the data resident on private storage, and the function being called on demand in the cloud.

    This type of capability is now readily available to machine learning thanks to the advent of virtualized GPUs provided by a choice of cloud services. This ability to dynamically and adaptively use infrastructure aligns well with the capabilities DevOps practitioners, who for a long now time have been used to managing the deployment of continuously developed releases of code. This agility, driven by the need for adaptability and innovation is very much what is required for successful machine learning and AI. But underpinning all of this is the need to maintain the conversation with the business around the open, yet appropriate accessibility to data.

    — Matt Watts

  • 3 Ways to Minimize the Impact of High Severity Incidents

    3 Ways to Minimize the Impact of High Severity Incidents

    According to Information Technology Intelligence Consulting Research, 98 percent of organizations report that a single hour of downtime can cost upwards of $100,000. That doesn’t take into consideration the long-term effects on brand, potential loss of trust with existing customers and engineering time. This is why making sure high severity incidents are resolved quickly and safely is extremely important.

    Here are three simple yet effective ways to minimize the impact of high severity incidents within your organization:

    Establish an Incident Manager On-Call Role and Rotation

    One of the most impactful ways to help deal with high severity incidents is by establishing an incident manager on-call (IMOC) role and rotation. The IMOC is responsible for resolving high-severity incidents in a safe and fast manner. IMOCs are responsible for the management of high severity incidents through their life cycle: detection, diagnosis, mitigation, prevention and closure. As such, the IMOC role provides everyone in the company with a single point of contact. Ultimately, the IMOC role leads to a significantly reduced mean time to resolution (MTTR) for SEVs.

    Identify and Assess Your Critical Services

    Does everyone in your company know what your top five most critical services are? Take a proactive approach to improving your overall reliability by making a determination and reliability assessment of these services a top priority. It is important to understand your system and recognize how critical service failures could result in high severity incidents. Reducing downtime is certainly a goal every engineering team should have, but the fact is, things inevitably go wrong.

    Practice Chaos Engineering

    Chaos engineering is the practice of performing thoughtful, planned experiments designed to reveal the weakness in our systems. While it may seem counterintuitive, like a flu shot it helps to be proactive now to prevent something more harmful in the future. By having a better understanding of your system’s weaknesses, you’ll be able to more effectively troubleshoot issues and minimize the impact on your customers. Chaos engineering will empower you to identify weaknesses before they become SEVs.

    In Conclusion

    Implementing an IMOC role and rotation, identifying and assessing your critical services and implementing chaos engineering are three ways you can minimize the impact of high severity incidents, keeping your business running smoothly even when things aren’t.

    — Tammy Butow

  • DevOps: What a Wonderful World it Would Be

    DevOps: What a Wonderful World it Would Be

    The promises of DevOps are clear. They offer faster iterations, allowing ops to keep up with agile development and make an IT organization more responsive to business needs.

    DevOps doesn’t make less code. It doesn’t reduce configuration files. It doesn’t eliminate deployment issues (though it might reduce them). It doesn’t eliminate testing; indeed, it creates more if Agile and DevOps are taken to their logical conclusion:  More testing, earlier. DevOps doesn’t reduce the need for security.

    In short, it doesn’t reduce workloads. It might automate some of them, freeing up man-hours, which is absolutely a positive result—one of the more positive results, in my opinion. But it also introduces the overhead of the DevOps toolchain, so some of that automated freed-up time is already spoken for.

    And automation of testing (for example) is something that’s been around for decades. The first UI-based automated testing system I recall was Microsoft Test. I managed a team that used it extensively to automate test runs of the software we delivered to financial services. It worked well, and there was much gnashing of teeth when Microsoft discontinued the product.

    The same is true of most of the time-saving benefits of DevOps: We’ve tried them before, but DevOps approaches them from a different perspective—and, in many cases, a more sustainable perspective.

    I’m a DevOps aficionado, make no mistake. The reason I’m stressing what it won’t do here is because misperceptions drag it down. Don’t go into it looking to reduce costs, because at least in the short run you won’t. Don’t go into it looking to save work, because at least in the short term those savings will be minimal. Going into it for the wrong reasons means the things you are measuring will ultimately look like failures, even if all the good DevOps offers is reaped.

    Instead, go into it looking to be more responsive. Go into it looking to beat your competition to market. Go into it looking to increase communications across the team. Delve into DevOps for the right reasons, and you’ll be measuring the right outputs.

    And remember, it is an iterative process. Don’t overstate expected benefits on the first pass. Low-hanging fruit makes it look as if DevOps will do a lot of things, but low-hanging fruit often has tight ties into firmly rooted trees, making it harder to grab than originally thought. Use a measured approach and look for consistent improvements. In some organizations those improvements might be small at first. But use your knowledge of the organization and application portfolio to create a measured success plan.

    Reaping the benefits is worth the change DevOps brings. The ability to tell a line-of-business leader that the new functionality they are considering could be mocked up and ready for review in a week or two is a huge benefit to the organization. And it doesn’t hurt at all that such responsiveness will raise the rest of the organization’s opinion of IT.

    So it is a wonderful world, if your goals are the right ones.

    — Don Macvittie

  • IBM: DevOps Begins and Ends with Culture

    IBM: DevOps Begins and Ends with Culture

    Making the transition to DevOps requires as much a shift in cultural mindset as it does new platforms for building, deploying and managing applications. Between the two, however, it’s often the extent of the cultural challenges needed to be overcome that gets most underestimated.

    Michael Elder, a IBM Distinguished Engineer, said one of the most important DevOps philosophies that organizations must embrace is that No Code is Sacred (NCIS). NCIS fosters an experiment-driven culture that encourages self-editing and promotes a “don’t be afraid to fail” mentality. This approach allows teams to form questions, quickly implement means to gather data, try alternatives and incorporate real user behavior and feedback, he said.

    Manage the Scrum (Meeting)

    Other cultural issues that need to be addressed include leadership and how daily scrum meetings are managed.

    An overzealous build champion can wind up adding technologies to the continuous integration/continuous deployment (CI/CD) platform that don’t get adopted simply because the team as a group did not agree to their inclusion, said Elder.

    Daily scrum meetings don’t necessarily need to take a full hour. Non-critical, time-consuming status updates should be eliminated from the agenda. Daily scrum is more about problem-solving and looping the wider team in on critical decisions than talking about updates most team members can already read about for themselves, Elder noted. The goal of the meeting is not necessarily to inform team members of events. They should be expected to know what’s occurred. The real goal of the meeting is to decide what to do next.

    Mindsets Matter

    Elder said DevOps teams be cognizant of Conway’s Law, which says that organizations that design systems are constrained to produce designs that are copies of the communication structures of these organizations. If the goal is to develop a more agile application environment by employing microservices, Elder said, then the organization should create smaller, more independent teams to construct and manage them. However, that doesn’t mean every monolithic application needs to turned into a set of discrete microservices.

    Elder also advised organizations to embrace Design Thinking principles that encourage DevOps teams to focus on what needs to be achieved instead of enabling processes. The focus ultimately needs to be on the experience of the users who are being asked to employ an application, he said.

    The Tools Issue

    The biggest cultural challenge, however, still may be to tooling. Organizations trying to embrace DevOps clearly need new tools. But acquiring new tools won’t drive meaningful change. Too many organizations buy tools in the hopes that processes will eventually be transformed. The decision to acquire new tools needs to be closely connected with the transformation of processes, said Elder. That said some tools, such as an enterprise instance of Github, are core to encouraging the sharing and reviewing of code within a DevOps process, he added.

    Of course, Elder noted the most critical cultural change as far as tooling is concerned is keeping an open mind. Developers and IT operations teams tend to get overly attached to specific programming languages and platforms. A true DevOps culture, above all else, encourages experimentation.

    — Mike Vizard

  • DevSecOps: Digging into Root Cause Analysis

    DevSecOps: Digging into Root Cause Analysis

    We have all been there in a postmortem when someone says, “Let’s get to the root of the problem.” And, we all know what that means: Who or what is to blame? We also all know that no one wants to play the blame game, yet we all do. But it isn’t our fault (no blame, see what I did there?). It has been the default system for solving problems in business for decades. It is called root cause analysis.

    We can change—for the better.

    I recently watched a presentation from Matthew Boeckman (@matthewboeckman) titled, “There is No Root Cause: Emergent Behavior in Complex System.” Matthew is a developer advocate with VictorOps and a technology strategist with Dryan.io. He grew up a systems guy and jokes that he has been in DevOps for 18 years, even though DevOps wasn’t around, because he has always been nice to developers.

    Digging in (pun intended), root cause analysis focuses on what went wrong, and how we can prevent it from happening again.

    null

    The core problems with root cause analysis for development is that it doesn’t provide for enough complexity and its natural focus is blame, which can undermine a positive DevOps culture.

    Root cause analysis was more applicable when waterfall was the development methodology because states stayed consistent for months or even years at a time. In the age of Agile, DevOps, CI/CD, microservices, etc., states of work are in a constant flux. Root cause analysis can’t provide solutions quickly enough. As Matthew notes, in root cause analysis, things are either good or bad, working or broken, uptime or failure. The reality is that our world is more nuanced.

    What Matthew recommends is to look at it through the principle of emergence because it, “separates judgment from the good and the bad binary approach to our system health, and instead focuses on behaviors and interactions, patterns and complexities of our system. With practice and effort we can manage them to more desirable states.”

    But what does this look like in practice?

    Getting back to the analogy of the tree and its roots, the answer is more of a forest than a tree. Trees are one living organism, forests are ecosystems.

    Matthew takes this philosophy and mental picture and gives us a better system: Cynefin. It is a Welsh word that means habitat, and was created by Dave Snowden (@snowded), originally for managing IBM’s intellectual capital. It draws on research in systems, complexity, network and learning theories.

    null

    Starting in the bottom right quadrant, working counter-clockwise, its goes from simple to more complex.

    Simple

    These are patterns or behaviors that don’t require a great deal of understanding. DevOps is increasingly setting up automated systems to respond to simple issues.

    Complicated

    These are known unknowns. You can imagine a set of realities where they can occur and they are probable, but not certain. For instance, a busy harbor might get a storm that causes damage to boats, docks, etc. It is hard for the harbor manager to manage and they need to think about it. This requires people to do some thinking, and it is difficult—if not impossible—to automate.

    Complex

    This is where we start to see emergent behaviors occur. We don’t have the metrics need to understand or manage these problems or you haven’t looked at that metric before. We start with probing, going into the system and exploring. Think of any collection of humans at any scale. Things are still in the scope of probable, but things change quickly. There are many moving parts that aren’t predictable and that we didn’t fully encounter in our test methodology.

    Chaotic

    This is, well, chaos. Matthews’ real-world example was when an entire region for AWS went down, causing other regions to be overloaded as system admins were moving services. In chaos, you act, then get a sense of where things are and then respond.

    Disorder

    In DevOps, this is where you have lack of communication and collaboration. Here teams need to: reduce: figure out what you agree on; analyze: build consensus; and, iterate: move to a quadrant and continue.

    Matthew notes that knowledge and practice move patterns toward more favorable quadrants. But, complacency erodes the process. Complex systems left poorly managed will create increasingly complex processes to manage.

    null

    How to adopt Cynefin

    • In the moment: Ask, what quadrant does this map to?
    • In the post incident report: How did we manage the pattern? Was it complicated, complex, simple? What can we do to change it?
    • In your sprint planning: Devote time to manage your patterns clockwise. What can we move with a little bit of work?

    null

    The reality is that root cause analysis is really only present after the fact. Cynefin calls us to action.

    Convinced that Cynefin might be just what your organization needs or want to dig a little deeper? Share and watch Matthew’s full talk here. You can watch any of the 2017 AllDayDevOps sessions free of charge here.

    — Derek E. Weeks

  • How DevOps Startups Can Deal with GDPR in 2018

    How DevOps Startups Can Deal with GDPR in 2018

    Data protection is not a new issue. There have been numerous high-profile data hacks in recent years, with sensitive customer information released into the public domain, causing widespread condemnation and fury. Trust is a big issue. When Equifax revealed hackers had stolen the personal information of 145 million people, there was an uproar. The problem is that we are increasingly moving our lives online. Criminals have been quick to realize that data equals serious money. For this reason, DevOps startups need to pay close attention to the new GDPR data privacy regulation changes that are set to come into effect May 25.

    What is GDPR?

    The General Data Protection Regulation (GDPR) is an EU Data Protection Directive. It has been carefully designed to shape the way businesses deal with data privacy and has been described as “the biggest change to data protection law for a generation.”

    The new directive will apply to all startups and businesses operating in the EU, and any company that holds data related to EU citizens. So, if your CFD trading startup is based in the United States, but you retain data on customers who reside in France or Germany, you are bound by the GDPR.

    Data compliance is a huge issue. Cyberthreats are common and criminals are happy to take advantage of businesses that have left a backdoor wide open. Secure data management is not something DevOps startups can ignore. Any data stored in the cloud or on a server must always be protected.

    Many tech startups hold significant amounts of information. This helps them deliver a software solution more effectively. However, even if your company holds no customer data, if any of your development team are EU citizens you still must adhere to the regulations. And with many DevOps startups recruiting remote employees from far and wide, this is not as unlikely as it may first appear. So, if Carlos from Barcelona is a key player on your development team, GDPR will apply to your startup.

    Company and Customers

    GDPR is split between company data and customer data. For a small startup, it may be a case that employee data is more of an issue, but if you hold customer data, it is time to look at how you manage this data. A good way to do this is through an internal audit.

    Create a data flow diagram (DFD) that maps all the data that enters, is processed and stored and exits your systems. Provided with a good overview, you should be able to spot any data that isn’t required or needs modifying so you can remove or adapt certain funnels to save time attempting to meet compliance measures for data you don’t use.

    Planning and Preparation

    GDPR is a big step up from the Data Protection Act, so you can’t sit around and wait for May 25 to come and go. It’s vital that you examine the GDPR in detail to see how and if it applies to your organization. If it does, you will need to make organizational changes to the way you deal with data.

    Create a compliance checklist to be adequately prepared, based on the requirements from the National Institute of Standards and Technology (NIST) and Cloud Security Alliance (CSA), combined with any state or regional regulation.

    Hire a Governance Trainer

    Legal language used in compliance can be difficult for those from an engineering and IT background to comprehend. Hiring an external compliance by design practitioner or trainer will help in this respect. Ensure they have a good technical as well as legal knowledge so that they understand both sides and are able to accurately review processes, make suggestions and make sure that GDPR compliance is met.

    Task this person with downloading the Control Objectives for Information and Related Technologies and CSA requirements. They can then work with program management to put such company directives into play, alongside working with your checklist. This checklist can be used to create an open source tool for scrubbing applications and determining whether they should be onboarded or pushed back, depending on whether they meet the compliance policies.

    Notifying Employees/Customers of a Breach

    If customer or employee data is breached, accidentally or otherwise, you have 72 hours to notify them. Sticking your head in the sand and hoping it all “goes away” will not cut it. There are two tiers of fines for non-compliance: up to 10 million Euros or 2 percent of turnover, or up to 20 million Euros or 4 percent of annual turnover.

    To ensure this risk is minimized, hold everyone accountable and put in place consequences. Decide on what these will be and how they will be enforced to best implement GDPR compliance in your startup, as such hefty finance can be especially damaging for small businesses.

    Data Consent

    You must let individuals know if their data is collected. Consent is a major issue in the GDPR and ambiguous tick-boxes are not acceptable. Privacy notices must be crystal clear and written in Plain English. Clients must have the option to opt-in or opt-out if you collect data from them.

    The Right to Data Erasure

    All individuals have the right to request that their data is permanently deleted. So, if Carlos from Barcelona decides to jump ship to another DevOps startup, he can legitimately request that you delete his HR file. He can also ask you to send his file to his new employers.

    Data Protection Officer

    Companies that process large amounts of data or are involved in the monitoring of individuals—this is applicable if you track online behavioural patterns for marketing purposes—must appoint a data protection officer. This person can be anyone, but they must have knowledge of data protection practices and the law related to data protection.

    Not everything has to cost in meeting GDPR data compliance, though; there are many free resources your startup can use as well.

    Speed

    One concern is the impact on speed that GDPR will bring. Eighty-one percent of IT operations professionals believe information security policies slow them down. Avoid this for your startup by doing the DevOps yourself as it will make software deployment faster. However, insecure system changes could then be shipped through. Automation can offer a solution.

    Automation

    Treating compliance through continuous automation means your DevOps startup will put code-based controls through the normal development process: test, version, apply and modify. This makes such controls easy to collaborate on and running compliance scans will become a common feature in the development stage, testing environments and production systems.

    The average idle time before identifying a system breach is believed to be 200 days. Continuous automation means that such an issue could be detected on the engineering team’s development workstation, before it reaches production and ensuring it is dealt with to make things safe and guarantee GDPR compliance.

    What Not to Do

    The thing you absolutely shouldn’t do is ignore GDPR on the basis that it doesn’t apply to you, “because who’s going to know if Carlos from Barcelona works for my startup and so what if one of my computers is struck with ransomware and everyone’s bank details are leaked onto the Dark Web?”

    The GDPR will affect millions of businesses, so all startups need to be aware of how it will affect their operations. If you don’t comply with the directive, it could prove to be a costly mistake. Ignorance is never a legitimate defense in the eyes of the law, so check out the EU GDPR website and get informed. If GDPR doesn’t affect you, great; but if it does, think like a Boy Scout and Be Prepared.

    — Murray LeClair

  • How to Get More Visibility Into Your Continuous Deployment Process

    How to Get More Visibility Into Your Continuous Deployment Process

    I know far too many developers who cringe at the thought of continuous deployment. Thoughts of broken code and an endless stream of customer support requests puts continuous deployment into the “nice to have, but not right now” pile.

    But, it might just be a case of clearing the fog on process so you get visibility into the deployment pipeline. If your team can get more visibility into releases, you will have more confidence your shipped code is quality. With more visibility, you’ll be deploying faster and earlier before you know it.

    What are the Results of an Effective Continuous Deployment Process?

    Shipping a big release is taxing on your QA and testing teams and the pressure to get every part of the build correct is huge. Continuous deployment allows you to deploy smaller chunks of code more frequently so you have more control over the release.

    Think “shipping responsibly.”

    With that in mind, it’s easy to see why a lack of visibility can hinder continuous deployment’s goal of shipping more code to a tight schedule.

    Why is Lack of Visibility a Problem?

    Best practices for your software development lifecycle (SDLC) demand that you know what’s going on in your code. And if you are responsible for your release process, you are probably held accountable for what customers receive.

    Production demands visibility into code quality; otherwise, you risk economic damage to your business in the form of bugs and crashes. With this in mind, understandably, many teams hesitate to continuously deploy thorough lack of confidence in the quality of shippable code through the build process. If more visibility means more confidence, let’s take a look at the stages in a build process and how to increase visibility at each stage.

    How to Get More Visibility Into Your Continuous Deployment Process

    Invest in modern architecture

    Continuous deployment will require resources and modern tooling initiated by your best people. Although budget often can be one of the biggest roadblocks to continuous deployment’s success, don’t withhold funds. Invest in the tools and people first, and the process will come. Modern tooling that is updated regularly will give you the best of what’s available, allowing you to effectively see what’s happening in each stage from source control to deployment and beyond.

    Allow for manual intervention

    You may not want your continuous deployment to be fully automatic. There are risks associated with continuous deployment, and to mitigate those risks you need a human to push the “deploy” button. It’s more about making your team feel accountable for their actions. The developer sees the change go live and is the first to test them.

    A viewable stage environment

    Do you have key stakeholders waiting for updates on feature releases? Rush boxes might be the answer. A rush box is a server used for work that is not ready for the QA process. This will give anyone to give feedback and authorize changes.

    Report

    Continuous deployment is certainly not without its risks. But, if you incorporate a culture of visibility into your processes, you can quickly identify where the gaps are so you can start to scale. Your toolkit will generate a huge amount of transactional data that will help to steer your company in the right direction.

    Monitor

    It amazes me how many companies still push out changes and rely on the end user to tell them whether the deployment was successful. Defects in production code have economic impacts on your company, and undiscovered bugs and errors can rack up into technical debt. But, you can’t change what you can’t see. Small changes are always going into production, so time for QA and regression tests naturally lessens. Therefore, you need to know what happened, and when, which is where a monitoring tool such as crash reporting can help. I’ve written before about the benefits of monitoring in continuous deployment, and it’s an integral step that is often missing. Monitoring can also help you make informed decisions about when to rollback a deployment.

    Rollbacks have an economic impact to your business, much in the way errors do. However, if a decision to roll back is data-driven, it’s an easy call for a senior developer to make.

    In Conclusion

    Don’t let the lack of visibility into your deployment pipeline affect your decision to implement a continuous deployment process. Invest in people and tooling first, and the process and scale will come.

    — Freyja Spaven