devopsy.kinsta.cloud

Category: DevOps Practice

  • Top DevOps Challenges at DOES19

    Top DevOps Challenges at DOES19

    I recently attended the prestigious DevOps Enterprise Summit in London. During the three-day event, I asked people what their major DevOps transformation challenge is. I wanted to examine these at global events, asking the same question.

    I was GamingWorks’ booth boy at the event, which gave me a chance to ask a lot of delegates visiting the stand.

    Here is a summary of what they said, with my commentary based on what I have picked up on my travels as The IT Paradigmologist.

    Business Value

    • Understanding the link between IT work and the business strategy.
    • Seeing the whole end-to-end picture (visibility, transparency).
    • Measuring the right things—team performance in terms of business outcomes.

    Commentary: Not only do IT people have difficultly understanding the business value of their work, they often are poor at articulating it to business executives. My DASA white paper, “The Business Value of DevOps,” and accompanying webinar elaborate on the topic.

    Governance

    • Finding the right balance in standardization across teams.

    Commentary: Governance is such a tricky topic. What do people mean by it in the first place? How negotiable should the boundaries be? Particularly in complex and therefore unpredictable systems, professionals need to make judgment calls and act for the best. Regarding standardization, the question is always about value: Does standardization and centralization lead to significant improvements in results without introducing frustrating bureaucracy? Otherwise, you’re better off letting the individual teams decide for themselves what is most effective for their value streams. Yes, it might be more expensive than one-size-fits-all, but the benefits usually outweigh the costs.

    Velocity

    • Doing things quickly while maintaining quality.

    Commentary: It’s a question of balance. Not only speed and quality (depending how you define it, of course), but also resilient operations, co-created value and assured conformance. I like the Deloitte #BVSSH way of putting it: Better, value, sooner, safer and happier.

    Value Streams

    • Breaking down silos.
    • Collaborating better between silos.
    • Giving more autonomy.
    • Integrating DevOps parts in a predominantly non-DevOps value stream and vice versa.

    Commentary: On the conference scene, almost everybody has an idea of what value streams are. Visit an average IT department though, and for most people it’s an alien concept. We live in a bubble. You have to take a few steps back and start from where they are. Take it step by step. Collaboration seems to be one of the top challenges as confirmed by the recent DOI skills survey, which had this as a top soft skill. It may be a soft skill but it is hard to achieve. The article “DevOps and Collaboration: Fraternizing with the Enemy” provides some guidance on collaboration.

    Ops

    • Discovering a new way of doing ops.
    • Shifting from old ops to new ops.

    Commentary: This is a very hot topic. Witness the promising changes in ITIL 4. Less process, more heuristics, more value-orientation (for instance, value streams and co-creation with the service consumer) and more human. It’s very encouraging to see how DevOps communities—some more enlightened than others—are developing their ops capabilities to restore the balance. But ops work differs fundamentally from dev work, and that has to be reflected in the way of working.

    Organizational Change

    • After the initial enthusiasm, “what’s next and how?”
    • Overcoming resistance to change and inertia in general.
    • Accepting the consequences of doing DevOps.
    • Unlearning 30 years of doing the same thing.
    • Learning quickly.
    • Developing skills to transform.
    • Assessing the risk of transformation.
    • Sharing (and learning from others’) practices.
    • Upscaling.

    Commentary: There is too much lip service and too little commitment. There are notable exceptions, but too many people are looking for the quick fix and believing they’ll find it. Good luck. As Taiichi Ohno said, “You have to think for yourself and face your own difficulties instead of trying to borrow wisdom.” If you don’t know who Ohno-san is, that’s a good place to start. If you’re serious, take a look at Paul Wilkinson’s astute observations about challenges and enablers for DevOps success at DOES18 USA. You’ll notice many similarities with my findings at DOES19 UK.

    Culture and Emotions

    • Getting enough diversity in the team.
    • Getting different personalities to work together.
    • Trusting and being trusted.
    • Feeling non-DevOps teams are worth less, but not worthless.

    Commentary: More on culture and people can be had on the DOES18 blog, but it is encouraging that people are actually talking about this more often. It’s something that almost always crops up in The Phoenix Project business simulations that I facilitate for organizations. Topics such as psychological safety: whether you feel comfortable being and expressing who you are. So, do you? Dr. Christina Maslach referred to the toxicity of some workplaces with the analogy, “You want less toxic mines, not more resilient canaries.” Work-in-progress.

    Tooling

    • Understanding which tools are around (tool confusion).
    • Integrating and then orchestrating tools.

    Commentary: Finally a topic I know little about, but I wholeheartedly endorse many vendors’ socks.

    — Mark Smalley

  • Using Automation to Control Cloud Costs

    Using Automation to Control Cloud Costs

    By building in cost control automation from the very beginning, you can keep your cloud costs low while encouraging innovation.

    Infrastructure automation has become vital for every modern sys admin and operations team. Applications and systems are evolving from single-host mainframes to distributed microservices architectures. This automation has gone from basic operating system installation and setup to full-blown multistep deployments of production code from a single developer’s commit. By automating these mundane processes and eliminating the element of human error, production systems are much more stabile than ever before.

    But why stop at automating deployments? There are other elements that need to be automated, too. One of the main ones is cost control.

    Continually rolling out new infrastructure without ever taking a step back to analyze the cost often leads to the panic-driven cloud-bill-based phone calls from your finance department. Instead, applying automation to your cloud costs can help you to save money, so you never have that unpleasant surprise of a major bill.

    Scaling Up Without Ever Spinning Down

    Developers and operations teams often use infrastructure automation early in application development and deployment processes to get servers and databases deployed and functioning. Modern automation tools aren’t just powerful, but they’re also quick to deploy and fit into your current workflow. This is fantastic, but the problem is the automation effort can taper off once the environments are running—too often, users and teams move on to the next project before figuring out a way to keep costs from getting out of control. Then it’s too late, and users simply accept that money needs to be dumped into the deployment pipeline to keep everything on task.

    Easy-to-use automation is the key to spinning these environments up efficiently. It also can be key for keeping the costs of these environments low. Sure, you may need to keep the production systems scaled up for maximum application performance and customer satisfaction, but what about the test lab, sandbox environment, dev systems, UAT servers, QA deployments, staging hosts and other pre-production workloads? Having giant environments with system sizes that match production can be useful for some testing, but leaving it all running can be easily doubling your cloud costs. This is true for each environment you have, including for elements that are only used for a fraction of the time.

    As your infrastructure automation toolkit grows and evolves, there are a few things you should start building into all of your applications and deployments. These include:

    As this list grows, there’s one more thing you need: continuous cost control.

    By building in cost control automation from the very beginning, you can keep your cloud costs low while maintaining the flexibility required to keep up the pace of innovation. Without this, your costs are destined to rise faster than you intended, which will then cause headaches (and endless meetings) in the future. It may not be coming out of your bank account directly, but saving money at an enterprise organization is everyone’s job. Automating this is the key.

    — Jay Chapel

  • Doing DevOps in an Interconnected World

    Doing DevOps in an Interconnected World

    Doing DevOps would be simple if you had total control over the tools, code and infrastructure that your company used. But the fact is, you don’t. Instead, we live in a world where most CI/CD chains connect in some way to third-party resources or processes.

    In other words, doing DevOps effectively means integrating with entities that are external to your organization. Here’s a look at what this entails in practice and why it’s so important.

    The Interconnected World of DevOps

    Conversations about DevOps often assume that all DevOps processes happen internally. In reality, many software delivery pipelines depend on third-party resources and services. Consider the following examples of ways companies could make their software delivery chains dependent in part upon third-party agendas:

    • They integrate open source code into their codebases and therefore depend on an open source community to write some of their code.
    • They maintain relationships with partners and need to be sure their release schedules and toolchains are compatible with their partners’ needs.
    • They need to make sure the applications they deliver work on different operating systems, browsers and hardware devices that are controlled by third parties.
    • They are subject to regulatory rules handed down by third parties and must ensure their delivery chains meet those regulatory requirements.
    • Their applications depend on APIs that are designed and updated by third parties.

    In these ways and more, the CI/CD pipelines organizations build to do DevOps need to be able to integrate with, and adapt to, decisions made by third parties. If they don’t, the organizations risk problems such as broken code, the loss of partners and regulatory compliance issues.

    Succeeding at Interconnected DevOps

    This is why it’s critical to design CI/CD pipelines and other DevOps processes in ways that make them mesh with third-party resources and services. In practice, this means:

    • Adopting tools that are flexible and have broad coverage. You don’t want to tell a partner you can no longer work with their code because your CI server doesn’t support their new programming language.
    • Use branching in the CI/CD process. Multiple branches give you more flexibility to support different target environments. They also help you react to sudden changes made by third parties without having to overhaul your entire CI/CD process.
    • Make sure your collaboration and communication channels integrate with those of third parties on whom you depend. If you use open source code in your application, for example, add the open source project’s security announcements to your Slack channel so you’ll be notified right away when a vulnerability arises.
    • Choose your integrations wisely. Just because you can integrate a third-party API into your application, for example, doesn’t mean you should. Too many dependencies put the efficiency of your CI/CD pipeline at risk.

    No matter the details of your CI/CD chain, the most important thing is simply to recognize that we live in a very interconnected world and most organizations don’t have total control over their software delivery processes. That’s just a fact of modern DevOps. But with the right planning and strategies, you can ensure that third-party dependencies add value to your software delivery pipeline rather than creating unnecessary risk and overhead.

    — Chris Tozzi

  • Making Your Cloud Work For You

    Making Your Cloud Work For You

    Does your cloud work for you, or do you work for your cloud? Given the enormous dependency and costs associated with cloud computing, that is a question more and more DevOps teams are asking these days. After more than a decade of pressure from higher-ups to move workloads to the public cloud, DevOps teams are now facing the reality that there is such a thing as too much cloud.

    Don’t take that the wrong way. I’m not here to tell you the cloud is inherently bad or we should return to the days when all our critical workloads ran on bare-metal servers housed in a closet in the back of the office. Public cloud services offer a number of critical benefits and DevOps teams should take advantage of them.

    However, to use the cloud responsibly, it’s crucial to control for two risks. The first is cloud computing costs, which easily can get out of hand if not managed properly. The second is a rigid dependency on public cloud services that undercuts agility and makes it more difficult to deploy applications at scale—which is exactly the opposite of what the cloud is supposed to do.

    In this article, we’ll take a look at what causes each of these risks and how DevOps teams can respond to ensure their cloud strategy actually delivers value.

    Risk 1: Cloud Computing Cost

    You’ve probably heard a lot of talk about how the cloud saves money. If you use the cloud effectively, that is true. However, because of the complexity of cloud computing cost structures and the difficulty of comparing cloud computing costs to on-premises costs, it’s easy to end up with a cloud hangover after paying a much higher cloud computing bill than you bargained for.

    There are several distinct challenges at play in predicting and understanding the true costs of your cloud. Let’s take a look at each one.

    Never-Ending Cloud Computing Costs

    When you move workloads to the cloud, your costs vary month by month and are spread out over time. At first, moving to the cloud feels like a great money-saver because it frees you from having to make large, upfront purchases of on-premises hardware. However, just as leasing a car ultimately costs more in the long run for most people than owning, paying monthly for cloud services hosted on someone else’s infrastructure adds up. If you ignore this fact, you can end up failing to appreciate the true cost of your cloud computing bill over the long term.

    Unpredictable Cloud Costs

    Another fundamental challenge of cloud computing cost compared to on-premises is that, in the cloud, costs structures are opaque and unpredictable. Public cloud vendors change the prices of services without notice. The same service might cost more in one cloud region than another, while some services have multiple fees attached. For example, if you store data in the public cloud, you pay not just a per-gigabyte cost to store it but also fees whenever you move the data. This adds up to tremendous difficulty in predicting your true cloud computing bill.

    On-premises costs are not entirely straightforward to predict either, of course. There are some fluctuating expenses, such as the cost of electricity. However, the lion’s share of on-premises computing costs is the hardware itself, which is a very simple and tangible expense. It’s also a one-time expenditure that you can amortize over the life of your servers. Once you have purchased a server, you don’t have to worry that its effective cost will change month to month, as you would with a virtual server in the public cloud.

    Overlooked Workloads

    In the cloud, it’s trivially easy to spin up a new workload. It’s also easy to leave a workload running after you no longer need it—and public cloud vendors don’t go out of their way to notify you when you have resources that are activated but sitting idle. You can’t blame them; that would be like a restaurant telling you you’re ordering too much food or a builder telling you your house is bigger than you need.

    If you don’t carefully manage your cloud workloads, they add up to extraneous and unnecessary costs.

    To be sure, you can have unnecessary workloads running in an on-premises environment, too. But the difference is there is no substantial cost increase on-premises that accompanies each extraneous workload. You’ve already paid for the servers and you don’t get a bill for every service you have running on your own infrastructure.

    Risk 2: Lack of Cloud Agility

    Beyond unnecessary costs or inefficient cost structures, relying too heavily on the public cloud also undercuts the agility of DevOps teams.

    When you run a service or workload in the public cloud, moving it to different infrastructure can be challenging. Not only is it usually difficult to migrate from one cloud to another, but also doing so entails adapting to a new cost structure and possibly paying more for the same set of services.

    There is an irony here: Conventional wisdom suggests that one of the main reasons to migrate to the cloud is that the cloud is inherently more scalable and agile than on-premises infrastructure. That may be true in some cases, but when you factor in the cost and the difficulty of migration from one cloud to another, the cloud quickly starts to look less agile and scalable.

    On-premises infrastructure doesn’t always scale painlessly either, of course. I don’t dispute that. But the cloud doesn’t always live up to the hype when it comes to scalability and agility. On-premises and cloud-based infrastructures both have limitations in this regard.

    Cutting the Cord: Making the Cloud Work For You

    Now that we’ve identified the inherent (and underappreciated) cost and agility challenges of cloud computing, let’s consider solutions.

    Optimize Cloud Computing Costs

    Cost optimization is something you should always do on any infrastructure, and the cloud is no exception.

    There are many guides out there on cloud cost optimization. Some vendors also offer tools designed to monitor your cloud workloads and identify underused resources or help you right-size cloud service instances. These are all good ways to help reduce your cloud-computing bill.

    Be Cloud-Agnostic

    Another useful strategy is to architect your cloud workloads in a way that makes them as cloud-agnostic as possible. Instead of deeply integrating with a particular cloud vendor’s services, rely on third-party tooling whenever you can.

    For example, if you run containerized apps in the cloud, run your containers in standard virtual server instances and use a third-party registry instead of using the container registry and managed container service of one cloud vendor. This will require a bit more work on your end, but it will give you the freedom to move your workload to another cloud easily. It also enables cost-efficient scalability, because you are not beholden to the cost structures tied to the vendor’s managed container service. Rather, you are using only virtual server instances (and perhaps some storage to go along with them), which have much more flexible pricing on public clouds.

    Don’t Be Afraid of On-Prem

    My final recommendation, and my most important one, is to embrace on-premises infrastructure.

    Again, I say this not because I want to move backwards and return to a world where everything runs on-premises. I’m not anti-cloud.

    However, from a cost and agility perspective, the public cloud is not the best fit for every type of workload. Workloads that have very convoluted cost structures in the public cloud—or that would be difficult to run in a public cloud without sacrificing a great deal of mobility and scalability—are often a better fit for on-premises environments.

    You may be thinking that embracing on-premises means sacrificing all of the advantages of the cloud or having to forgo cloud-native workloads. Fortunately, that’s not the case. There are software vendors that offer the services of a public cloud, but on your own enterprise infrastructure using the public cloud APIs.

    Thus, you can obtain the cost and agility benefits of on-premises infrastructure (not to mention the security advantages) while still running cloud-native applications. In fact, you don’t even need to overhaul your CI/CD pipeline; you can simply deliver your cloud-native apps onto on-premises infrastructure that runs services designed to be compatible with those of the public cloud, even though they are hosted on your own servers.

    Conclusion

    Don’t move workloads to the cloud just because it seems like the thing to do or because someone told you it’s necessarily better. Some workloads work better in the cloud while others are best hosted on-premises. An all-in cloud strategy is not typically the most cost-efficient or agile architecture you can adopt.

    — Ariel Maislos

  • How DevOps Training and Certification Benefits the Team

    How DevOps Training and Certification Benefits the Team

    Cisco Systems recently announced sweeping changes to its training and certification programs to increase the real-world value of both. Cisco’s move demonstrates a trend by organizations that provide training and certification to place more value on the practical aspects of educating technology professionals—which, in turn, should have a major impact on those looking to build DevOps teams.

    “Advances in network technologies have enabled new applications and business models—converged IP networks enabled voice over IP, 3G networks enabled mobile applications and 4G networks enabled mobile video,” said Susie Wee, senior vice president and CTO of DevNet at Cisco Systems. “DevOps has become a key factor for bringing those new networked applications to life. Training and certifications for network professionals and software developers have become a must. The goal here is to bring enhanced software skills to networking, enabling application developers to write applications that leverage the full capabilities of the network.”

    Cisco’s realization that real-world knowledge brings demonstrable value to businesses has culminated in new DevNet Certifications, which are designed to demonstrate an individual’s ability to bring software skills to evolving network technologies. “The only way to fully leverage the capabilities of the network is to build DevOps teams with both networking and software skills that can address the intricacies of both software-defined networking and physical networking solutions,” Wee added.

    Cisco is not the only company pursuing a change in training and certification processes. Industry group CompTIA recently revamped its certification programs. For example, the latest version of the CompTIA Linux+ exam has been redesigned to validate the competencies and skills required of early career IT professionals supporting Linux systems.

    “For certifications to have real value today, people must immerse themselves in the technology,” noted James Stanger, CompTIA chief technology evangelist. “The new Linux + certification uses performance-based testing that presents candidates with real-world scenarios, meaning that achieving certification demonstrates the ability to function in the real world.”

    CompTIA may very well be one of the leaders in the movement to bring real-world relevance to the certification process. The company has been using performance-based testing and has been embracing certification pathways to build teams. “For DevOps teams, critical skills, including project management, both agile and waterfall, are important. Developers will need to know the operating systems they are developing in, so they’ll need to understand Windows and Linux,” Stanger said. “The idea is that for DevOps, you need to get deep into the guts of operating systems and understand them well. You can specialize in one area, but you’ll also need to understand databases, relational databases and object-oriented databases.”

    Cisco also understands the importance of educating DevOps professionals. “Our new Cisco Certifications, with new DevNet Certifications, are designed for training and certifying network professionals and software developers, so they can bring software skills to networking,” Wee noted. “This new education process teaches application developers how to write applications that leverage the full capabilities of the network, and it helps build IT teams with network skills and expertise and software skills and expertise so they can leverage the full capabilities of the network, helping to support the DevOps process.”

    Cisco is also investing in the idea of building a community in the form of the company’s DevNet offering. DevNet was created some five years ago to help network engineers and application developers take full advantage of the revolutionary changes shaping the industry. The company is now focusing its efforts on what it sees as its customers’ biggest problem—network automation—and has created the DevNet Automation Exchange, a community-based developer center where networkers and developers can learn to “walk, run and fly” with network automation using Cisco’s products and the full capabilities of the new network.

    According to Wee, those three processes can be broken down into:

    • Walk: Get visibility and insights into their networks in a read-only way. This enables use cases such as gathering telemetry and insights from running networks and performing audits to ensure the ongoing security of the network.
    • Run: Activate policy and intent across different network domains. This enables use cases such as providing self-service network operations that comply with security policies and operational guidelines.
    • Fly: Proactively manage applications, users and devices with DevOps workflows. This enables use cases such as deploying applications in continuous integration and continuous delivery (CI/CD) pipelines while configuring the network in accordance with new application policies.

    Wee noted Automation Exchange has been seeded with more than 50 code repositories, many contributed by Cisco’s engineering teams and SEs. Now, she said, the company is inviting DevNet community members to submit code that solves their use cases.

    Updated certification and training resources from the likes of Cisco and CompTIA will go a long way to helping build DevOps teams that can hit the ground running.

    — Frank Ohlhorst

  • The Purpose of Microservice Architecture

    The Purpose of Microservice Architecture

    Microservice is an architectural style that can be implemented during complex application development within an organization. The implementation of this architecture allows the application being developed to be highly maintainable and testable, loose, independent during deployment and also strongly organized around business capabilities. In another way, the microservice architecture can be explained quite easily. It is a software development technique that allows for the structuring of an application. The structure of the application being considered will be that of a set of coupled services.

    Any application developed using the microservice architecture is easy to understand, develop and test. More so, the application will have a sort of rigid attitude to degradation in the long run. This is due to the concept adopted during design.

    The design of the microservice architecture ensures that applications are separated into smaller services, states, etc., until they become fine-grained and light in weight. Therefore, the modularity achieved encourages the adoption of methodologies that will allow for the division of labor. These methodologies will aid in the concurrent design, development and testing of the sub-services independently.

    While this architecture is suited to certain scenarios, it (like every other architectural style) has its demerits. Microservice experts advise on the study of the architectural platform before adoption. The implementation of the microservice architecture can be affected by a number of programming languages.

    Within the context of this article, I discuss the purpose of the microservice architecture, and also examine the upside and the reasons why they should be learnt and implemented. I also take a look at particular niches that allow the use of microservice architectures and why they use it.

    Architecture, Examples and Types of Microservices

    The microservice architecture is a major decomposition of the monolithic architectural method. The monolithic architecture had applications developed from the top to bottom and also had the entire development process run as a single unit. The microservice architecture works in contrast to this by enabling modularization. The core of the microservice architecture has requirements that are intensified and made possible by the separation of the primary task into smaller services, in which one service represents just a requirement, function or need.

    Examples of microservice architectures can be found in airline service scenario. If we pick out the flight booking component, we could find out that this primary task could be separated into inventory adjustment, seat allocation, fare calculation, timetable lookup, reward management and customer update.

    Applying Microservices

    Research has it that the majority of the companies that make use of microservices are quite on the high side. This majority includes organizations such as Netflix, eBay, Amazon, PayPal, Twitter, The Guardian, etc. This list also includes the United States Government Digital Service.

    The application of microservices can work with a number of technologies but a number of researches attribute effective usage of this architecture with the cloud environment and the DevOps environment as well. These tools and environments help to manage the operational complexities that affect the swift functionality of the architecture.

    Why Do They Use It?

    Most of the organizations that make use of the microservice architecture did transcend from the monolithic method. This is mostly because the method offers organizations the ability to decompose primary applications into smaller composed pieces. These pieces can then be built independently (which is easier) and also maintained independently as well. Developed pieces have the ability to communicate with each other and work together as well.

    The adoption of microservice architecture provides the management of an organization and everyone involved in the development of the application with a decentralized approach towards building the software. This approach breaks the development process into a series of deploy, rebuild and redeploy.

    The choice of a microservice architecture also grants organizations the ability to implement variant strategies, tools, techniques, etc., for services that are involved. Studies show that organizations that implement this architecture have a high proficiency rating. This is due to the usability of the architecture in independency, deployment, replacement, etc.

    The architecture encourages easy debugging of errors and detection of faults in general. When an error is spotted, the segment from which it originated can be easily traced and the fault can be found shortly after. This, as explained earlier, is because primary tasks are separated into smaller services. Each service can then be fine-grained to simple state. Before the simple state is attained, it is also possible that the service can be separated into a smaller group of services. This option helps the development team actualize a major concept within this architecture, which makes sure that a service handles just one task. Also, the architecture allows for the adoption of management techniques that assist the organization. These management techniques include options such as change management process, continuous software development processes, problem and incident management techniques, etc.

    Benefits to Using Microservices

    The benefits to using the microservice architecture include:

    1. The actualization of scalability when considering customer-focused services.
    2. Independent deployment of services.
    3. The ability to work with more than one development tool (such as  programming languages).
    4. Easing the modularization of tasks within the project as a whole due to the modularization of the application being developed.
    5. Support for a wide range of platforms and devices such as web, mobile, internet of things, etc.

    — Subramonian

  • 6 Traits That Define DevSecOps

    6 Traits That Define DevSecOps

    How do we define DevSecOps? A combination of DevOps and security is readily apparent, but the philosophy goes much deeper. In a recent eBook, The State of DevSecOps, we asked industry experts to define what DevSecOps meant to them. Below, we’ve condensed their answers into five core attributes.

    Following these principles, CIOs or CTOs now have a DevSecOps doctrine applicable to nearly any software development and release environment.

    Security Automation

    DevSecOps emphasizes security objectives within the automation processes. Ben Newton, director of product marketing and evangelism at Sumo Logic, defines DevSecOps as first making security requirements and objectives a clear part of the continuous integration (CI) and continuous deployment (CD) cycle.

    Culture of Security

    DevSecOps establishes a team culture that embraces security concerns. As stated by Ben, a security team must provide “clear guardrails for developers as to what is fair game and what is not appropriate from a security perspective.” Building security expertise to integrate security into the entire DevOps lifecycle is critical.

    De-Siloing IT

    DevSecOps is about eroding boundaries. Just as DevOps has eroded the traditional separation between software engineering and IT operations teams, “DevSecOps further erases the walls between the DevOps team and IT Security,” said Tim Jarrett, senior director of product management at Veracode. “DevSecOps is about building a bridge between the security and DevOps teams,” echoed Dan Hubbard, chief product officer at Lacework.

    Security Shifts Left

    DevSecOps places security earlier on in the development process. IT security is traditionally viewed from a risk avoidance and compliance standpoint. Rather than viewing security as a gate, Tim noted that within DevSecOps, “security is better positioned to integrate earlier in the development cycle where they can actually make a difference.”

    Security Enables, Not Stalls

    DevSecOps supports, not stalls, agile development. DevSecOps doesn’t have to be sluggish. As Dan described, DevSecOps must “support the need for DevOps to move fast, but in a way where security is not ignored.” By embracing a security-as-code mindset and involving practices such as automated threat detection, agility is not sacrificed.

    — Bill Doerrfeld

  • Data Mapping and GDPR: How Are They Related?

    Data Mapping and GDPR: How Are They Related?

    Every business enterprise receives data from an array of diverse data points. The volume of data is growing, and businesses understand the importance of leveraging data and converting them into actionable insights. Owing to data diversity, it becomes important to integrate them in a standard format. This is where data mapping takes a precedence. It helps establish relationships between different data models. Although data mapping offers several business benefits, it is also an integral aspect of the General Data Protection Regulation (GDPR) compliance. This post discusses the concept of data mapping and how it helps you adhere to GDPR.

    What is Data Mapping?

    As the name suggests, data mapping is a term for mapping the journey of data in an organization. A data map answers questions ranging from the origin of data to its journey through the organization. Today, most business enterprises are using various types of data mapping software, including ETL data mapping tools, to achieve their data mapping targets. Depending on the capabilities of these software applications, data mapping is used to perform a wide range of data management tasks such as data integration, data migration, data warehousing and data transformation.

    How Does a Typical Data Map Look Like?

    Typically, a data map includes the following information:

    • Type of data collected
    • Basis of data processing (personal or sensitive)
    • Purpose of data collection
    • Storage period of data
    • Place of storage
    • Conditions of data storage
    • Various data transfer destinations
    • Location of third party vendors (especially, international third party vendors)
    • Protocols followed during the data transfer (to international third party vendors)
    • Legal basis for data processing

    GDPR Compliance and Data Mapping

    The GDPR mandates business enterprises to ensure fair handling and security of the information they own. To assure the security, it is important to track the data journey from its point of origin to a conclusion. Without a detailed insight into data lifecycle, it is difficult to implement any security. Thus, data mapping becomes an essential step to achieve GDPR compliance. The following GDPR articles and their requirements will help you understand the importance of data mapping.

    • Article 30: Maintaining a written record of data processing activities in an organization is mandatory. A data map serves as a best written record, which organizations can easily retrieve and present at the time of GDPR audit.
    • Article 25: Data mapping helps organizations showcase their commitment towards data privacy.
    • Article 6: This article governs personal data processing under a lawful basis. Data mapping helps organizations to show the basis of processing–personal or sensitive.
    • Article 28: Data maps enable GDPR auditors to understand the extent of third party access to organizational data. All organizations are required to maintain a list of third-parties with whom they might be sharing their business data for various reasons.
    • Article 35: This article discusses Data Protection Impact Assessment (DPIA), and data maps enable business enterprises to meet this requirement effectively. To analyze the risk of a particular process, an organization needs to understand the data flow across an organization–how the data is being collected, where it is stored, who uses the information, how long it will be retained and so on.

    How an Efficient Data Mapping Software Helps in Achieving GDPR Compliance

    Now, what is an efficient data mapping software? It can be any application that aids a business organization to exploit the benefits of data mapping while adhering to the GDPR regulations. The following are a few important features of such GDPR compliant data mapping tools, which helps distinguish them from others.

    • Ability to Identify and Map Personal Data: Advanced data mapping software applications, which may or may not involve ETL data mapping software, allows users to identify and map personal data flow in an organization. They are designed to handle a large influx of data. These solutions utilize metadata to identify various data elements. The metadata helps in creating data maps that answer questions ranging from the origin of data, type of data collected, data subjects involved in process, service level agreements, among others. Thus, advanced data mapping software applications enable users to maintain, manage and monitor data maps for business efficiency and GDPR compliance.
    • Helps Ensure Data Security: Professional-grade data mapping software solutions are equipped with various security features that enable users to secure their database while providing them access to data, whenever required. They also allow organizations to conduct risk analysis of their data and ensure adherence to DPIA.
    • Assures a Better Control: Earlier reference to GDPR articles suggest how GDPR mandates enterprises to ensure data protection while handling personal data. An efficient data mapping software will help them address this need. They enable users to modify their applications to establish control, as well as safeguard personal data effectively.

    Achieving GDPR compliance is not an easy task. However, you can easily address it at various stages and it will help your business adhere to privacy standards while effectively meeting customer demands. You can take advantage of various data mapping software applications including advanced ETL data mapping tools to create effective data maps, stay compliant and, above all, drive your business forward.

    — Chandra Shekhar

  • Collaboration Inside and Out for Innovation

    Collaboration Inside and Out for Innovation

    Across all industries today, organizations increasingly are migrating their infrastructures to the cloud in a bid to boost agility, lower the cost of IT ownership and reduce the risk of outages.

    Firms are under more pressure than ever before to innovate new products to keep up with newer, more agile firms. But before they can reap the benefits that come with going totally serverless, organizations must deal with the immediate challenge of meeting the ever-growing expectations of today’s digital-first customer.

    For newer firms, this is arguably a much easier undertaking, as their IT infrastructures will have been built using modern programming languages, technologies and software development practices. They also will most likely have adopted an agile approach and be highly adaptable to changing business needs. Conversely, older firms typically have a large body of IT that lives in an entirely different world. With infrastructures made up of a patchwork of interdependent legacy applications, built on top of monolithic architectures, these firms struggle to innovate customer-facing applications at pace. But if they are to compete, established players must adopt a similar approach to their more agile counterparts.

    Don’t Hate the Players

    It is completely natural for technology to evolve. The problem faced by many organizations that were pioneering in their use of technology is that they have now become victims of their own success. Take the world of financial services as an example. Banks that began building out their IT infrastructure decades ago have had to adapt gradually to new technologies. But it’s only in recent years that digital transformation has become a key business objective. In the not-so-distant past, banks may have been able to get away with taking their customers for granted, as the competitive landscape lacked any real diversity. But today competition abounds, with challenger banks and fintechs bringing innovative new products and services to markets traditionally owned by the banks.

    If the banks are to compete, they must seek to collaborate with the fintechs. Cloud-based open platforms that focus on open APIs are the most effective means of achieving this. By providing fintech partners with access to core systems via open APIs, the banks get to concentrate on their core operations while innovation from the outside helps them to deliver new applications to their customers.

    Historically, the banks would have been fiercely protective of their IP and core systems, but in collaborative environments, it is exactly this IP that can be used to create value in a safe and secure way. This is why banks increasingly are seeing fintechs as partners rather than poachers, as they are above all technology providers that are eager to have their services leveraged by large financial institutions. Similarly, banks understand that they must become more agile and entrepreneurial to deliver what their customers want. So, working with third parties that embody these characteristics is the most effective means of doing so.

    Embrace Internal Collaboration

    As well as overcoming legacy systems, legacy culture must also be addressed if true digital transformation is to be achieved. The first way to do this is to align application development with business goals, showing how a focus on the former helps to achieve the latter. Here, the adoption of a DevOps mindset can bring about the right culture for increasing innovation within. Though not entirely applicable to legacy systems, which must be evolved at a much more gradual pace using a waterfall approach, DevOps can be applied when it comes to systems of differentiation—which include decision-making, pricing, analytics and risk management tools in the middle office, as these tools will need updating more often to accommodate changing business practices or customer requirements.

    Creating cross-functional teams that include business analysts, developers, data scientists, QA, UAT and compliance experts means there is collective focus and responsibility for the delivery of applications. It also enables more agile development so that applications can be pushed through to the production environment continuously, rather than simply throwing a finished product over a wall with little care as to how it performs.

    In recent years, IT increasingly has been seen as a business driver. As software development methodologies and practices that rely on a collaborative culture continue to mature and further prove their value, partnerships between larger organizations and disruptors will be viewed in much the same way.

    — Mitesh Soni

  • DORA Adds to DevOps Brain Trust

    DORA Adds to DevOps Brain Trust

    I am super excited to announce that Jessie Frazelle is joining the DORA team this year as an author on the “Accelerate State of DevOps Report.” I am a big fan of hers and we are all thrilled to be working with her. For those of you not familiar with her work in containers and security, she wrote a blog post to introduce herself. ~Nicole

     

    Hello! 👋

    I am thrilled to join as a co-author of this year’s “Accelerate State of DevOps Report.” For those who might not know me: I have worked as an engineer on container technologies and ran containers in production since Docker was at version 0.4. Before that I worked at a bunch of different startups as an engineer. I have been a long-time reader of the State of DevOps Report and jumped at the chance to get to work on it this year. It’s an amazing team of people!

    Each year I have admired the report for its insight and intuition backed by data. This year, I am excited to add my experience in open source and perspectives from working at various companies both large and small. Open source is very near and dear to my heart and I think a lot of the insight gained from the report can also be applied to open source projects to make contributors more productive. Through various jobs over the years, I have seen a lot of different DevOps practices at many different scales and I am looking forward to applying my perspective from those experiences as well.

    As someone who has been an engineer for more than a decade, I find the study of high performers in organizations fascinating. Personally, when I put my full-effort into something I am building, it comes from a place of caring. The link from culture and a strong mission to outcomes, as seen in past reports, falls in line with a lot of my experience. I can see myself using this year’s report a lot in future conversations to show something I know is true and is also backed by data—I can’t wait to share what we’re finding!

    Lastly, I love how the State of DevOps Report focuses on outcomes, not whatever is the latest hype or argument between thought leaders on the internet. Getting companies to a place where they can make changes and ship features faster, safer and more securely is a noble mission! I can’t wait for you to get your hands on this year’s report!

    Best,

    Jessie

    — Jessie Frazelle