devopsy.kinsta.cloud

Tag: scrum

  • Netskope Draws Strength from DevOps In Support Of Scrum Development

    Netskope Draws Strength from DevOps In Support Of Scrum Development

    Netskope keeps enterprises and their data secure despite the unbridled mix of approved and unapproved mobile apps in the market and on employee devices. Employees use unapproved mobile apps (BYOA) on their BYOD smartphones, tablets, and laptops when connected to networks away from the enterprise. IT professionals are keenly interested in tools that help them to have visibility into these apps and devices and to secure them.

    Enterprises face many issues with mobile apps. They confront challenges where data location and access as well as security are concerned when complying with PCI DSS, HIPAA, SOX, GLBA, FISMA, FERPA, and the ISO standards. Netskope works to control app usage, protect enterprise data, and maintain compliance on various endpoints whether the device is on premise or remote / mobile, all in real-time.

    “Netskope uses a distributed policy enforcement model that lets your people use these apps securely so you don’t have to block them completely,” says Abhay Kulkarni, VP of Engineering, Netskope.

    Netskope’s Security Duties Dictate Its Development Path

    “We do not follow Continuous Delivery,” says Kulkarni. Here are two of the many reasons for that.

    First, Netskope’s customers demand that software releases are predictable. “Our customers are large enterprises that want predictable change windows so they can ensure that they make the necessary changes on their side or that they prepare for our updates,” says Kulkarni.

    Second, because Netskope is a SOC-1 Type-II and SOC-2 Type-II certified company, it must comply with the requirements that comprise these certifications. Tightly-knit change management is critical to SOC certification. “As we do each change, we need to make sure that all changes meet the security, availability, integrity, and confidentiality criteria of these certifications by testing the entire product end-to-end,” says Kulkarni.

    But Netskope follows other DevOps principles such as using its own adaptation of a well-accepted source control workflow model as well as an automation model, a model for bootstrapping machines, and also models for software installation and change management.

    Netskope performs a balancing act between complying with SOC-1 and -2 trust services criteria and principles and maintaining rapid feature development. On the one hand, they manage to limit what individual developers can do on production machines—many times disallowing access. On the other hand, they use Sumo Logic to enable their coders to peruse anonymized logs in order to gauge how the service they instantiated is working.

    Goals, Tools, & Challenges

    “The key goal for Netskope’s DevOps approach is to ensure developer responsibility for running, monitoring, and scaling their services,” says Kulkarni. Netskope’s developers and operations staff work together from design through development while complying with a common framework, which helps in resolving development issues and brings consistency to data center operations, according to Kulkarni.

    Netskope holds developers individually accountable for making their software ready for automated deployment. To help with that, Netskope uses familiar tools such as Ansible, which helps Netskope to automate everything in production. “Right from bootstrapping a machine to changing network configurations, we automate all commonly repeated tasks,” says Kulkarni.

    Netskope deploys Sumo Logic agents using Ansible as a part of bootstrapping so that machines are configured from the get go to send logs to Sumo Logic. “Each service has standard logging methods that allow Netskope to create performance dashboards and to track errors,” says Kulkarni. Netskope uses a number of other tools that are commonplace in DevOps, such as Nagios, a monitoring tool and various external monitoring tools for tracking end user performance.

    The biggest challenge for Netskope was not in tooling but in building a DevOps culture where developers lead the charge in responsibility for quality, error-free operation of their services. “In a non-SaaS world, developers ship a version of their product and celebrate. However, shipping a SaaS product is just the beginning. There is really no celebration since you are now responsible for the upkeep of the services,” explains Kulkarni.

    A Trip Through Netskope’s Development Process

    Netskope starts with a Scrum sprint planning meeting where in they decide what features to address for the given sprint. “We try to be realistic about our expectations in delivering code in a sprint by applying quality criteria—i.e. we expect code to be of a certain quality or else we cannot deploy it,” says Kulkarni.

    Then developers bundle various modules on a single branch, which they test for functionality, security, performance, and integration. As the modules and the branch increase in stability, the developers upgrade them from one stack to the next until everything is deemed fit for a production deployment, explains Kulkarni.

    Results

    “Often we see proactive notifications from developers about issues they see in production. Detecting problems before our customers see them is a key benefit of the DevOps model,” says Kulkarni.

  • Service virtualization – The hows and the whys.

    Service virtualization – The hows and the whys.

    Close your eyes and take a deep breath. You are on an island, smelling the iodine coming from the sea, the sun is gently caressing your cheeks and you are sipping on a cocktail in the middle of the afternoon. Your phone is off, in a place you forgot and you do not have direct access to the internet. You are a sysdamin and for the first time since the beginning of your career, you take a break, an actual break to reload your batteries and you are not even thinking about work because you KNOW. You know it is all going to be alright, as it has been since you implemented service virtualization in a really DevOps oriented way.

    What is service virtualization?

    First of all, let me say that service virtualization is not something new but many are still unaware of what it actually is and only a few good products are available.

    You can find a really good explanation of what it is in this post from Chris Riley: https://devopsy.kinsta.cloud/blogs/service-virtualization-window-advanced-devops/ but I will try to sum it up.

    In order to avoid confusion, it is also necessary to briefly explain mocks and stubs. Mocks are usually programmers written tests that check the behaviour of a specific piece of code like which methods are called, in which order and if sub processes were created. Stubs on the other hand check that for any given input, the expected result is outed. I often refer to them as logic tests and I/O tests.

    Service virtualization has nothing to do with mocks or stubs. It mimics the entire behaviour of an application as if it was an actual one deployed on a server and connected to a database, backend etc. In order to enable this, data will be recorded from QA (and anonymized) and the whole behaviour of the application will be recorded. Going through security layers, calls to a database, updating some values, even timings can be recorded for performance testing. Absolutely every single operation is stored and if the test data is well chosen at the time of recording, all cases can be covered.

    For those of you who are into gaming, think of it as an emulator. You do not need a SNES, Genesis or Neo Geo to play your favourite oldies, instead, all you need is an emulator software and the ROMS. The emulator will behave like a gaming console on your computer, reproducing inputs and outputs with high fidelity and translating CPU instructions into different ones your computer can understand.

    If you are not into gaming, and that is a shame if you ask me, think about virtual machines. When you install a Fedora in a confined environment on top of your favourite edition of Windows (or Mac OS), what you do is pretty similar to service virtualization as well. Your Guest OS will send instructions to the different components of your computer which the virtual machine will catch, translate into commands your Host OS can understand and send to the concerned components. It sounds complicated but it works like a charm.

    What it can do for you.

    As it operates on local servers or even better, on containers (check out Docker if you have not done so yet), it is virtually always on. Downtime is a thing from the past and updates can be scheduled and performed when people are not at work. How much of your project you want to cover is up to you and your needs but once you have started the process, why not go all the automated way? If you do, that means that your programmers will have access 100 percent of the time to environments that are up, running and that reflect actual services.

    What is the point of having test environments then you might ask. And here is the answer you can give stakeholders: none. You can achieve good environment resilience, actual data tests and higher-than-ever response time through service virtualization. If well applied, and I really want to emphasize on that, service virtualization makes good ol’ test-environments obsolete.

    Once again here, the benefits mentionned by Chris Riley will speak for themselves: https://devopsy.kinsta.cloud/blogs/service-virtualization-window-advanced-devops/

    Taking DevOps one step further.

    Like all great things, service virtualization comes at a cost. In order to fine tune it to your requirements, it is going to take great efforts. You have to decide which services, ideally all of them, you will provide your programmers, which data sets are representatives of a functional application and of course, install and configure it all. The biggest benefit that you can identify here already is that developers, testers and operations have to be involved and work in a close collaboration. This is your opportunity to make different teams understand what others do. Get them all together, make them work together in a form they are not necessarily familiar with: cross-teams brainstorming.

    Brainstorming, if well done, can be a very powerful tool to come up with creative solutions to problems that were not identified as such beforehand. It has to be directed though. To brainstorm doesn’t mean talking nonsense about non topic related things. Having a neutral person in the room who focuses the discussion around a few pre-established points and who keeps track of time is the way to go. Scrum masters are generally good at this since they have an overview of the project and its problems and can quickly identify the right persons for a task.

    Getting developers to update their own service virtualization server with the latest data from QA means better communication with testers. Operations have a critical role here too since they setup the environments and give the how-to to the different teams.

    Another glass sir?

    Once your teams work together on an almost 24/7 available server and understand how it works, all you have to do as a sysadmin, is to relax a bit for once and focus your brain and time on how to improve processes even more, enhance the quality of deliverables and tighten up the now-existing bonds with your colleagues. Oh yeah, and if you’re on holidays, stop worrying about that potential catastrophic phone call, just worry about yours and yourself.

  • In Support of DevOps: Kanban vs. Scrum

    In Support of DevOps: Kanban vs. Scrum

    The Kanban software development methodology is getting a lot of attention as of late, particularly for its ability to enable DevOps. Some organizations are even going so far as to move from Scrum to Kanban to improve efficiencies. We use Scrum at OutSystems, since we’re doing feature development that requires stakeholder feedback. The sprint lock and end-of-sprint demos to stakeholders are invaluable to us. But I was curious to see how the two methodologies differ and how Kanban may further enable an organization’s move to DevOps. Here’s what I found.

    There are two key differences between Scrum and Kanban: the rules and the workflow.

    The Rules

    Both software development methodologies provide rules for how work is performed. If your organization uses Scrum, then you know that it is quite prescriptive. Agile Advice lists 23 mandatory and 12 optional rules for implementing Scrum. These include, for example:

    • A product owner creates and manages a product backlog
    • Teams are cross functional
    • Interruptions are prohibited during sprints
    • Work is time boxed
    • Scrum meetings are held daily
    • A burn-down chart is used to measure progress

    Taken together, these rules create a fairly rigid box in which teams must operate in order to be successful with Scrum. As I see it, this presents two big challenges. First, few organizations actually follow all of the rules, leading to what is called ScrumBut – We practice Scrum, but… ScrumBut means a company ignores specific Scrum rules for internal reasons, leading to a suboptimal use of the framework.

    Second, at the core of Scrum is the time box. Time boxes are good for providing distraction-free time for developers to deliver the product, and define regular milestones for stakeholders to evaluate and steer the project. But, from a DevOps point of view, these artificial delivery checkpoints break the flow. Coordinating dependencies between sprints and ensuring their successful move to production becomes a big challenge.

    Kanban, on the other hand, is less restrictive. Its rules are as follows:

    • Visualize the workflow
    • Limit work in progress

     With only two rules, Kanban is an open and flexible methodology that can be easily adapted to any environment. (Some companies are using it across the entire organization – from manufacturing to marketing.) You can add rules as needed, borrowing from Scrum, if you so wish.

    The focus on the flow instead of time boxes makes Kanban an attractive choice to use with DevOps. Deploying work items becomes just another step in the process, and with its emphasis on optimizing the whole delivery rather than just the development process, Kanban and DevOps seem like a match made in heaven. With the contrast in rules understood, let’s look at the workflow.

    The Workflow

    In addition to rules, the workflow differs in Kanban vs. Scrum. This is a direct consequence of the rule sets. In Scrum, you decide beforehand what features will be completed in the next sprint. The sprint is then locked, the work is done over the course of a couple weeks (the usual sprint duration), and the queue is empty at the end of the sprint. Locking the sprint ensures the team has the necessary time to work on a problem without being interrupted with other apparently “urgent” requirements, while the feedback sessions at the end of the sprints ensure stakeholders approve the delivered work and steer the project as the business changes.

    In Kanban, you do not have the time constraints of a sprint, and the focus is on making sure the work keeps flowing with no known defects moving downstream. However, limits are placed on the size of the queues. This means that a maximum number of features or issues can be worked on at a given time, allowing the team to focus on delivering those work items with high quality. Making the flow visible creates a sense of urgency to keep things moving along. Note that Kanban was born in manufacturing, and its big focus is on productivity and efficiency. Kanban has to be extended in order to incorporate fundamental aspects of software success, such as stakeholder participation.

    Enter DevOps

    Organizations that adopt DevOps are looking to increase efficiencies, deploy features more frequently and be more responsive to business demands. You can see, then, how each methodology addresses various aspects of DevOps to greater and lesser degrees. Choosing one over the other isn’t cut and dried. Here’s what I advise:

    If your team is responsible for feature development, which requires stakeholder feedback, and developers need to focus, Scrum is probably a better fit. The sprint lock and end-of-sprint demos to stakeholders can be invaluable.

    If your team is responsible for maintenance and tends to be more reactive, then consider Kanban. There is no need to lock your backlog and you have more flexibility to respond to customer input.

    Ultimately, every team is different. You know where your team excels and where it needs extra guidance. I suggest you borrow from both methodologies to give your team just what it needs to achieve DevOps’ goals.

    About the Author

    Rodrigo CoutinhoRodrigo Coutinho is Principal Software Engineer, OutSystems. A member of the founder’s team at OutSystems, Rodrigo has a passion for web development, great products, and geeky stuff. He spends his time designing future versions of OutSystems Platform and dreaming about the cool future of the web.