devopsy.kinsta.cloud

Category: Spotlight

  • Shhhh! It’s a Secret

    Shhhh! It’s a Secret

    In one of our regular back-and-forth breaks, Lori and I were talking shop a week or two ago, and she shared an excellent report by GitGuardian with me. Frankly, none of the information in the report was a huge surprise for me–Git is filled with information that companies probably don’t want out there, but that is nearly impossible to track proactively. That’s what GitGuardian tries to do for you, but honestly, I’ve not yet dug into them, so caveat emptor. Or, read someone else’s opinion to help you decide if they’re a good idea – I’m just going to talk about the report.

    The key takeaway is that you are asking your DevOps teams to move faster and faster. Putting a section of code they are working on into a public repository is an easy way to “take work home” or share it, especially during lockdowns. It is also a good way to hand over critical information to anyone searching for it—about your infrastructure, or even critical bits of data like SSL keys or credentials. And with the current level of cyberattacks against nearly every industry, I’d say there are people looking for that information, no matter what industry you are in.

    This is not really a DevOps-specific issue–credentials and such in code has been a problem for IT for as long as I’ve been working in the field. At some point, any automated system needs to perform AAA, and that means the data required is stored somewhere. A multitude of approaches have been attempted to resolve the problem technologically, but none have been a silver bullet. Yet. Eventually, someone will figure out a way to make it work; then a whole new attack vector will arrive, probably aimed at the tool that is settled on to protect this very data.

    This is a personnel/policy issue that is exacerbated by the speed of Agile and DevOps. Your best solution, at this time, is to write a clear policy that contains consequences for putting secret data into a public repository. We’re short on “consequences” right now, preferring to say, “Right, let’s make sure that can’t happen again,” but in the case of putting secrets out in public, it is a different level of ‘bad’ than a coding error getting checked in. Those secrets can actively be used against the organization, and if the not-to-be-shared data is clearly delineated, any violation can be viewed as intentional, be it because of convenience or malfeasance.

    According to the report, there are millions of bits of privileged information out there. Not all of them are from enterprise IT shops—but some are. And some might be from your organization. Set expectations, get feedback. Make it clear you do not want to stop progress on delivery of systems, but you also don’t want to implement a system with a built-in back door for attackers to use. It is easy enough to use alternative delivery methods for those few bits of data, and put the bulk of the source out where it is needed.

    And keep rocking it. Toward Data Science says there are 128 million public repositories out there (Q1, 2021). Think how many lines of code are in those repositories, and how many secrets potentially are, too. Just make certain yours are not included in the list. No sense keeping your next great solution from running because your critical data leaked.

  • Transition to DevOps Increasing Across IT Firms

    Transition to DevOps Increasing Across IT Firms

    Demand for faster product development and quick market release created a need for increased collaboration between Development (Dev) and Operations (Ops) teams, which eventually led to “DevOps.” With this, transition to DevOps has become one of the key points of discussion across many IT firms. This article gives you an overview of the demand and trend for organizations to transition to DevOps.

    Rise in IT innovation and burgeoning digital revolution have led to the demand for faster development and quicker releases to stand out in the IT competitive scenario. This transition eventually led to a new culture that saw collaboration between two key IT teams, development and operations teams, which the industry calls DevOps.

    DevOps established a new style of IT functioning, easing different phases of software process from product development to testing and final market release, all in a rapid and reliable way.

    Why DevOps Transition?

    Rapid delivery of independent software components and continuous integration of the same with production systems forms the key principle behind DevOps functioning.

    DevOps is often confused with Agile methodology. Though they share similarities in providing improved IT performance through rapid development, DevOps stands apart from Agile because of its efficacy in the software deployment phase, ensuring a constant communication between the two teams. Thus, DevOps is often termed as a successor to Agile.

    A collaborative approach, key for information exchange across the line, is one of the major factors driving firms to undergo DevOps implementation.

    Though DevOps is well-known as a merger of development and operations teams, an effective DevOps model extends beyond that to other key functional areas of organizations.

    Breaking the old trend of limiting individual teams to their respective work domains, DevOps brought in a scope for knowledge-sharing among them, wherein developers can extend themselves to testing and deployment and operations team members can upgrade their skill set, learning new languages, architectures and tools.

    Increased Demand for DevOps Resource

    Transition to DevOps holds a key demand in the fast-growing IT market, wherein managing with the existing infrastructure or resource plays an almost equal role as coming up with something new.

    Greenfield projects aside, DevOps deployment is most often seen in brownfield projects that encourage building on the existing resources.

    With this situation in place, there is also a lot of demand for DevOps resources across firms. The “2017 Open Source Jobs Report” from the Linux Foundation shows 60 percent demand for DevOps human resources among more than 2,000 IT open source professionals and IT hiring managers.

    This clearly shows the level of demand that transition to DevOps is holding in the IT industry.

    “If there isn’t an existing DevOps role within your company, software developers should spend time sitting with the operations team or vice versa to understand how their processes work and how they can be improved through collaboration,” said Mike Durney, CEO of the job-hunting site Dice.

    The report found:

    • Nearly 50 percent of hiring managers are keen about those DevOps resources with professional credentials such as relevant certifications
    • More than 47 percent of the companies are ready to fund certification programs for their existing personnel
    • 42 percent of companies are keen on adding DevOps skills to their hiring portfolio
    • “DevOps Engineer” profile ranks second in Glassdoor’s Best 50 jobs in America

    Despite its advantages, however, DevOps has a few challenges to overcome related to application integration, test automation, budget and skill set, among others.

    — Veritis

  • The Executive’s Guide to Microservices Chapter 3: Choosing A Starting Point

    The Executive’s Guide to Microservices Chapter 3: Choosing A Starting Point

    In the previous chapter, Steve had been asked to provide a comparison of costs and resources needed to migrate to a commercial system versus modernize the current system. He works with Stacey, head of development, who comes up with a modernization plan that includes certain changes that the team will implement to limit downtime.

    Steve has discussed Stacey’s plan with the business, which agreed with the approach toward modernization of the supply-chain system. The business particularly likes the idea that in the future the reduced fragility of the system might mean its IT teams can introduce ongoing improvements even through the holiday season, rather than having to freeze all systems development for a period of four months.

    Now Steve and team are challenged with where to start.

    The team has decided that any changes will require decomposition of the existing monolithic design. The approach to date had been to address requirements functionally, which meant that every time the business requested changes to the system, the code was changed to support those requirements by adding specialized logic to various modules.

    Unfortunately, this approach has resulted in significant amount of technical debt. Each compounded change introduced conditional logic to support prior requirements. Moreover, each new change had the potential to break prior changes, so time-consuming impact analyses had to be done before any changes could be made.

    Steve and team want to decompose the system into a set of microservices that closely resemble the business function they support. Moreover, they want to segregate the specialized business logic into modules that consume the generic business services.

    Obviously, this is a long-term strategy, so Steve and team need to choose how to approach incrementally to enable them to make major strides before this year’s systems freeze. Given the proximity to the holiday buying season and the availability of resources, Steve and team will need to choose wisely which microservices they will create. Moreover, Steve and team know they will be limited by the level of entanglement within the existing code.

    After an initial analysis Stacey’s team determines that the costliest issues stem from the logic to support multichannel customer engagement. That part of the system integrates custom logic with some off-the-shelf ecommerce products. During the holiday season, customer support calls increase in volume and each call can last five to 50 minutes. The large variation is often caused by the inability of the system to address changes in a particular channel post order-placement. For example, if the customer wants to add a gift receipt or wrapping after the order is placed, it must be dealt with out of band. This means that the business has had to add support people during the holiday season just to deal with these issues.

    Stacey and team also believe they can easily isolate the logic for multichannel support within the existing code base and decompose into a set of microservices. The team decide that the existing customer records system must stay as is for the time being, but they can create a new API to more easily access that data from the new microservices. Stacey calls these macroservices.

    The team will create a set of microservices to support the web channel and a set to support the distributor channel, since these each have very different requirements. The current system combines the logic to support both of these in one module today. Since microservices are supposed to be focused on doing one thing well, the team have decided on the following bounded contexts: order creation, order execution (payment), order fulfillment and returns.

    Steve asks Stacey, “Why not just have an order service?”

    Stacey replies, “That is one approach, but we would call that a miniservice as it manages all aspects of orders. However, our goal is to allow parts of the system to change over time with minimal testing and maintenance required. A microservice change should not have broad system implications. Changing the order miniservice could mean that an entire channel could be impacted. With this approach we could change any of these supporting order processes without impacting the entire order function within the company.”

    Steve thanks Stacey for her answer and tells her he has a much better understanding now for what the team is trying to achieve.

    — JP Morgenthal

  • The Executive’s Guide to Microservices: Chapter 1

    The Executive’s Guide to Microservices: Chapter 1

    Meet Steve. Steve is responsible for a mission-critical application for a very large enterprise. He has a problem: It takes Steve and his team a very long time to make changes to this application and release it to production. Steve has been talking to vendors, consultants and peers about agile software development techniques and DevOps, but Steve and his team have assessed that his application does not lend itself to these approaches.

    See, the system Steve looks after is called a monolith, which is a very confusing title since it was developed using the latest modular software development approaches when it was created, in the latter half of the 1990s and early 2000s. Steve and his team are constantly advised that they need to decompose their monolith into microservices. However, this advice only confuses Steve and his team even more, since the system was designed using a service oriented architecture (SOA) with the intent of ensuring that it was easier to maintain and scale.

    Steve’s team fell into a very common trap associated with many SOA-based designs: The services were designed with deep dependencies, they share a common database and they are very coarse-grained. Indeed, the collections service, which is one of eight services in the system, handles every potential operation a collections agent might ever need. Modifying any operation within the collections service means the entire collections service must go through regression testing and production must enact a very cumbersome deployment strategy that requires a five-hour maintenance window.

    Needless to say, Steve cannot keep up with the requested changes from the business with these constraints. Steve’s advisers have given him some good advice on organizing to develop faster, but he would only be applying Band-Aids faster. However, once the constraints identified above had been pointed out to Steve and his team, they better understood the challenge ahead of them.

    Steve and team are going to have to redefine the data models so they support service independence, then extract subsets of functionality out of the large, intertwined services to deploy as within a well-defined business context—following the “do one thing well” mantra. They then can start to extricate the complex business logic trapped inside the enterprise service bus so it can be incorporated into the services at the edge.

    Steve and his team realize that continuing to enhance the monolithic code base avoids the challenges of redevelopment but reduces the value of that codebase to the business. While there will be a cost of untangling the monolith, the cost of not doing so is lack of competitiveness, lack of agility, long wait times for new features, possible opportunity loss and continued investment in technical debt. Steve’s job will be to present the justification for this to the business based on these risks.

    When the task is completed, Steve and team will be able to deploy new services and modify existing services within weeks. The costs related to development and testing will drop significantly and the finer-grained services will allow developers to address business needs more quickly.

    We’ll check in with Steve and his team as they embark on this journey and deal with the challenges ahead.

    — JP Morgenthal

  • Welcome to Microservices Journal

    Welcome to Microservices Journal

    Welcome to Microservices Journal, a community dedicated to the design of microservices-based applications and related technologies. The introduction of microservices into the digital application landscape enables a new level of agility for businesses that adopt this model of design. As with any architectural construct, there are no absolutes; hence, no wrong or right way to build microservices-based applications, however, there are some well-accepted tenets, patterns and anti-patterns associated with this design approach. Microservices Journal is being established as a site where practitioners can share their points-of-view and their experiences.

    Following in line with emerging agile development approaches, we are introducing Microservices Journal as a minimal viable product and will iterate rapidly to add new features and content. As with any community, we will evolve and thrive with contributions. These contributions can come in the form of blogs, articles, ebooks, podcasts and videos. Soon, we will add discussion forums, deliver webinars and hold live Q&A sessions with renowned experts in the field.

    In addition to the website, we are also creating Microservices Day events to be held regionally. Our first event will be in Austin on November 16, 2017 as part of the Innotech Austin event.

    Like any community, we’re only as good as our contributors and contributions, and we look forward to your contributions.

    If you wish to contribute, please email editor@microservicesjournal.com

    — JP Morgenthal