Category: Enterprise DevOps

  • What Developers Must Know About Threat Modeling

    What Developers Must Know About Threat Modeling

    Threat modeling is a process that far few developers seem to pursue, but it is a process that helps you and your team to model all potential threats to your application. Essentially, threat modeling is your thinking through all of the potential threats against an application. Doing so is virtually as easy as putting together a list of threats in a structured manner that lets you assess any perceived risks, allowing you to formulate how you are going to respond to the threats.

    Ultimately, through threat modeling, you are optimizing network security by identifying objectives and vulnerabilities and creating measures to prevent the effects of threats to the system. Through this definition, anything described as a threat has the potential to be malicious to your organization and data systems; for example, incidental occurrences like the failure of a storage device–which can compromise the integrity of the entire enterprise.

    The reasons for threat monitoring may be pretty obvious, but any lack of progression here is quite common.

    Threats are one of the significant factors that led companies to spend $89.1 billion on enterprise security solutions. Gartner predicts threats will reach $96.3 billion by the end of 2018. According to Raytheon’s 2018 Study on Global Megatrends in Cybersecurity, 36% of senior IT and cyber professionals identified malicious or criminal insiders as a top cyber threat. Even with that knowledge, many organizations lack insider threat policies and policy enforcement tools, and they struggle to align security and IT responsibilities to tackle the threat effectively.

    Threat monitoring allows you the ability to determine which threats might exist for your application. Knowledge is power, and it’s always better to know beforehand what dangers might await you. Understanding these threats lets you decide whether you will accept the risk.

    For a threat to exist, there must be a combination of the following where the combined likelihood and impact are significant enough in which to do something. A framework for understanding threat modeling, includes: asking what you are working on, what might go wrong and what to do about it, as well as if a job is well done.

    According to the Open Web Application Security Project (OWASP), threat modeling is a process that lets you continually refine your processes based on what you have done so far: “Starting with all possible vulnerabilities is usually pointless, as most of them are not attackable by the threat agents, protected by a safeguard, or do not lead to a consequence. Therefore, it’s generally best to start with the factors that make a lot of difference.”

    Those steps include assessment scope, identity threat agents, existing countermeasures, identifying vulnerabilities, prioritizing identified risks and identifying countermeasures to reduce the threat.

    In the assessment scope, you’ve got to understand the price of what’s at stake. Identifying tangible assets, such as databases of information or sensitive files, is usually straightforward, and realizes the capabilities provided by the application.

    Next, identify possible threats or attacks. A crucial part of the threat model is characterizing different groups of people who might be able to attack your application–this may include insiders and outsiders, performing both inadvertent mistakes and malicious attacks. During this phase of development, consider existing countermeasures that allow you to determine what possible exploitable vulnerabilities exist that must be addressed. Through your search, look for weaknesses and connect any attacks you’ve identified to the negative consequences you’ve identified.

    When identified, risk management can be prioritized. For each threat, determine possible outcomes and the impact of those to understand how to mitigate these issues. Next, look for ways to reduce any future risks.

    While many of the threats might appear to be security related, many others have more natural explanations. Threat assessments can include power outages or even a flooded server room. These and other factors can lead to severe threats to your organization.

    When to Model the Threat?

    Threat modeling is often best at the beginning of a project, but it’s better to conduct a threat modeling project halfway through a project or even at the end than not at all. Threat modeling at the end of a project can have benefits, such as understanding the architecture and how data flows through it. However, threat monitoring at the end of a project can lead to finding more work may be required to fix poor design decisions not caught at the beginning.

    For those entering into threat modeling, conduct your first session on an existing project, letting you experiment with a threat modeling session so that you can familiarize yourself with everything required. This way, you’ll be better prepared to analyze all assets and flow of data to assess the risks. Once the process is pounded out, you’ll have a better chance of succeeding in your threat modeling of a new project. You’ll also be able to better prepare for threats that may exist in the design phase of the project. From here, over time, you’ll begin to take into account threats during the initial design of a new feature.

    Approaching the Threat Model

    Each method of approach depends on the organization and its goals. Large organizations usually have previously established processes for threat modeling. Smaller organizations new to threat modeling can begin by hiring in experience to initiate the task or seeking outside support from experts or those who can advise the program.

    Next, you need to understand who the users are within the system–regular users, administrators, outsourced contractors and perhaps hackers with apps or other tools looking for vulnerabilities within your network. There are also other actors, including disgruntled former employees.

    Defining the user actors is important since missing a group can mean you might be missing an entire category of threats. Also, consider the various types of hackers or groups of hackers in your models.

    Data and Information Flows

    At this point in the threat modeling process, track each of the various data flows running through the system. For each of these, you need to know where the data goes. What does it interact with or encounter along the way? Then look to identify where data can leak or where it might be exposed. More data components create more opportunities for a hacker to gain access to the information.

    Individual data means the information flows differently based on the architecture and the specific situation. One example might include a request from a user’s browser where the cookie is sent through and interacts multiple components as it does.

    For each data flow, search opportunities for any actors to gain hold of valuable information. If anything makes you frown, step back through the process and move that threat through until you can mitigate the frowning moment.

    Threat List Complete

    When the threat list is observed and complete, prioritize the list from very unlikely to improbable to not likely. No matter the probability of the risk, work through the risk as a real possibility. Do so to consider the real impact of the threat and what it may mean to the organization. Is the company going to crash and burn completely? Will every bit of data get pulled into a ransom situation? Place a number or priority of the impact, such as low, medium and high.

    When you have your risk list, manage the risks by:

    1. Accepting: If the risk is quite low, accept the potential impact. It’s also possible you’ll find the potential negative user impact is more important. You can also reassess later.
    2. Transferring: Perhaps another team is responsible for managing a particular risk. If that team can pick up the defense easily, outsource it.
    3. Avoiding: Don’t do this. If you must, consider complete structural changes to your data flows to avoid altogether avoiding risk. This might mean architectural changes are required or killing a particular feature altogether so as not to avoid danger.
    4. Reducing: Take actions to lower the risk by lowering the likelihood or impact of the risk. You may need to take multiple steps to reduce the threat. No matter the action, effort is required to reduce the risk.

    After threat modeling is complete, don’t stop. Keep working because threats and actors change. List all potential issues or risks that must be addressed and work them into your plan.

    Finally, remember you should be careful with whom you share your threat modeling plan. You never know where that information might end up otherwise, or who might be a threat to your organization and its data.

    One last note: The final result and usefulness of threat modeling depend almost entirely on the parameters established. Defining the values for the probability of a threat and impact may be subjective, but let your experience, input from colleagues and experts, and your gut guide you.

    — Jeroen Boks

  • Optimism Matters and so Does a Sense of Wonder

    Optimism Matters and so Does a Sense of Wonder

    I’m a terrible video game player. For as much fun as people have with me at the table for Table Top RPGs (TTRPGs), I’m a pain when playing video games.

    As a guy with a masters in computer science, and years of experience ranging from embedded drivers to enterprise architecture, I see bugs or bad design decisions in video games and they tweak my sense of right and just. Ask my wife and youngest about playing Fortnite with me, you’ll get an earful. My wife uses her masters to analyze what could have happened in the code, and why it’s understandable. I fume and say “Yeah, well the code–not me, not the other guy–just eliminated me.”

    In short, I’m jaded where video games are concerned. As a casual gamer, I see the bad even while the good all around is rocking–EPIC devs, you’re doing a great job, particularly with the rate of change the game sees. Gamer rage is about the occasional oddity like getting stuck waist deep in the snow. Or about things beyond your control, like Skill Based Matchmaking that isn’t.

    I’ve noticed a trend in the portions of DevOps that I keep an eye on lately–fatigue. A sense that the change is non-stop, and the volume will just keep increasing. It only makes sense, we’re knee deep in sea change and new terms, products and processes seem to crop up on a daily basis. It can be wearing. Just looking at requirements for jobs–there is the persistent joke about 2000 years’ experience in Jenkins for a reason–it shows a fractured and constantly changing environment that begets unrealistic expectations.

    This has a tendency to make people jaded about new technologies and techniques. After all, Agile and DevOps have brought a huge return on investment, the law of diminishing returns might be kicking in, right? And lots of new shiny have turned out to be smoke and mirrors.

    The thing is, the rate of change in DevOps is still moving so fast because there is still more to gain. Continuous optimization, aka AIOps, is cool and promises a lot. Automated testing is still ramping up, even though automation of build tests is solid in most environments. Other kinds of testing (security, regression, live and even A/B) are still lagging, and test automation is offering help.

    What we need to do is try to keep that sense of hopefulness that we embarked on DevOps with. New tools might give us an appreciable ROI. They might free up time for time-constrained teams, which is the best ROI you can hope for. New processes might address shortcomings in existing ones. Will new tools and techniques always pan out? No, your environment is unique so even popular tools or techniques might not be best fit for your particular organization/apps. That’s okay, keep poking around at what’s new and remember the wins.

    I’ve got like a 4% win rate in Fortnite. Those other 96 games out of a hundred? All learning experiences to (hopefully) increase my win rate. Same with evaluating new tools. The failures feed the successes. It’s far easier to go “Product X didn’t work for us, and this is very much like it. Let’s look closely before even trying a pilot.”

    While there are more people in IT these days by conscious career choice, the bulk of us are still here because of the wonder moments–first time you got code to compile, spun up a cloud instance, figured out network segmentation or divided physical disk into partitions. The same sense of new and exciting should be brought to the current set of changing technologies, while still rock-starring your job. After all, it is some pretty astounding stuff we’re getting to play with every day.

    Maybe I’ll take my own advice, chill a bit when playing video games. After all, that’s another space that has jumped light-years in quality and availability.

    — Don Macvittie

  • From Automated Cloud Deployment to Progressive Delivery

    From Automated Cloud Deployment to Progressive Delivery

    Your team is on its agile journey and you can more or less track a deployment from a story, through git commits, to an automated builder, to an artifact repository, into a container and then onto the cloud. Or, in fact, any other of the many valid variants that lead you to believe your deployments are largely automated. What you can point to is a business request comes in and the result is a service or a new app. Your purview seems to stop before your users can even respond though. Is that good enough?

    Let’s go back to the early idea of a release. It was a collection of all the features and fixes the loudest stakeholders had persuaded the product owner to put at the top of the story list. A date was promised, but QA was late, and the persistent stakeholders squeezed a little more juice out of a story. Then it was stuffed into a ball and delivered to your servers overnight, lest anyone notice. The next day, the developers had to deal with the fallout as the support queues grew with confused users.

    While developers were getting the hang of the agile notion of automated deployment, the release lifecycle did pretty much end with a deployment going live. This gave certainty that what had been built and placed in the artifact repository was the same thing the testers had seen in the staging environment. And it also had the changes Jira said it had.

    This type of release was very much like a birthday present. Your uncle talked to your dad briefly, without your knowledge, and agreed you really would benefit from a new pair of socks. Then these were delivered on the day with the receipt in the bag, just in case you needed to take them back.

    The first evolution came with the concept that a release wasn’t exactly synonymous with a deployment. Finally, the end-user’s perspective nudged its way into corporate release strategy–updates could be deployed without necessarily impinging directly on every user. What if there were two identical release environments with clever network switching, meaning only one was truly live? This type of flip-flop arrangement (sometimes referred to as blue/green deployment) allowed changes to be tested internally in a realistic environment before going live.

    It was the advertising industry that prompted the idea of A/B testing. Instead of guessing one variation of a campaign might be superior to another, the surprisingly scientific method of using a controlled sample and a variation was used on audiences to see which they preferred. In the digital age, it is possible to deploy two variations of the same release to different server sets. This might mean each user session could be using either release. It would then be necessary to associate the resulting click-throughs, or whatever success measurement is chosen, to the specific release flavor. The worth of this kind of experimentation was only as good as the question you asked, but it was at least observing real user interaction.

    Moving on, DevOps environments began to use more feature flags or toggles. These were code paths that could deliver feature changes in live servers, controlled through configuration changes. This reduced the need to re-deploy to make certain changes. Release minded teams were coalescing around configuration, as opposed to just code. Whereas code once compiled and built was locked away in unreadable artifact files–a larger stakeholder community could administer configuration files, usually encoded in English. This helped to keep the feature changes closer to the stakeholders and less likely to slip back into silos.

    As a more defined understanding of delivery arrived, so did the understanding of the user community. It used to be the case that a user was a one-line entry in a database. Even firms whose business is not mining their user’s data understand there are different sets of users for products and services.

    Traditionally, there has always been internal users or beta testers–those within the firm whose job is to check the bits they understand, or are responsible for, are working as expected. Then outside the firewall, there are the tech savvy community who actively want the latest updates. These guys report bugs and often compare your product with the competition, perhaps using both. This audience will not lose their marbles if you deliver a bug to them. More to the point, they will notice rapidly–perhaps telling everyone on social media–about your failings.

    The disengaged users who may only have the free version or tier of your product come next. These people are more likely to work on impressions and are best not disappointed. They are also unlikely to be keen on updates. Finally, the large core of solid users who pay their way, just want things to work and need to understand a path to continuity will already appreciate the software and will understand the roadmap by osmosis. These will probably include your biggest users, too. Of course, if you have no way of distinguishing your users all the above is moot.

    With services sitting on a public cloud and CDN like edge services such as CloudFront, the ability to release by territory is much more straightforward. This is also increasingly necessary for legal reasons–EU regulations, for example. This allows for tactical releases to communities in different time zones and sizes. The practice of deploying changed services to a small group for risk mitigation, or “canarying,” is another method that takes advantage of smart routing. This requires the ability to quickly observe and rollback releases though.

    Today, most software has some degree of dark launch that restricts the visibility of a release to the appropriate community, until all is well. In this sense, a progressive delivery (defined as “continuous delivery with fine-grained control over the blast radius” by James Governor) can be seen as the virus-like spreading of your software or services from your development teams’ laptops to the last user.

    Observing service use is simply part of the core concern of virtually every company in the tech space. From this perspective, progressive delivery changes the way to see your development team as mere builders to user fulfillment specialists, which of course, they always were.

    The future will undoubtedly involve increasingly customer-driven releases, which goes hand in hand with the increasing use of data science to study user communities and their exhaust. The important take away is to make sure your team is reaching further to the right-hand side of the product journey, and keep the user’s reaction to what you make as a larger part of your sensory input. Think more about constant small changes and less about ceremonial releases.

    — David Eastman

  • How the World Measures DevOps Quality

    How the World Measures DevOps Quality

    With continuous everything, knowing whether each new release will ultimately enhance or undermine the overall user experience is essential. Yet, most of today’s go/no-go decision still hinge upon quality metrics designed for a different era.

    Every other aspect of application delivery has been scrutinized and optimized for DevOps. Why not re-examine quality metrics as well?

    Are classic metrics like number of automated tests, test case coverage and pass/fail rate important in the context of DevOps, where the goal is immediate insight into whether a given release candidate has an acceptable level of risk? What other metrics can help us ensure the steady stream of updates are truly fit for production?

    To provide the DevOps community an objective perspective on what quality metrics are most important for DevOps success, Tricentis commissioned Forrester to research the topic. The results are published in the 55-page report, Forrester Research on DevOps Quality Metrics that Matter: 75 Common Metrics—Ranked by Industry Experts. The report takes a deep dive into the global findings, complete with heat maps, quadrant mappings and some fun lists such as “Most Overrated,” “Hidden Gems” and “Top DevOps Differentiators.”

    One of the most common questions we received after publishing the report was: “How do the results vary across regions?” In response, we performed some additional regional analysis—and I’d like to share those results here.

    To start, let’s take a look at the global top 20. The following metrics were ranked as the most valuable by the DevOps experts who measure them (across all regions).

    Europe DevOps Quality Metrics Trends

    Looking specifically at Europe, the top 20 changes as follows:

    Interesting trends in this region:

    • There is a greater commitment to measuring quality metrics. European respondents reported a higher level of DevOps quality metrics measurement across the board. For almost all metrics, the usage rate was at least 6% higher than the global average. For metrics related to time, coverage, risk, effectiveness and efficiency, the usage rate was over 14% higher. This speaks to European organizations’ commitment to scrutinizing and continuously optimizing their quality processes—especially in terms of time and resource utilization.
    • Risk and coverage metrics are valued more than in the global average. European respondents also ranked risk and coverage quality metrics a surprising 21% higher than the global average. This could be related to the fact that the respondents from this region came primarily from the financial services and insurance sector, with healthcare and government close behind. In such highly-regulated industries, measuring and mitigating risk is certainly a core concern. This finding could also indicate European organizations place a greater emphasis on protecting the corporate brand.
    • Test data preparation time seems to be a greater concern. European respondents were more likely to measure (+18%) and highly-value (+23) time spent preparing test data than their global peers. Given the restrictions GDPR placed on test data as of May 2018, it seems likely that European organizations have significantly changed their test data management processes (e.g., to masking and more synthetic test data generation), and are cautiously monitoring how the changes are impacting their overall efficiency.

    Asia Pacific DevOps Quality Metrics Trends

    Now, let’s shift focus to Asia Pacific. The following 20 metrics were ranked as the most valuable by the Asia Pacific DevOps experts who measure them:

    Notable trends in this region:

    • End-to-end testing metrics are valued—and measured—more than in the global average. Although Asia Pacific respondents measured fewer build and functional validation metrics than the global average, they measured (and valued) end-to-end testing metrics much more than their peers around the world. For example, percent of automated end-to-end tests was measured by 47% of the organizations (versus 36% globally) and highly-valued by 84% (versus 70% globally). Risk coverage measurement was significantly higher; it was measured by 49% (versus 34%) and highly-valued by 71% (versus 59%). This speaks to the region’s focus on digital transformation and commitment to delivering exceptional user experiences.
    • API testing metrics were also valued—and measured—more than in the global average. Asia Pacific respondents also measured and valued API testing quality metrics more than the global average. Overall, API quality metrics were measured by 16% more organizations in this region than globally. The highest valued API quality metrics were API test coverage (63% versus 39% globally) and API risk coverage (79% versus 62% globally). This prioritization of API testing is likely a side effect of the regional trend towards API-driven open banking (the majority of respondents indicated they were in the financial services and insurance sector).
    • There is a greater leader/laggard quality metrics measurement gap. Part of the study involved classifying the respondents as either DevOps leaders or DevOps laggards, based on their responses to various questions about the maturity of their processes. Although the percentage of DevOps leaders in the region was lower than the global average (18% versus 26%), the DevOps leaders from Asia Pacific generally measured quality metrics at a comparable rate to their global peers. However, the DevOps laggards in Asia Pacific generally measured quality metrics at a much lower rate than their global peers. This suggests the select set of firms that have truly prioritized DevOps initiatives have made great strides—and the laggards have a lot of catching up to do in order to remain competitive.

    Research Methodology

    Here’s a look at the process behind the research discussed above:

    1. Survey 603 global enterprise leaders responsible for their firms’ DevOps strategies.
    2. From that sample, identify the firms with mature and successful DevOps adoptions (157 met Forrester’s criteria for this distinction).
    3. Learn what quality metrics those experts actually measure, and how valuable they rate each metric that they regularly measure.
    4. Use those findings to rate and rank each metric’s usage (how often experts use the metric) and value (how highly experts value the metric).
    5. Compare the DevOps experts’ quality metric usage versus that of DevOps laggards. If there was a significant discrepancy, the metric is considered a DevOps differentiator.

    — Wayne Ariola

  • OpenLegacy Partners With Compuware to Advance DevOps on Mainframes

    OpenLegacy Partners With Compuware to Advance DevOps on Mainframes

    OpenLegacy has allied with Compuware to make it easier to build and debug microservices-based applications employing REST application programming interfaces (APIs) to access data residing on mainframes.

    Zeev Avidan, chief product officer for OpenLegacy, said the company’s namesake platform analyzes mainframe environments to create microservices-based APIs written in Java without requiring any changes to the legacy environment. That approach not only minimizes disruption to the mainframe environment, but it also eliminates the need to rely on enterprise service bus (ESB) platforms and legacy web services interfaces to access mainframe data, he said.

    The alliance with Compuware now makes it easier access the OpenLegacy platform from within Topaz Workbench, a set of tools from Compuware that are used to apply DevOps practices to applications that either run on a mainframe or need to access data residing on a mainframe.

    Avidan said it is becoming increasingly common for organizations to want to build and deploy applications that need to access mainframe data as part of various digital business transformation initiatives. The challenge they face is that most legacy monolithic applications running on a mainframe don’t have an API to invoke. OpenLegacy provides a framework for generating those APIs, which in turn makes it easier to incorporate mainframes within the context of a larger DevOps process. The alliance with Compuware furthers an ongoing effort by the two companies to foster the adoption of best DevOps processes in mainframe environments, he said.

    Legacy waterfall-based application development processes are still relevant in mainframe environments, he added, but it’s also clear organizations want to apply DevOps processes to applications spanning hybrid environments that include mainframes. The majority of data in many enterprise IT environments is created on a mainframe. Rather than being forced to copy that data, API platforms such as OpenLegacy make that data more accessible where it resides. Once the APIs are in place, it becomes practical for DevOps teams to create software development kits (SDKs) to access various applications running on mainframes, Avidan said.

    Organizations that have invested in mainframes generally are interested in agile development methodologies that enable them to derive more value from those investments. The challenge they face is finding a way to make that data available in a way that doesn’t disrupt the mission-critical applications running on the mainframe, he said. Over time, IT organizations running mainframes will adapt their processes to incorporate best DevOps practices, but the level of disruption they can embrace in pursuit of that goal has to be minimized, Avidan noted, adding it would be unreasonable to expect that level of cultural change to occur overnight.

    Despite ongoing predictions about the eventual demise of the mainframe, the number of workloads running on the venerable platform continues to increase, thanks in part to the ability to run Linux on a mainframe. What many organizations that opt to run Linux on a mainframe don’t always initially appreciate is the degree to which that decision also opens the door to transforming their internal IT culture, regardless of whether they are ready.

    — Mike Vizard

  • Common Cloud Security Mistakes and How to Avoid Them

    Common Cloud Security Mistakes and How to Avoid Them

    Over the last few years, it’s become apparent that traditional on-premise security policies are not a good fit for newer cloud-native environments. Even though the writing has been on the wall for a long time, the brewing security crisis still hasn’t been resolved. Far too many organizations continue to use security measures that are ineffective for cloud-native environments, along with inconsistent policy enforcement that leads to huge oversights and gaping organizational loopholes. The sad truth is that today, it doesn’t take a master hacker to infiltrate most cloud-native deployments.

    Many of the most infamous security breaches would have been extremely difficult to prevent even with an ideal security environment, but the weakness of the system is endemic, reaching far beyond a handful of vulnerable organizations. Even if some malicious actors could still get through, there’s no reason for organizations to leave their attack surface so blatantly unprotected.

    Here are increasingly common cloud security mistakes DevOps teams still make, and our suggestions for how you can stop them in your organization.

    Ignore Zombie Workloads

    Many organizations ignore zombie workloads that run on their architecture. When your team is stretched and you’re facing a number of serious security issues, it’s easy to dismiss zombie programs as nothing more than a harmless irritation.

    In actuality, having zombie resources scattered around your deployment blocks your ability to spot serious cryptojackers. Zombie workloads are often an indication of, and an invitation to, more significant invasions into your infrastructure. A report by SkyBoxSecurity revealed in 2018, cryptojacking was the leading attack vector. DevOps teams need to recognize the threat of hosting a program mining your resources for cryptocurrency, and take immediate steps to block them.

    Turn a Blind Eye to Leaky S3 Buckets

    AWS service buckets, particularly the S3 bucket, is one of the oldest cloud-native services. As a result, many S3 buckets are still governed by very old rules and regulations that leave many ways for someone to accidentally make the bucket public, open up access to unauthorized users and expose sensitive data.

    A study by HTTPCS reports 58% of AWS S3 buckets are accessible to the public, and 20% of them aren’t write-protected. Not only can malicious actors get to your sensitive customer data through S3 buckets, but often they can access your cloud credentials as well. Some of the most disastrous data breaches began from unrestricted access to S3 buckets, and it’s one of the most common causes of data breaches overall. It’s vital to regularly check your AWS console for public buckets.

    Allow Runtime Updates to Bypass the CI/CD Pipeline

    Every DevSecOps team agrees that your deployment is more secure when runtime updates are passed through the CI/CD pipeline, but that doesn’t mean the policy is always enforced. Developers will repeatedly ignore your security policies through practices, such as using open-source libraries, to avoid the security process of the CI/CD pipeline and speed up deployment.

    While this saves developers some time on releasing updates, it places a heavier burden on security teams, who have to run extra scans for these rogue workloads. What’s more, it builds complacency among DevOps teams, who assume there is no way to prevent unauthorized workloads and end up simply accepting them. Eventually, your security posture erodes to the extent that malicious actors can run harmful workloads without attracting your attention–until it’s too late.

    Permit Unrestricted Network Access

    Instead of spending hours on segmentation and separate access permissions, all too many DevOps teams fall back on a blanket set of network configurations that fall far short of the necessary access restrictions. They often place all their workloads into a single VPC, opening up access to third parties in the process.

    Without restrictions on public network access, it takes far longer for security teams to identify and isolate negligent and malicious activities. In a short space of time, DevSecOps teams find themselves overlooking serious oversights like unrestricted public root access, opening up enormous holes in their security profile.

    Faulty Rule Configurations for Micro-Segmentation

    When DevOps teams use micro-segmentation in containers, they face unique challenges. The more granular you get with your segmentation, the greater the risk you’ll miss faulty policy rules.

    Even your most familiar rule sets can create massive vulnerabilities. For example, if you allow your developers to use a specific IP to connect to the production runtime environment through SSH, you could unwittingly end up permitting unrestricted public access to sensitive areas. Sometimes, these faulty rule configurations go overlooked for as long as several months. To avoid supporting faulty rules, it’s important to audit them regularly with a tool such as Amazon Inspector’s Agentless Network Assessment.

    Avoiding AWS cloud security mistakes requires a shift in mindset. The harsh truth is a significant number of cloud-native deployments commit not just one, but several of the mistakes listed above. Cloud environments are deliberately decoupled from the network infrastructure in order to enable agile business practices, but DevSecOps teams continue to use network security techniques that aren’t capable of meeting the unique challenges of the cloud.

    The only way to fix these mistakes is to turn to new technologies designed for cloud security, such as using cloud workload identities to bridge the chasm between business logic and network infrastructure. Until ACLs and security groups are untethered from the network infrastructure and join cloud-agile business logic, they’ll fail to provide adequate cloud-native security.

    — Ran Ilany

  • The New Era of Kubernetes, With StackRox

    The New Era of Kubernetes, With StackRox

    Orchestrators are becoming the operating systems of cloud for the future. Which one do you think has become the dominant one? Ali Golshan, co-founder and CTO of StackRox, placed his bet on Kubernetes right from the start.

    StackRox is a Kubernetes-native security platform that helps DevOps and security teams operationalize security, providing richer context, native enforcement and continuous hardening.

    During this interview with MediaOps founder and editor in chief Alan Shimel, Ali discusses the adoption of Kubernetes and its prevalence today. More companies are realizing they can’t use traditional security. The tooling has to be native to the infrastructure they’re using, it must integrate into the build and the deployment. And Kubernetes is the best solution available, he said.

    Check out the interview below to learn more about microservices, containers and Kubernetes.

    — Veronica Haggar

  • Free DevOps Training at DOI’s Global SKILup Day

    Free DevOps Training at DOI’s Global SKILup Day

    The DevOps Institute (DOI) is hosting a global training day called Global SKILup Day on Dec. 10, 2019. It’s a free virtual event and they have arranged for some of the brightest minds in the DevOps space to speak/train during the course of the day.

    Why global? They truly are planning on a follow-the-sun approach to training, with courses going on in each of the different regions throughout the day. In total, the event will run for 18 hours and span the globe–meaning where ever you are, free training is available to you.

    The SKIL (Skills, Knowledge, Ideas and Learning) part of this free training day is DOI’s training methodology for the next phase of DevOps. They have been conducting SKILup Days around the globe for a while now, and this appears to be a virtualized extension of that effort.

    “What type of training,” you ask? Well, here’s a list of topics broken down by DOI.

    No matter what your interest and/or needs are, there will be something that you can pick up at this training.

    As of this writing, there are 15 speakers to cover 18 hours in three regions, so while I am not party to their internal planning, this author would guess they are still filling out the engagements. Considering we’re nearly two months away, that’s not terribly surprising. What is surprising is the list of speakers contain a high percentage of recognized experts. That’s good for the training, because superstars who can’t present generally don’t have the name recognition these presenters do. And a huge chunk of effective of training is presentation.

    So go sign up! Grab a spot, set the systems to do long builds and learn something new. I like to go to these type of events simply because our little lab is set in its ways, and getting exposed to new methods/ideas keeps things from getting to stale. I won’t be taking up space at DOI though, as I’d rather leave it for all of you when space is limited.

    Since my current responsibility is for a testing lab, this is more important for those of you who are running public facing systems that represent your organization. If I were, the big one for me would be AIOps, because it kind of snuck up on me while working on continuous test and integrating new tools into the DevOps toolchain. In fact, my recent piqued interest in the topic came from looking at an integration and the tool was big on AIOps–though CI/CD Reference Architecture would be cool too. Attempts to standardize are interesting, because even though some parts have stabilized, we still very much live in the land of flux.

    Pick something you don’t think you care too much about along with things you do. You just might find a good reason to care, and it will help you continue to rock it.

    Disclaimer: I have contracted for DevOps Institute in the past (through Ingrained Technology), but I am not currently engaged by them. I shared this because it is free training, and who doesn’t want to know about that?

    — Don Macvittie

  • Understanding and Implementing the APM Puzzle – Performance and Availability

    Understanding and Implementing the APM Puzzle – Performance and Availability

    While every organization is unique, there’s one universal goal everyone can get behind—creating and delivering an exceptional end-user experience. For both classic IT organizations and companies built in the age of cloud, delivering continuous availability and appropriate application performance is key.

    Although digital transformation has become a buzzword some would say is meaningless, the idea of digitizing a traditional company and building new businesses whose foundations are established upon the latest web technologies is critical in today’s IT and non-IT organizations. Think about the prevalence of cloud migration efforts during the past several years. Traditional organizations typically prioritize on-premises applications with increasing quantities of hybrid applications. But, because they aren’t yet fully transitioned to cloud—whether IaaS, PaaS or SaaS—their IT organizations need to live in both universes.

    Monitoring these environments at both the infrastructure and application tiers and across each stage in this transition requires support of a full stack of tools to cover web uptime, user experience, transaction monitoring, application tracing, log analysis and appropriate infrastructure monitoring. That said, modern organizations that have fully embraced cloud and leverage modern microservice architectures have similar challenges, but the exact same level of monitoring visibility requirement.

    Whether public, private, hybrid or multi-cloud, nearly every organization has moved to the cloud in some capacity. Regardless of what the company produces or the services IT provides, it must maintain uptime, availability and seamless performance to create and continuously deliver a user experience rivaling competitors and exceeding user expectations.

    Because let’s face it, even though every business is different and virtually every market is saturated, consumers and B2B companies have more options than ever. Without delivering a solid end-user experience, there’s an ever-increasing plethora of alternative vendors and providers from which customers can choose.

    Public Cloud Platforms for Classic IT Organizations and the Cloud Generation

    Let’s explore a few differences between how traditional IT organizations and cloud-era companies interact with the cloud. At a high level, Azure is a natural choice for IT companies with significant investments in apps traditionally acquired (not built) by the company and managed by their IT team, that are now tasked with digitizing their business by leveraging public cloud platforms. These types of companies often have Microsoft Office, Microsoft Exchange and other Microsoft products already implemented on-premises, and Infrastructure-as-a-Service (IaaS) or Platform-as-a-Service (PaaS) solutions as well. If they’re now adding a public cloud platform to their mix, staff expertise and solution familiarity make adopting Microsoft Azure a natural next step. Microsoft has long recognized the need to make these types of transitions and mixtures easier, demonstrated by its Office365 suite (as an example).

    On the flip side, companies that were born fully in the cloud, (i.e., relatively newer organizations without a legacy anything on-premises, with respect to IT infrastructure) will likely have a higher propensity to choose a provider such as AWS or Google Cloud even though Microsoft, Oracle and others are certainly competing there as well. So, it’s not a matter of one being intrinsically better or more fully-featured, but about which is the best fit for the organization and its priorities.

    At the end of the day, however—regardless of organization type or public cloud platform implemented—every company needs monitoring tools and solutions that keep applications up, running and performing seamlessly. With an integrated set of simple, easy-to-use, quick to value and powerful solutions, organizations can achieve truly proactive lifecycle assurance of application availability and performance.

    Uptime and Seamless Performance: Putting Together the APM Puzzle Using All the Pieces

    Each and every piece of the APM puzzle must be incorporated to achieve full-stack visibility and ultimately top performance: user experience, metrics, traces and logs. Having a log management solution monitoring availability, enables proactivity and enables IT to react quickly to help identify and remediate problems is paramount—it’s table stakes that everything be available. After all, if a website or page isn’t accessible, business transactions cannot be processed and that business is effectively down.

    Creating and maintaining a healthy application involves not only ensuring it’s functionally available to users; it must also be performing appropriately—continuously delivering both a reasonably responsive user experience and supporting the required volume of transactional work. This means a complete view into all aspects of performance—from the website and webpages all the way into and across all relevant application components and their associated infrastructure—be proactively visible and monitored.

    There is thus a natural and increasingly urgent workflow affinity between logs and APM—across the development lifecycle. When errors occur, it’s critical to understand, precisely and immediately, where are the impacts on service performance, or if there even are any impacts. Conversely, when application performance issues arise, it’s critical to immediately know exactly which errors—across the complex and constantly changing infrastructure and application stack—are directly related to the problem at hand.

    Without this bi-directional workflow integration, not only can application performance and availability not be managed proactively, but any issues with application service availability and performance will take far longer to diagnose and repair—seriously impacting the business bottom line.

    Just because everything is available doesn’t necessarily mean it’s appropriately performing for the end user trying to use the application.

    With all this in mind, it’s paramount to implement powerful, easy-to-use solutions that’ll achieve integrated application performance and availability management for any type of organization operating on-premises and/or cloud-based applications across all cloud environments.

    Conclusion

    While traditional IT organizations and cloud-only companies are different in many ways, providing exceptional end-user experience and achieving seamless availability and performance (no matter what) is a common and increasingly critical business goal. To achieve business success, organizations should choose a complete suite of simple, powerful and integrated solutions to optimize and manage their cloud-hosted applications and platforms. Anything less leaves the APM puzzle unsolved.

    — David Wagner

  • The Real-Time Innovation Dividend

    The Real-Time Innovation Dividend

    Time is compressing. The attention span of individuals and organizations is becoming ever shorter. There are only brief windows that can be devoted to any given transaction.

    Expectations around situational knowledge, updated in real-time, are compounding. Customers expect the applications they interact with know where they are, what they are looking for and what their preferences are in terms of price, value, design and timeliness of delivery. If the proper options aren’t provided immediately, the customer will move on to another provider.

    While the time available to an algorithm or application is contracting, the complexity of the tasks for successful interactions is increasing. Digital businesses must do more in ever shorter time windows. Differentiation in the digital world has a lot to do with a deep understanding of the customer based on multiple data streams and driving decisions that provide a deeply personalized experience.

    In business, we need to have a current picture of our situation. We build models and analytic dashboards, but these fall short in allowing us a real-time view. The latency of these models may be measured in seconds, minutes, hours, even days and weeks. As latency expands, the picture is not of the moment and that difference defines operating risk. The phrase “time is money” has never been more accurate.

    Brokerages require as close to a real-time view of positions allowing them to manage risk more closely, in terms of their aggregate position, based on a transactionally correct view of precisely what they have on their books to respond in real-time to global market changes.

    For retailers, it means knowing the content a customer has seen and the environment where they encountered it. Did they just come from a brick and mortar store? What digital journeys have they been on? What online content have they been perusing? Continually adaptive models use the answers to these questions to select active customers and recommend options that address what they are searching for.

    The digital shopping experience is composed of multiple linked processes, working in concert to continually understand the customer experience. Different system elements–user profiles, decisioning, recommendation engines and machine learning and AI–require data across different time windows. Some of these systems deal with terabytes of data and some petabytes. Some require absolute consistency and for some eventual consistency is good enough. In many of these systems, there is a demand for hyperscale within tight time windows.

    The ability to take in more data within a given time window feeds the currency of the context. It is not enough to have a lot of data, but you also need the most current view of that data. Operating at hyperscale enables an organization to deliver exponentially more data than traditional databases. This makes models more accurate and, by extension, more valuable.

    Scale Plus Speed Equals More Time

    In some sense the combination of scale with speed creates time, giving you the ability to do more within a given time window. Consider this a dividend, allowing the use of more data or allowing the process to be completed in less time.

    The systems that interact directly with either customers, employees or clients are edge computing systems, pulling in and processing data within relatively tight time windows. Edge systems may have expiry windows where data is deleted after a given time window where it is deemed to be stale.

    The data collected and processed at the edge may be moved to a system of record or core transactional store and kept for longer periods providing historical context. These data sets may grow to petabytes in size while still needing to support response times measured in milliseconds. Meeting these requirements requires hyperscale capability, where the performance of data access does not degrade as the repository of information grows.

    Here’s a real-life example. We must match the information collected regarding devices, identity and patterns that establish context against our database of user profiles. We want to run more sophisticated recommendation engines based on these patterns to ensure we are putting the closest thing to what the customer wants in front of them. We also want to make pricing decisions in real-time, to gain the largest basket at the best margins. We may have established a model that tells us customers with this profile are likely to make additional purchases after this transaction, so our analysis and optimization boundaries go beyond just this transaction. We will want to log this so we can track this behavior to validate the model. We have these processes going on, some in parallel, others in sequence:

    • Establish an identity profile of the customer.
    • Match that profile against a database of profiles that models likely desires and behaviors.
    • Based on that match, determine the products and/or services to recommend.
    • Select the content and media types based on the user profile.
    • Determine pricing based on the user profile and the transaction profile.

    The last element, real-time pricing, is something we want to modify to take into account the margin and profitability of the customer over time. We also know that if this impacts the overall window of the transaction the customer has a higher likelihood of abandoning the transaction and we can lose out on not just this transaction, but a whole set subsequent to this one.

    Doing Better for Both the Customer and the Business

    By building models based on hyperscale capability, 10 milliseconds are saved here, another 20 milliseconds there. This creates the time bandwidth in the overall process to add the notion of optimizing pricing around not just this transaction, but the value of this transaction profile and the relative margins of the various components in this sale and other subsequent sales that are likely to happen. We have created time to do this additional processing and do better–better for our customers and better for our business. And we know that as we add more data points to our profiles and scale up to include more data overall, we won’t experience performance degradation. These time benefits will remain. This is the real-time innovation dividend.

    — Lenley Hensarling