devopsy.kinsta.cloud

Tag: sdlc

How to Make DevSecOps a Reality

DevSecOps is an increasingly popular term; however, security vulnerabilities in software continue to proliferate. 2019 saw a surge in web application breaches shining a spotlight on the fact that DevSecOps remains elusive. The latest data from the Verizon Data Breach Investigations Report (DBIR) identified that web application vulnerabilities had doubled in the last year alone.

With the rapid pivot to all things digital in 2020, the pressure on software and applications continues to intensify. Security needs to be a fundamental part of software development in our digitally dependent world. This requires organizations to focus on more than the speed of delivery and turn their attention to the quality of software as they try to accelerate the pace of digital transformation.

Achieving DevSecOps

Organizations striving to achieve DevSecOps need to adopt the following steps to embed security throughout the entire software development life cycle (SDLC).

Make Security a Priority From Day One

Security can’t be a bolt-on at the end of the SDLC if your goal is to make DevSecOps a reality. The recent Zoom failures put security flaws in the spotlight and showcased the risks resulting from delivering innovative services without factoring in security. Every member of the team spanning product developers to technical architects to scrum specialists must consider security for it to be an intrinsic part of each stage of development.

Achieving this transformation to DevSecOps means rethinking how to determine success. Many organizations measure it solely in terms of velocity of time to market without evaluating software quality. Developers remain reticent to incorporate security and turn attention to the quality of code. As a result, software is released more rapidly but with scant regard for the quality, resulting in many applications not being ready for mass adoption.

One Size Doesn’t Fit All With Security

Don’t make the mistake of approaching security as a monolith. Instead, accept that it’s wide-ranging, incorporating multiple different areas spanning authentication to access control to confidentiality to integrity to non-repudiation, to name a few. Based on this broad scope, a single approach can’t hope to address all the different facets of security.

Teams need to rethink security and look at what it means to them and what is essential to their users. This could be keeping customers’ personal identifying information confidential and establishing the types of attacks that are most likely to occur. For example, if your product is inside a corporate file, then compromised credentials and password attacks are a significant risk while denial-of-service attacks are unlikely.

Every member of the team needs to understand security threats and techniques, so they have enough knowledge to deal with recurring issues. Threat models can include an employee unwittingly doing something wrong to trying to protect data from government-sponsored cybercrime professionals. Focusing your training on these areas will help address the vast majority of problems and ensure applications are secure. It’s vital to view security as a science rather that than an art.

Don’t Overcomplicate Security

Security is viewed by many as a problem that can only be solved by highly skilled and paid consultants who specialize solely in security. While there is an absolute need for these skills when reviewing architectures and carrying out audits, there is no requirement for this level of expertise when it comes to basic security checks, such as ensuring that the latest release hasn’t changed the authentication mechanism.

The vast majority of threat models are solved using standard static and dynamic analysis tools. Once this is accepted, you can resolve 90% of security issues through a combination of tools testers and developers. This enables security to infuse every part of the development life cycle and puts an end to the practice of being an afterthought.

Conclusion

Once organizations incorporate these three steps in implementing DevSecOps, they can ensure that achieving security becomes a core component of software delivery. Failure to do so will see that it remains an afterthought and security vulnerabilities will continue to grow.

July 20, 2020
The Secret to Winning at DevOps: Are You Up for the Challenge?
The main idea behind DevOps is to enable companies to keep up with the increased software velocity and advancements in agile culture for a smoother end-to-end software delivery cycle.

The main goal of DevOps is to accomplish integration and automation, which is why implementing this philosophy can be challenging. Not only do you need to replace older methods and practices, but you’ll also have to change the mindset of the developers and the operations team to ensure they work in sync.

Several IT operations teams and software teams have adopted DevOps practices into their work culture for faster evolvement and improved innovations. The pipeline comprises continuous integration, testing, deployment, as well as monitoring application and system performance.

Introducing new DevOps tools and changing the mindset to prepare teams for the fast-paced world remains crucial for a successful implementation.

Despite the revolution that this philosophy brings to software development, several enterprises find themselves taking DevOps to an unproductive extreme. To remain productive, one of the aspects that teams need to work on is reducing rework, and the best way to ensure this is automation.

Understand How DevOps Works

Under a DevOps model, the development and operations teams are integrated into one to prevent them from getting siloed. This also improves collaboration and productivity.

You see, it brings alive the possibility of both the teams thinking more alike, working together and sharing responsibilities. This is done by automating code testing and workflows, automating infrastructure and continuously measuring application performance.

In some models, even the QA and security teams merge with the dev and ops. In cases where security becomes the primary focus, the name gets modified to DevSecOps.

In essence, the focus of DevOps is on automating processes that were otherwise manual and slow. Teams use a technology stack and tooling that can operate and evolve applications more reliably and swiftly. As engineers will be able to accomplish tasks such as deploying code and provisioning infrastructure faster with reduced issues, teams can reach testing and maintenance stages quicker.

Balance Between Standardization and Adaptability

Software products are mostly developed from the customers’ perspective, and if not, from end-user experience. Often, developers work backward to identify customer needs and wants for ascertaining successful innovation.

One of the biggest challenges when developing a software development life cycle (SDLC) strategy for organizations is to consider dynamic customer needs, strict compliance policies, unpredictable market trends and company goals. To ensure a consistent workflow, DevOps teams need to adopt standardization. In fact, workflows, technologies, processes, protocols and metrics need to be standardized before they can be automated.

This is also the main reason why organizations adopting DevOps need to strike a balance between standardization and adaptability. One industry that has excelled at this has been the financial industry, which has been leading the charge when it comes to adopting the latest software delivery practices and technologies, including DevOps. Financial executives view DevOps as being a way to improve security and risk and governance strategies.

The result is that cloud-based financial services are now automating virtually all of their processes through their software delivering pipelines, which makes tasks such as scanning receipts, sending invoices, tracking expenses or making payments seamless as they sync to your account. This simpler process is beneficial for boosting developer productivity and resource utilization, and for enabling enterprise scale management. If anything automating processes is becoming necessary for achieving higher product quality, compliance and scalability.

As mentioned above, new DevOps tools need to be introduced to team members to keep up with the constantly evolving strategies and implementation. But to prevent the creation of silos between teams, mechanisms need to be developed to make the adoption of new technologies easier.

Achieving the delicate balance between standardization and automation will help eliminate communication gaps, performance bottlenecks and silos between the different teams.

Benefits of Automation in the DevOps Landscape

The following are the advantages of adopting a DevOps philosophy:
- Code Development. Source control will keep everyone in the loop by managing, tracking and documenting any changes in code. This can be time-saving, and reduce the complexity of software projects.
- Smoother Testing. Testing is a continuous process in DevOps frameworks that includes system communication, several test integrations, predicting and tracking issues, and integrating approved automated builds. In the absence of automation, executing and managing continuous testing could be a rather tedious job that might cause problems between developers and the operations team.
- Better Visibility. The main job of the ops teams is to be aware of the code changes, ongoing problems and the impact of a specific code on the end goals. Plus, when they operate in a DevOps environment, they need to keep the developers in the loop as well.
  For example, if a system administrator from the ops team identifies code issues weeks after submission, it puts more pressure on developers. Now, the developers have two jobs: solve the problem in the previous code, and simultaneously work on making progress with the new code. This is what gets time-consuming, which can harm the company’s market standing. To remain ahead in the competition, innovation needs to be frequent and accurate.
  When there is a constant feedback loop between the dev, ops and QA, bottlenecks, and silos are eliminated, productivity is improved and customers are happier due to quicker innovation.
- Continuous Integration and Delivery. Automation can help deliver appropriate software built, data, code changes and tests to target environments. An example is how web hosts for e-commerce businesses need to be updated frequently for robust deployments. Automated testing will help ensure that code is delivered properly and doesn’t introduce new bugs, thus creating a platform with a superior user experience.
  This, in turn, helps DevOps teams to introduce frequent software changes by deploying error-free codes and continuous testing.
- Data Processing and Monitoring. The teams also need to measure and monitor the performance of their environment, which can be done by reading and analyzing system logs. But the existence of hundreds or thousands of servers in the environment can make it complicated.
  Automation will allow the teams to make sense of all the data and let them know how an application is performing and whether there are any bottlenecks.
Getting the Balance Right

Organizations need to realize that automation involves placing technologies around once-manual processes. The catch here is as the processes will become inherently repeatable, they need to be correct. Otherwise, organizations will only have flawed processes running faster, which will ultimately defeat the purpose of adopting DevOps in the first place.

Companies need to understand adaptability as well. This can be done by mixing both the old as well as new systems. For this purpose, an inventory of all applications, data stores and development processes for promoting meta development processes that include design, development, testing, deployment, configuration management and so on can be made.

In the end, the secret to winning DevOps is aligning DevOps tools with your organization’s goals – not following the footsteps of other organizations.
June 23, 2020
Increasing Software Velocity While Maintaining Quality and Efficiency

At Compuware, we have been exploring the need for continuous improvement across three key dimensions of software development: velocity, quality and efficiency. Given today’s current market demands and the importance of digital services, this discussion couldn’t be timelier.

On my previous piece, I focused on quality — how to maintain and improve it as (supposedly) competing demands for velocity and efficiency increase simultaneously. This article will focus on increasing velocity and how it is possible to do so without compromising quality or efficiency. As I discussed in my earlier piece on quality — automation, integration and measurement are once again the key.

Compuware recently conducted a survey finding the overwhelming majority (97%) of application development managers said IT is under more pressure than ever to deliver software innovation faster. Higher demands for innovation are being imposed universally across platforms, from the cloud all the way back to the mainframe.

Mainframe software delivery is a particular area of focus, with 95% of respondents noting their IT teams are under pressure to adopt modern development practices on the mainframe in order to accelerate overall software delivery and innovation. Adding to this pressure, organizations reported they expect to increase the frequency of new mainframe application feature deployments by an average of 41% over the next year, compared to 38% for non-mainframe applications. This shouldn’t come as a shock, given the proliferation of modern web-based applications placing unprecedented demands on back-end mainframes.

Let’s take a closer look at how automation, integration and measurement are equally applicable to the goal of advancing velocity.

Automation

Throughout the software lifecycle there are many opportunities for automation — from software design, to development, to build, test, deploy and, ultimately, the production state. The more of these steps that can be automated, the faster developers can work.

Perhaps the biggest area for potential time savings is testing, because of all the phases of the software lifecycle, testing assumes the most manual labor. The Vanson Bourne survey found test automation to be the single most important factor in accelerating innovation, according to 90% of IT leaders.

On the mainframe specifically, development teams spend more than half their time on testing during the release of a new mainframe application, feature or functionality. If you can automate more testing, you can roll out applications faster while maintaining (or even enhancing) quality combined with an extremely high level of developer efficiency.

Integration

Integration helps software development technologies interoperate with each other. An integrated pipeline leverages existing investments and enables developers to build, test and deploy faster while providing the additional benefit of reducing errors resulting from human intervention (furthering quality). Integration also decreases the manual labor needed to execute and manage workflow within software delivery processes. When these technologies integrate out of the box, setup time is saved and productivity gains can be realized significantly faster.

For example, the integration of Compuware Strobe application performance solution with Jenkins, makes it easy for developers to take performance measurements early and automatically as part of a Jenkins CI/CD pipeline.

Ongoing Measurement

Measuring velocity is critical. Without measuring, it’s impossible to know how changes are impacting the software development process and what areas need to be improved. Measuring is the key to understanding if there are any activities that can be sped up across the phases of build, test and deploy. This enables development teams to find and address bottlenecks with a great level of granularity. Some common metrics for measurement include mean-time-to-build (MTTB), mean-time-to-delivery (MTTD) and mean-time-to-repair (MTTR).

Conclusion

Velocity does not need to exist at the expense of quality or efficiency — on the cloud, the mainframe or any other development platform. Like quality, the key to achieving and improving velocity lies in the disciplines of automation, integration and ongoing measurement. The next piece of this series will explore the keys to maximizing developer efficiency amidst near constant demands for exceptionally fast, high-performing rollouts.

May 22, 2020
The Secure Software Development Life Cycle: Syncing Development and Security
Over the last five to 10 years, the nature of software development has shifted dramatically. Whereas large software releases occurred every six to 18 months in the past, current release schedules have become much more frequent. The waterfall model of software development has morphed into what we now know as the DevOps model.

As a consequence, DevOps has instigated changes in the traditional waterfall security touchpoints. In essence, DevOps has changed the very nature of how application security needs to be implemented. To avoid friction in the development process, application security activities must be adapted to the new world of DevOps practices. It has to go beyond the basics of threat modeling, design review and two weeks of pen testing.

Under DevOps, some development organizations now do software releases on a daily, weekly or bi-weekly cadence. But they’re still grappling with older application security models. As a result, development and security testing can be out of sync—you cannot conduct a two-week pen test on software that’s released weekly. These organizations need effective security assurance in their SDLC that’s good, quick, early and—above all else—automated.

The Changing Nature of Software Development

The way software is developed keeps changing. And it keeps accelerating. Application security needs to change to catch up. Automation will play a key role in enabling application security to maintain the pace of DevOps. DevOps needs automation to effectively deal with software release cadences that occur on a daily, weekly and bi-weekly basis as well as to increase agility generally within the SDLC.

For example, DevOps accelerates when the automated pipeline embeds application security scanning into pre-commit checks. In this way, consistent levels of application security testing apply every time developers check in code, resulting in a fully automated check-in process. But automation doesn’t lock developers into the same level of security for every code change. For example, if the change only involves altering the color of a logo, heavy security scanning and manual won’t be necessary. At least not the way it would be if the application is accessing sensitive data.

So, instead of giving developers quarterly SAST results, automation puts the code analysis right there in front of developers as they’re checking in code (via certain IDE plugins). Presenting the code analysis findings to developers, while they are still in the code, has several advantages:
- The code is still fresh in the developers’ minds.
- Coding problems can be caught earlier and more easily while developers are still thinking about the code.
- It mitigates the risk of developers not remembering the code context at a later date.
Overall, the more automation there is in application security the better it is for open source validation, container scanning containers, QA testing, API security and more.

Automated Security Assurance and Out-of-Band Activities

When it comes to implementing testing in DevOps, you must balance automated security assurance with out-of-band activities. That means it’s sometimes appropriate to slow down the automated security process. Even in a completely automated DevOps process you may need to slow down to make necessary changes. It’s true a code change, such as just changing the background color on a web app, may not be too risky. There’s no problem with automating that DevOps step and pushing out the change. But, if your web app interacts with customer data, you’d be prudent to slow down and take the next application security test out of band, and make sure you’ve got the procedure right because it’s much riskier.

There are different levels of risk and severity within each release of an application. Development organizations need to accept this and adapt to each release’s potential risk and severity level. Then, they need to provide the appropriate level of security assurance and security coverage, whether that is automated within the DevOps workflow, performed out of band or some combination of the two.

Life in the Fast Lane

For DevOps organizations, one of two avenues beckons to them: the security express lane or the security slow lane. You could create an automated DevSecOps process across the development organization—the security fast lane. But, there’s no one-size-fits-all DevSecOps process for enterprises. That’s because there could be up to hundreds of development teams within an enterprise. And there’s not always a standardized process that all dev teams can follow. It’s a challenge to adapt automated security to each development team. Those in the security fast lane should use a security champion to work with the development team and be the point person to assist them with understanding application security needs and remediation guidance.

In the security slow lane, individual development teams may still be enabled to go fast. The difference is that they will pick the SDLC tools that make the most sense for that team. Then, they should also pick the application security tools that make the most sense for them. The bottom line for organizations is that application security needs to involve an acceptable level of friction for developers to make DevSecOps work. If there is too much friction, developers will bypass application security in favor of expediting coding activities.
May 13, 2020
The Key to Embracing DevOps: A Real-Time Roadmap

While the term “DevOps” has become somewhat of an industry buzzword, it represents an important shift in the way teams think about planning, building and deploying software. Drawing its inspiration from the Agile movement, DevOps extends many of Agile’s core principles beyond development to how teams deploy, monitor and inform their next round of product planning.

Because DevOps touches so many areas of the software development lifecycle, more stakeholders than ever before are now involved in the design, production and release of software projects. With more moving parts and people, the continuous delivery of valuable software depends on these stakeholders being aligned at all times and moving in close coordination with each other.

In reality, however, that’s much easier said than done.

The problem with most projects is that they face a lag time based on the manual nature of updates required from the people doing the work. Sometimes this delay may be considerable. At the same time, accuracy becomes a problem because it involves people manually making and changing entries. This only compounds as the number of teams and dependencies involved in a project grows.

Another significant issue is the disruptive nature of context switching on development teams. Ideally, developers can concentrate on coding and not have to contend with a myriad administrative issues. In reality however, most developers are constantly forced to leave where they’re working to provide status updates for the team. Every “start-stop” slows the process and can even impede creativity.

While it’s tempting to just do away with updates, they’re incredibly important for the business. Stakeholders and customer facing teams need to understand if projects are on track or running behind schedule. Similarly, management needs to understand what projects are at risk in order to step in and provide additional resources or work with the team to determine ways to reduce scope.

So, how can teams stay aligned without compromising the developer experience?

While there’s no one right way to achieve total team alignment, one solution is maintaining a real-time roadmap that is shared between development teams, ops teams and the rest of the business. In a true, closely coupled DevOps environment, data from testing, deployment, integration, monitoring or other steps can be fed into roadmaps to reflect new realities and considerations.

If such input comes fluidly and automatically, the roadmap will not only reflect the exact status of development but also the practical matter of having software work as intended in a production environment. Again, not only can business owners know the true status of a project, adjustments and decisions can be made along the way to ensure that goals and expectations can be met.

A by-product of accurate, fluid, real-time roadmaps is the trust that stems from all being on the same page and having joint ownership for the entire process. The transparency and currency of such a vantage lessens finger-pointing and lessens “us-them” tendencies, as the work and the results can be jointly held. What results from this is accountability across the entire software development lifecycle.

Real-time, integrated, roadmaps perfectly fit with the spirit of what DevOps tries to achieve.

They operationalize work being completed and allow for user feedback and telemetry to continuously flow back into the next rounds of iteration planning. Real-time roadmaps accurately reflect what’s happening at each stage in the DevOps lifecycle, creating a single source of truth for developers, business owners and operations teams working at each stage of the process to ship value to customers.

The spirit behind DevOps calls for tighter integration leading to increased levels of efficiency and effectiveness. Adding real-time, integrated roadmaps to the DevOps discipline makes it even better.

— Aaron Upright

April 16, 2020
The DevOps Sweet Spot: Inserting Security at Pull Requests (Part 2)

In part one of this two-part series, I explored how organizations can more effectively automate security quality decisions and discard doing automation for automation’s sake. I shared why security scans need to be faster, more reliable and comprehensive. Only then can security be meaningfully automated into developer workflows without slowing them down.

But, the other part of this is understanding where these security scans should take place throughout the SDLC. Why? Because inserting security too early or too late in development and pre-production can have drastic impacts on remediation efficiency, developer productivity and more.

Shifting Security as Far Left as Pull Requests

Identifying vulnerabilities once they’re in production is the riskiest, as well as the most time-consuming and expensive remediation. Working in pre-production test environments or builds reduces the risk, but not much of the time or efficiency in remediation. Conversely, inserting code analysis into developer IDE’s is too early. While in theory the IDE could be a good place for security feedback, in practice, it slows down the developer’s machine and comes at a huge cost of productivity.

Instead, commits and pull requests are the ideal place to insert security because it delivers the right vulnerability information to the right developer at the right time. Combine this with integrations into ticketing and other systems and the full developer workflow can be meaningfully automated.

Simply by getting the right developer, the right vulnerabilities information breaks down one of the biggest challenges in AppSec. Developers typically outnumber AppSec team members by 50-1 or more—hence a security scan that generates 100 vulnerabilities creates a tremendous amount of work for a single AppSec professional. However, assigned to the right developers, at an average of two per person, the challenge immediately becomes more manageable. Still accuracy matters because, while the burden may be spread out, developers triaging false positives is an expensive resource focusing on a low value task.

Getting developers security feedback at the right time is also vitally important. When developers get immediate security feedback, vulnerabilities become easier to fix because the code is still fresh in their mind. Furthermore, fixing vulnerabilities early has compounding benefits. Since insecure code never goes into production, not only is the attack surface reduced, but additional code that is dependent on the vulnerability never gets written. This minimizes the most expensive and time-consuming security fixes, which require not only fixing the underlying vulnerability, but also rewriting dependent code. Best of all, with immediate, regular, comprehensive and accurate security feedback, developers will ultimately learn and adopt more secure coding practices overtime.

Lastly, by inserting security at the commit or pull request, security and developers can establish a shared responsibility model on more realistic terms. When security can commit to accuracy and timely KPIs, developers can commit to triaging and fixing vulnerabilities faster. This reduces MTTR, puts a downward trend on defect density and reduces the application’s overall attack surface. It also reflects the realities that AppSec professionals have a limited ability to triage vulnerabilities, because they don’t know the code intimately and that only developers can fix the underlying vulnerabilities.

By solving the fundamental challenges of speed, accuracy and comprehensiveness to deliver the right vulnerability to the right developer at the right time, security earns the ability to ask more from developers, and CISOs and CTOs can jointly accept responsibility for protecting applications and data.

The rate at which software is being created and shipped isn’t slowing down, but many organizations have struggled to find where security most effectively fits into this process. By inserting security at the pull request, not only can developers identify and remediate vulnerabilities faster and more accurately, they can better automate security quality decisions. By taking this approach, organizations are fundamentally changing how engineering and security collaborate within enterprises.

— Chetan Conikee

March 17, 2020
The DevOps Sweet Spot: Inserting Security at Pull Requests (Part 1)

Today’s organizations are increasingly benefiting from the modernization of the software development lifecycle (SDLC), including the adoption of cloud, DevOps, Agile methodologies, containers and more. Thanks to this modernization, organizations are innovating and releasing updates faster than ever before. But these benefits aren’t without challenges.

Application security arguably tops the list of challenges. Security has inherently manual bottlenecks, and is unable to keep up with the pace of automated DevOps pipelines. The result is that security is often left behind, overlooked or simply ignored in the development process. Developers aren’t willing to slow innovation down today for the potential of security breaches tomorrow.

As a result, more and more applications are being shipped into production with vulnerabilities and security flaws. Today, most of the vulnerabilities are found in post-production either via red teams, bug bounty programs or siloed and irregular code analysis scans. At best, these vulnerabilities are fixed in the next version that’s shipped. At worst, your organization is making headlines for millions of personal records being breached.

However, security in the modern SDLC is becoming more symbiotic thanks to two key advancements—speed and accuracy. Overcoming these two challenges enables security to be inserted into developer workflows earlier than ever before. By automating security with every pull request, development organizations can finally embrace security without slowing down innovation.

Automation — Security at the Speed of DevOps

These days, many vendors claim to insert security into DevOps pipelines, often during the build. While it’s true that security scans may be automatically started, they still finish in siloed processes that don’t keep up with the pace of releases. We call this “automation for automation’s sake.” It does not measurably improve defect density, slow the accrual of technical debt or reduce mean time to repair (MTTR). The only value that automating this way provides is checking a box on a compliance form.

So, how can organizations more effectively automate security quality decisions?

First, security tests and scans need to be dramatically faster. Legacy code analysis takes such a long time that many vendors recommend sacrificing comprehensiveness by scanning only differentials. Even then, they still aren’t fast enough to keep up with CI/CD. It goes without saying that organizations cannot meaningfully automate security into processes such as builds or pull requests, that take just a fraction of the time.

Second, the results need to be more reliable. The goal of security automation should be to fail insecure code. However, historically, code analysis tools are plagued by so many false positives that no code would ever pass any test. Only with reliable results can insecure code be confidently failed.

Third, security scans must be comprehensive as false negatives lead to vulnerabilities going undetected into production. This is, in part, because developers are not asking the right security questions when scanning their code. Traditional code analysis tools leverage pattern matching techniques—they look for symptoms of a vulnerability that are common across all code bases. For example, lack of input validation from sources is a classic cause of injection vulnerabilities. However, there are many vulnerabilities that are unique to each code base, such as business logic flaws, data leakage, hard-coded literals and insider threats.

For example, all applications are purpose-built to meet specific business needs, meaning vulnerabilities may go beyond coding errors and present themselves as business logic flaws in the design of an application. Take the First American Financial breach. Thanks to an insecure direct object reference (IDOR), an attacker has the potential to manipulate an internal implementation object. In the case of First American Financial, this was a URL or form parameter. Given the asymmetrical advantage that attackers have, defenders must protect from all possible vulnerabilities, but attackers need to only find one vulnerable route. As such, partial security can be as problematic as no security.

Only when security is completely comprehensive—reliable enough to fail with confidence and fast enough to keep up—can it be meaningfully automated into developer workflows without slowing them down.

Yet, comprehensive security scanning is only part of the equation. In the second article of this series, I’ll dig into where these security scans should be taking place in the development cycle to make them most effective.

— Chetan Conikee

March 11, 2020
Software Supply Chain Attacks: How to Disrupt Attackers
Supply chain attacks—compromising an organization via insecure components in its software supply chain—are a growing concern for organizations. Throughout the past three years, an increasing number of open source software package repositories have been found to contain malware, making it clear that all installation and update pathways for software and library code must have security controls applied to prevent and mitigate supply chain attacks.

Growing Risks of Open Source Software to the Business

Digital transformation has made every company a software company. As a result, open source software has grown 75% over the past two years. Despite this growth, few companies accurately track the components used in their code. Most lack the policies, processes and tools to keep up with the choices made by their developers, which introduces risk. In fact, 60% of the codebases audited in 2018 contained at least one vulnerability.

Bad actors have turned to exploiting these weaknesses to compromise organizations. If production code is infected with malware or vulnerabilities, inadvertently sourced from the repository, it may contaminate all organizations that come in contact with it—whether by using the code already in their SDLC or by launching presumed trusted applications from third parties who failed to validate their own code.

Attackers are also shifting their techniques to mask malware among large packaged objects using obscure formats we refer to as destructive objects. They infiltrate at various points in the development processes opportunistically, knowing they can infiltrate systems outside a managed operational environment. They take advantage of third-party dependencies and use payloads that can sit and wait until an unsuspecting developer implements their software. Hence, software supply chain attacks are difficult to detect, and often achieve dwell-time of hundreds of days.

Assessing Supply Chain Surface and Sources

In order to assess weaknesses and identify what warrants continuous monitoring, we need to understand both the surface and sources.

When evaluating surface risks, consider the following questions regarding the development environment:

What technologies, repositories and packages are in use?
- Inventory technologies used in development: Python, Ruby, JavaScript, Node.js, .NET, etc. This can also refer to operating systems.
- Third-party repositories of code: PyPI, RubyGems, NPM, NuGet, PEAR or PECL, and OS repositories such as DebianRepository, CentOS, Ubuntu, AUR and more.
- Packages: There can be multiple packages from a single repository. How often are packages updated? How often are new packages added?
Are there update procedures in place?
- What are the dependencies for every package? How many other packages are pulled from a single package update, and how often are they updated and/or reviewed?
Who’s managing the verification processes across all these dependencies? What update procedures are in place if there is a personnel change?

Once these questions have been answered, they should be addressed regularly. The process must be iterative to be successful.

When considering attack sources, it’s important to understand the characteristics of repositories teams draw from in order to understand potential areas of weakness. Third-party repositories rarely infect a big project as they are typically found elsewhere (e.g. GitHub) and have thorough code reviews. Official repositories, unofficial mirrors and internal code repositories represent increasing risk potential.

Random malicious packages, typosquatting, bad code review and phishing developer credentials to hijack accounts are all less technical ploys used by attackers, but equally dangerous.

The Software Supply Chain Challenge

The size, breadth of formats, trusted third-party and open source risks, and misused certificates have made it nearly impossible to stop malware-infected files and objects with today’s signature based, sandbox and vulnerability-based solutions.

Threats evade dynamic analysis remaining unknown to defenders or lacking details when detected. The explicitness of signature blacklists misses new or never-seen-before code, and sandboxes or dynamic analysis solutions are resource intensive and unable to scale to the breadth and volumes of formats required. Without a way to inspect large or cross platform archives, installers and source code, it’s impossible to extract indicators of compromise. These destructive objects can also impact SOC triage, response and continuous improvement.

Rising to the Challenge

How can you address these risks, not only in the software development life cycle (SDLC), but as SOCs address vulnerabilities that may come through other applications and repositories? How do you evaluate and inspect wrappers, digital certificates, executables, scripts, files and libraries, resources, documents and icons?

It’s vital to establish security practices within SDLC, from training developers on secure coding practices using open source libraries to factoring in detection capabilities including static analysis, dynamic analysis, software composition analysis and manual penetration testing. Implementing a secure SDLC process ensures that the development effort is protected by these activities, augmenting code reviews and infrastructure analysis.

Teams can extend security controls into the SDLC with technology that provides deep visibility into all aspects of the supply chain, from open source dependencies through continuous integration and delivery of packaged applications. Capabilities that provide greater application security across code development and deployment activities include:
- File reputation identifies and classifies “known bad” and “known good” files by applying a file’s unique hash against a database of goodware and malware.
- Automated static analysis detects and classifies unknown malware in code repositories by deobfuscating and decomposing objects and files without executing, and extracting threat indicators.
- Dynamic analysis executes files in a safe environment and captures the behaviors and results of malware, allowing the security team to understand what is going to happen.
- Software composition analysis analyzes applications to detect vulnerabilities, dependencies and licensing requirements in software.
- Threat intelligence is served up at various stages of the analysis cycle, and represents evidence-based knowledge including context, mechanisms, indicators, implications and action-oriented advice about existing or emerging threats.
By incorporating these technologies into the SDLC, development teams can obtain continuous validation across the software supply chain. Further automation, implementing new security guidelines and advancing the activities within the existing framework will allow the enterprise to progress its security measures and close gaps in the SDLC and supply chain operations.

At the same time, it’s important to recognize that threats can still make their way into the software supply chain, and it’s equally important that the SOC remain vigilant about maintaining a clear triage and escalation process. Continuous monitoring of software repositories should be an integral part of good hygiene processes, and incorporated into best practices applied to operational infrastructure, including the SOC. The SOC serves as a centralized management infrastructure to respond, contain and remediate attacks. This includes aggregating and consolidating threat data, building a case for managing a response which can include actions to formulate triage and incident response plans, and moving beyond SIEM.

— Tomislav Pericin
January 17, 2020
5 Best Practices for Managing Open-Source Components
In today’s connected world, software development environments focus a lot on faced-paced development. Organizations that adopt agile development practices such as DevOps and use Open-Source (OS) software and components to their advantage have a much better chance of keeping up with demand and shorten the Software Development Lifecycle (SDLC).

However, incorporating OS components into applications does not come without risks, as they often contain bugs or vulnerabilities, and are highly exposed to exploits. In this article, we will cover what is Open-Source Software (OSS) and OS components, discuss the importance of security for open source and cover five best practices for managing open-source components.

What Is Open-Source Software?

Open source refers to any software with accessible source code that anyone can modify and share freely. You can head over to the Open Source Initiative (OSI) and see a detailed list of criteria for the definition of OSS. While OSS allows many more modifications over proprietary software, you must still accept the terms and conditions defined in its license.

Some estimations suggest OSS adoption saves $60 billion dollars for consumers per year, while about 96% of all commercial applications contain some components or elements of OSS.

The Benefits of Open-Source Components

Open-source components are software units that meet the definition provided by the OSI. Modern development environments put a lot of pressure on developers to build and deploy applications more quickly. To successfully achieve their goals within short software release cycles, developers frequently use OS components.

The Importance of Security for Open-Source Components

OS components are incorporated into the vast majority of software and applications. Even though they allow developers to be more efficient and get more done, they can also introduce new vulnerabilities and jeopardize the integrity of the entire code.

Since OS components are developed in a distributed system, they are often more vulnerable to exploits than proprietary code. Some of the reasons for this are:
- Distributed development: Compared to a distributed OSS developed by many individuals, it is much easier to track and manage a code developed in-house and ensure you follow policies and protocols.
- Fast release cycles: OSS development is much quicker and agile, which means that very often, by the time a new software release passes all the quality and vulnerability checks, it has already become obsolete with the release of a newer cycle. This makes it almost impossible and in most cases useless to run the necessary tests and ensure the safety of using the code.
- Lack of quality standards: OSS elements do not pass the same quality and standard checks as proprietary code. Thus, unless you evaluate each OS component before implementing it, you can often incorporate low-quality code that is exposed to vulnerabilities, lowering the overall quality of your code.
- The double-edged sword: When members of the open-source community discover a vulnerability, they publicly share the details of the problem in online forums to help developers learn about the risk and how to patch it. Unfortunately, this solution is often a double-edged sword. While this is a step in the right direction, it exposes the details of the vulnerability to threat actors as well, who can exploit it against systems that have not yet patched the vulnerability.
- Lack of security measures: The problem with OSS is even greater for smaller companies, who prefer to use it to save time and resources but lack the necessary security measures to ensure the components they are implementing are secure enough to be implemented into the code.
Now, let’s take a look at the five best practices for OS components management.

Form a Policy

Establish a clear policy on how to incorporate and manage OS components. This step will give usage guidelines to your developers and ensure OSS elements are incorporated responsibly without damaging code quality. Additionally, if you are facing problems with OSS elements, this policy can act as a playbook and give instructions to your security teams on how to handle the threat.

Carefully Choose Software

It is important to choose the right software for you. While it might be tempting to use many kinds of software and enjoy all their possible benefits, you have to consider your requirements and ensure the software you use can answer all your demands. You should consider the amount of support you require and the risks you might face. One of the most crucial aspects is how secure the software is. A good way to measure the software’s security is the Open Web App Application Security Project (OWASP) dependency-check, a utility tool that identifies project dependencies and checks if they contain any open-source vulnerabilities.

Track and Update OS Components Regularly

Track your OS components throughout the entire product life cycle and ensure transparency to find critical security vulnerabilities. Updating OS components versions frequently is typically the best way to deal with bugs and vulnerabilities.

Forking

One of the best features of OSS is (as long as you keep up with the policies) you can modify the source code and customize the software to fit your needs. Thus, you can take software intended for specific use-cases and modify it to provide a better solution for your situation. However, this process can be intensive and take effort and time. Thus, before you attempt forking, consult with other community and team members to ensure you have the right idea and execution.

Use Automated Tools

Cybersecurity is highly time-sensitive—the first moments of dealing with an attack are the most critical. Automated tools, such as Security, Orchestration, Automation and Response (SOAR), can help you identify threats quicker and make mitigation efforts much faster to minimize and dramatically minimize the attack’s potential damage to your system. You can also use automated tools to enforce policies and check your OS components for vulnerabilities, making the process much faster and more secure.

Wrap Up

OSS and OS components have many advantages for developers and facilitate agile development. However, the flip side of their ease of use is the danger they pose in terms of open-source vulnerabilities. To mitigate this risk, you should incorporate the right practices for managing OS components into a clear strategy. This will allow you to enjoy all the benefits of OS components without exposing your application, network and systems to unnecessary risks.

— Ilai Bavati
September 11, 2019
5 Tips to Eliminate Knowledge Gaps in Dev and QA
To create and release high-quality software, the software development lifecycle (SDLC) was established using different roles and processes designed to work seamlessly together. This approach is of critical importance because it spans the initial planning phase, all the way through production release and monitoring.

Roles vary within projects but typically include designers, business analysts (BA), developers, testers, scrum masters, project managers, product owners (PO) and technical architects (TA). Each one brings different expertise and skills to a project but ultimately, they are meant to collaborate and determine how to implement different aspects of the software.

The amount of processes can also vary for projects based on team size, release schedules, availability of resources and the complexity of the software.

What is the Knowledge Gap?

Without a distinction of responsibilities or a roadmap that identifies who owns each task in the SDLC, everyone assumes someone else will complete the tasks. The result is that the majority of these important steps remain incomplete or undone.

The reason this happens is because a knowledge gap exists among the team. This is especially true when teams are a mix of entry-, mid- and expert-level individuals. Sometimes, assumptions are made regarding the skill set, experience and domain knowledge that each individual brings to the project. Knowledge gaps can also develop at a more granular level, with teams using the same tools but in different ways. As an example, developers and testers might be using a single testing platform quite differently.

As teams begin collaboration on features for the software, they may encounter the following situations:
- Each team member has a different interpretation of the project requirements
- Developers assume testers have their same level of technical expertise
- A lack of ownership and assigned responsibilities among the team
- A PO has a feature idea and each team member has different ideas for how that feature should work
- The TA explains how a feature should be implemented using technical jargon that designers, POs, BAs and testers don’t understand
- A tester is not allowed appropriate time to complete testing due to tight release schedules
Identifying Knowledge Gaps

Knowledge gaps not only negatively impact the release of high quality software, but also team morale, productivity, job satisfaction and empowerment. The first step to finding acceptable solutions is to identify exactly where a knowledge gap exists. Some questions to help teams do this include:
- What are the common problems that occur in each phase of the SDLC?
- What processes are in place during the requirements, design, development, testing, acceptance and release phases of the SDLC?
- How effective is the communication between different roles in the team?
- Are the responsibilities of each team member clearly identified?
- How visible is the status of the project at any given time?
- How often are release cycles pushed to accommodate for more development and testing?
- Are the teams aware of what kind of customers are going to use the product?
- Is the velocity of the team stable? How often does it fluctuate and by how much?
Bridging Knowledge Gaps

Finding gaps in a team’s knowledge is the first step. The next step is determining how to narrow the divide. The following key factors help bridge the knowledge gap in teams:

Training

Sufficient training and tools should be provided to help teams do their jobs better. Proper training has a domino effect on the success of the project. It helps designers understand how to design mockups so developers can effectively implement the feature; testers can attend code reviews without intimidation and use tools more effectively; developers will understand why the testability of a feature is important and understand which tools can aid the development testing effort; the scrum master can better manage project tasks; the project managers can ensure the team collaborates to meet release schedules and deadlines; and finally, stakeholders can have an overview of the project through reports that are generated using various tools.

Visibility

For teams to work seamlessly together, we need to ensure the results of everyone’s efforts are visible to the entire team. It is important to understand how each contribution maps to the overall goals of the project. To increase visibility, teams can consider using the following:
1. Checklists: a list of items to be completed during each phase of the SDLC. If the items in the list are marked as done, then there is no ambiguity about which tasks need attention.
2. Visual Dashboards: give a high-level overview of the project’s health and status. These can be created on whiteboards, easel boards or in software and during daily meetings, teams can address the dashboard to ensure everyone, including business stakeholders, are aware of the high-level project status.
Clear Definition of Roles and Responsibilities

Each person needs to understand his/her role on the team, as well as the tasks they must accomplish on a daily, weekly, monthly or quarterly basis. Goals, objectives and expectations for each team member should be defined and discussed with respective peers and/or managers. This eliminates confusion and increases productivity and task completion.

Empowerment

In an age where individuals are increasingly more technical and skilled, what they still lack in their job functions is empowerment and autonomy. Contrary to the conventional wisdom that a project needs a single leader, the leadership responsibility can be effectively divided within each role on the team. Having one point of contact each from the design, development and testing teams means each can help to lead their respective sub-teams and help to ensure everyone within the project is working toward the same goals and objectives. By operating within this structure, the whole team becomes empowered.

Experimentation

Once sprints are completed and knowledge gaps have been identified, the whole team should come together to discuss solutions to existing problems. Based on this discussion, the team can start experimenting with different solutions to determine what works well or doesn’t. This constant experimentation and feedback loop helps to make the team more creative and empowers them to come up with solutions that work for their organization.

In summary, the knowledge gap has been a major obstacle for development teams in reaching their fullest potential. By identifying and reducing these gaps, teams can be more efficient and achieve faster release cycles with higher quality software.

— Raj Subramanian
August 23, 2019