Tag: development

Implementing DevSecOps in the Mainframe-Driven Enterprise

DevSecOps is emerging as an important trend in corporate IT. It makes sense. With each passing day more applications serve a larger user base. Whereas in the past the typical business application was accessible only from within the confines of an enterprise’s internal network, today, the software-as-a-service (SaaS) paradigm enables millions of users work with a single application.

In the SaaS paradigm, a single application will service a number of companies and give each user in the company access to data and functionality based on a given company’s subscription plan. Take Saleforce.com for example: There is but one Salesforce.com providing service to thousands of companies worldwide, each with one to thousands of employees. Each company and each employee within a company will have particular access rights to the application. A paradigm in which there is one application servicing millions, perhaps billions of users requires a comprehensive approach to security. The slightest breach can result in havoc. Imagine what would happen if a SaaS application contained a flaw that allowed a company to see the financial data of its competitor that also subscribed to the SaaS.

Given that the internet is the de facto network for all worldwide, today’s security concerns go way beyond restricting network access. Keeping bad actors away is becoming more difficult all the time. The sad fact is that a seemingly good actor can turn into a bad actor instantly and wreak havoc not only on the servers with the network—as is the case of a DDos attack—but on the application layer as well. In fact, according to IBM, an enterprise’s network layer is half as vulnerable as the application layer, yet more money gets spent protecting the network.

This is an eye-opening fact and one not on the minds of most application developers. But, it might be, if developers had the benefit of the expertise and ongoing guidance required to implement bulletproof security at the application level.

Which leads us to the mainframe.

In today’s world of modern, web-based cloud computing, the mainframe is a major player. Seventy percent of corporate data lives on a mainframe. Also, now that the Linux operating system is pervasive in the mainframe environment, enterprise-level application developers can—and do—write Java, Python and NodeJS code for the Big Iron just as they do for X86 commodity environments. Technologies such as IBM’s z/OS Connect EE and StrongLoop make it possible to use an industry-standard specification such as OpenAPI to expose mainframe services as REST API without a lot of low-level fiddling around.

This is good news.

The bad news that application developers are just not that adept at security best practices. They don’t have the experience of a CISSP professional who has spent years, if not decades, in the trenches battling the bad guys in cyberspace. But that was then and this is now. Today we have DevSecOps.

DevSecOps is the next generation of IT transformation. The goal of DevOps is to create an IT culture that is unified, automated and self-correcting. DevOps combines development and operations into a single organization focused on creating and deploying quality software at ever-increasing rates. DevSecOps adds security into the mix. The scope of DevSecOps activity is not confined to software development for commodity-based X86 servers and mobile devices. DevSecOps is intended to apply across all facets of the enterprise, from mobile to mainframe.

DevSecOps is about a lot of things. It’s about the adoption of tools such as IBM’s AppScan, a tool that allows an enterprise to implement and monitor a variety of tests (web, mobile, dynamic, static and interactive) in a unified dashboard. It can include leveraging Secure Service Containers to ensure external security and safety from insider threats.

DevSecOps is about putting in place a set of policies, along with appropriate operational checks and balances, that ensure that the enterprise follows security best practices at all times. It’s about following practices that go beyond episodic security testing. This means continuously analyzing the IT infrastructure to identify vulnerabilities as well as defining ways to mitigate risks and prioritize prevention activities accordingly. It means making compliance to regulations such as GDPR, PCI-DSS, PA-DSS, ISO 27001 and ISO 27002, HIPAA, GLBA and Basel III an enterprise-wide responsibility, not just something that’s siloed away to the Compliance Department.

DevSecOps is about acting proactively by using automation to rigorously inspect all code to uncover potential hazards. It means using HTTPS as the default protocol for public-facing URLs. It means making sure that all third-party components are built from source code when possible—and, if building against source is not possible, ensuring third-party components and libraries are certified safe by an accepted authority.

DevSecOps is about a lot of things, but there is one principle that overrides all others:

When it comes to DevSecOps, the most important principle to follow is to make sure that a security expert is a starting member of any team involved with making software for the enterprise.

It doesn’t matter if the company is 30 years old running on a mainframe or 30 days old using native cloud services, there must be security experts as part of its development work from the start.

This may sound obvious, but it’s easier said than done.

It’s not a new challenge. It took the enterprise a long time to get QA personnel involved at the start of development process. Before the DevOps/Agile sensibility took hold, QA personnel showed up at the end of the release cycle, well after code was hardened.

The same is true of release personnel. Before DevOps, release management was given some code and told to just get it out as soon as possible. There was limited knowledge as to what the code was about or the potential impact it might have on the infrastructure. As a result, weekends were spent in war rooms struggling to deploy code that was sparsely documented and mysteriously tested. You can bet that once release management showed up at the first project planning meeting, things changed.

Now, with the emergence of DevSecOps, it is security’s turn in the tumbler. Security has always been an important part of enterprise IT. The number of compliance regulations on the books attest to the fact. However, security was usually the responsibility of the Compliance Department more than part of the IT organization overall. For many companies, the only time security ever came to the forefront was during audit time. As a result, security practices were more reactive than proactive. And, for many companies, they still are—think back to reactions to the Equifax and Target breaches.

However, the success of DevOps has not gone unnoticed. The standard features of DevOps—unifying groups toward a common purpose, implementing automation whenever possible and using sprints and retrospectives to facilitate fast releases and address shortcomings quickly—has had positive results. Forward-thinking companies understand that DevOps works, and its successor, DevSecOps, will work provided the essential principle is followed: Make sure security experts have a seat at the table from the beginning.

For many companies this means transforming from a culture of isolated silos in which managers serve as gatekeepers and powerbrokers to one of unified open groups in which a manager’s job is to assemble self-directing groups and to make sure that all the right people are involved from the beginning of a project. It’s a fundamentally different way of doing business.

When it comes to realizing the promise of DevSecOps, tools are important. So are processes. But, the most important thing is to make sure that security is built into the fabric of the development life cycle, from start to release.

DevSecOps will fill the missing link in the software development chain. It has to. There is little choice otherwise. Adhering to the most important principle of DevSecOps—to have security involved as an equal player from the beginning of the process—will go a long way toward ensuring that all software, from mobile and IoT to native cloud installations and mainframe instances, will provide the degree of safety and reliability required to serve the needs of users and companies in the digital economy of the modern world responsibly.

— Bob Reselman

August 9, 2018
Speed NPM Releases and Gain Confidence Using an NPM Registry
Building applications in the NPM development life cycle can be very complex. Let’s review the main factors that contribute to this complexity:
- An endless number of variables: Building applications involves a huge number of variables. Not only are there locally installed dependencies in the package.json (which should use absolute versions for each dependency), but there are also environment variables and global system settings.
- Different environments across multiple builds: It can become impossible, especially in larger projects, to maintain the same environment between builds. Even if the same host is used for both builds, local system changes between builds can have a dramatic impact on the published packages. In addition, having a master-slave continuous integration (CI) process further complicates this problem.
- Lack of trust in your binaries: When you pull an internal package in an application, you need to be confident that the binary you are using has been thoroughly tested. Otherwise, it will be the first thing you’ll be blamed for when their project fails.
In this blog post, I will show you the best practices to apply during the npm build process by using an npm registry to ensure that the same binary passes QA and makes it to the production repository.

Streamlining Your NPM QA-to-Release Pipeline

You need to have the following set up prior to creating your streamlined pipeline:
- A repository manager recommended for binary management.
- An automated build server, for example, Jenkins.
- Staging and release environments to be configured as site hosts similar to the production environment, where automated load and quality tests can be performed as per your QA team’s requirements.
- An npm project.
Stage 1: Software development

During the development process, run an automated QA cycle to ensure that the binary qualifies this stage.

Stage 2: Source code deployment

In this stage, complete the function and merge it to your Git branch. This triggers the Jenkins npm install to build the package.

Stage 3: CI server pulls dependencies

The CI server pulls the dependencies when you run ‘npm install’ command.

Note: To simplify the project, use a Binary Repository Manager to store the exact binary dependencies used in the project.

Stage 4: Deploy the Binaries to the target binary repository

Running the ‘npm deploy’ command causes the CI server to deploy the produced binaries to a target binary repository manager repository.

Note that at this stage, the binary is untested and is tagged as a “snapshot” build. As the package is untested, it should be moved to a non-production “snapshot” or “staging” repository where developers can use these packages at their own risk.

Stage 5: Run a second cycle of automated QA on the package

I recommend performing a second round of automated QA in addition to the development stage to ensure that package passes this stage.

If the package passes the above tests, it qualifies to enter the staging environment. The same package is “promoted” from snapshot mode to staging. After the app successfully meets the intended basic functionality, it can be deployed to the staging environment.

Stage 6: Deploy the package to a staging environment

This is typically where the development split occurs happens.

A common bad practice would be to rebuild the application after it passes QA, producing a new binary that differs from the original package.
I recommend using the same .tar.gz when promoting to later build steps.

Perform any QA scripts, which may differ depending on the project. The tests running in the staging environment should closely reflect the real environment. This can also include beta and manual testing.

Proceed to move the pulled binary “Staging” repository.

Stage 7: Deploy the package to a release repository

Once again, it is bad practice to rebuild the same package using the same source code but in a different environment during this phase.

The intended developers, who will eventually be using this package, need to be sure it is the same package that has passed the entire round of testing. Otherwise, you will be the one to blame if their application fails because of using this package.

Hurrah! Your npm registry is up and running.

The npm registry DevOps pipeline is now simplified by using the same binaries during the entire process. To conclude, these are the best practices that are recommended to follow when creating your npm registry:
- Ensure the exact same binary that was built makes it through the entire process.
- Instruct project developers to provide meaningful names to repositories. For example: “Snapshot” = Untested.
- Do not re-upload binaries that have been tested. Testing can modify the underlying binary and have unexpected changes.
Try creating your own npm registry using my customized projects as an example framework:
— Patrick Russell
August 8, 2018
Siemens Acquires Mendix to Advance Low-Code Platform Adoption

Siemens this week moved to substantially accelerate its transition into becoming a software company by acquiring Mendix, a provider of low-code application development tools, for $730 million in cash.

Mendix CEO Derek Roos said Mendix will operate as a business unit of Siemens, which has poured more than $10 billion over the last decade to software development as part of an ongoing digital transformation to a multibillion-dollar conglomerate.

Siemens over the last year determined that Mendix was now a strategic component of that initiative, Roos said, as more applications are shifted to and developed in the cloud. Many Siemens employees already use Mendix tools to develop custom applications with limited to no programming skills. That capability is especially critical in line of business (LOB) organizations that are starting to deploy their internet of things (IoT) applications, he noted. Mendix also recently revealed it is starting to infuse its low-code application development platform with artificial intelligence (AI) capabilities.

Roos said the $730 million investment Siemens is making to acquire Mendix is larger than all the venture capital investments made in rival companies delivering low-code platforms combined. As low-code tools have gained mainstream adoption, however, Mendix increasingly finds itself competing directly with larger rivals such as Microsoft and Salesforce, he noted.

The Siemens decision to acquire Mendix comes at a time when archrival GE is throttling back on digital process transformation efforts. Roos said GE’s decision is unique to structural issues within GE versus being a decision that reflects loss of faith in digital process transformation driven by custom application development. Software is now the fastest growing segment of the Siemens conglomerate, he said.

A recent survey conducted by Mendix in collaboration with ASUG, an end user association for organizations that have deployed software from SAP, finds 64 percent of respondents are experiencing increased demand for custom applications within their organizations. The primary problem is that same survey finds the average custom application development backlog is 24 months. Primary issues cited by survey respondents include high development costs (50 percent), difficulty integrating with existing systems (47 percent) and lack of skilled development resources (45 percent).

Of course, as Siemens becomes a larger software force, some organizations will start will take into consideration how much they may compete with the company before selecting an application development platform. Mendix is betting that as application development expands well beyond internal IT organizations, its status as an independent business unit of Siemens will provide enough distance between Mendix and Siemens business units that a Mendix customer might compete against.

In the meantime, organizations that are embracing low-code platforms are simultaneously increasing their investments in DevOps processes. As the number of applications being developed using low-code tools increases, there is a corresponding increase in application release management that requires them to modernize those processes. As Mendix plays the role in enabling applications that can be deployed from the cloud to the network edge, it’s now only a matter of time before the Mendix low-code platform becomes a more significant element of a larger set of DevOps processes driving digital process transformation projects as unprecedented levels of scale.

— Mike Vizard

August 2, 2018

Threat Modeling: The Why, How, When and Which Tools

You may have heard of threat modeling as a structured activity for identifying and managing threats. Threat modeling can be applied to a wide range of things, including software, applications, systems, networks, IOTs and business processes.

Threat modeling’s motto should be, “The earlier the better, but not too late and never ignore.” Without threat modeling, your security is a gamble—and in today’s business environment, you’re sure to lose.

When you design an application, you will face several security issues during different phases of the software development life cycle (SDLC), and so having threat modeling in the SDLC from the beginning can help to safeguard that applications are being developed, with security built in.

Simply put, threat modeling is a procedure to identify threats and vulnerabilities in the earliest stage of the development life cycle to identify gaps and mitigate risk, which guarantees that a secure application is being built, saving both revenue and time.

Why Threat Modeling?

It is better to find security flaws when there is time to fix them.
It can save time, revenue and the reputation of your company.
To build a secure application.
To bridge the gap between developers and security.
It provides a document of all the identified threats and rated threats.
It offers knowledge and awareness of the latest risks and vulnerabilities.

How To Do Threat Modeling

Many people think only security engineers can do threat modeling. That’s not true. Anyone, from developer to software project manager, can threat-model. In fact, I would suggest they should also know a little bit of threat modeling as part of their work.

Let’s look at the elements of threat modeling:

Assets: What valuable data and equipment should be secured?

Threats: What the attacker can do to the system?

Vulnerabilities: What are the flaws in the system that can allow an attacker to realize a threat?

Steps to Threat Modeling

Step 1: Identify the assets (database server, file servers, data lake stores, Active Directory, REST calls, configuration screens, Azure portal, authenticated and anonymous web user, Azure AAD client apps, database users, DB administrators)

Step 2: Outline details of architecture on which the valuable asset is being processed. It may include the software framework, version and other architectural details (ASP.net web application connection to cloud data stores and third-party services using JWT tokens).

Step 3: Break down the application regarding its process, including all the sub-processes that are running the application. We create a data flow diagram (DFD).

Step 4: List identify threats in a descriptive way to review to process further.

Step 5: Classify the threats with parallel instances so that threats can be identified in the application in a structured and repeatable manner.

Step 6: Rate the severity of the threat.

Threat Modelling Methodologies

STRIDE (Uses application-centric approach)

Spoofing of user identity
Tampering
Repudiation
Information disclosure (privacy breach or data leak)
Denial of service (DoS)
Elevation of privilege

PASTA (risk-centric approach): Process for Attack Simulation and Threat Analysis

TRIKE (risk-based approach with unique implementation and risk-modeling process)

VAST (Visual, Agile and Simple Threat modelling)

OCTAVE (focused on assessing organizational (non-technical) risks that may result from breached information assets.)

When to use Threat Modeling

In simple words, at the early stages of the SDLC, perform threat modeling:

Every time there is a change in the system’s architecture.
After a security incident has occurred or new vulnerabilities are introduced.
As soon as the architecture is ready.

It is not a mandate to perform threat modeling at the early stages of the SDLC; you can still pick up threat modeling at any stages even if it is close to deployment.

Threat models are continuously changing, and the models you prepared today may not be efficient tomorrow. And it is difficult to say that you are covered from all the threats. But if you have performed threat modeling and done whatever it takes to minimize your exposure to security risks, at least the impact of something very bad happening will be manageable (again, hopefully, but not a guarantee).

Tools to Perform Threat Modeling

I have used two tools for threat modeling, both of which are free to use.

OWASP Threat Dragon
Microsoft Threat Modelling Tool 2016.

Let’s take a look at both the tools:

Microsoft Threat Modelling Tool 2016	OWASP Threat Dragon
Full version available for free (as of now)	Alpha version available, flaws are still there. It is an OWASP incubator project, so it is at its early stage.
Installable desktop application available.	The tool comes in two variants: an online web application for GitHub and second an installable desktop application.
Documentation easily available on Microsoft website. Default threat template is available.	You have to search, default threat template is present
I found the UX better, unique methodology available, the user can understand threats better. Guidance and feedback in drawing a model.	Good UX. Fun and engaging user experience.
Presently using this tool for threat modelling.	Maybe I will use. I think will continue to improve.
End-user documentation is available.	End-user documentation is also available. Automated threats.
Threat Grid, Template editor, Mitigating existing DFDs.	Mitigating existing DFDs. Powerful threat mitigation. Rules have not yet been coded as it is the Alpha stage.
Comes with a base set of threat definitions using STRIDE categories.	Even in its current stage (alpha), you can create threats from those STRIDE categories.

— Debarghya Pandit

July 25, 2018

Connectivity as Code: Making Network Security DevOps-Friendly
DevOps is all about agility, with fast, short delivery cycles and automation for software development and applications. Enabled by recently introduced technologies such as virtualization, cloud and software-defined networking (SDN), spinning up new servers, provisioning storage in a public or private cloud or even launching whole environments can take just minutes or even seconds. But if that new application, service or environment needs a change in network connectivity or firewall rules to enable it to work, then the pace of delivery can often slow to a crawl.

Unlike the typical DevOps scenario, in which developers create a CI/CD pipeline that includes running build processes and tests and delivering the new application completely automatically, provisioning new network connectivity usually involves a slow, laborious process. The application developer needs to manually open a change request and then wait for approvals and implementation of the new connectivity flow before proceeding with the DevOps flow.

The Application Development Security Disconnect

As part of this, the developer is usually required to provide information about firewalls, zones, subnets and other information related to the underlying network infrastructure in these change requests—information that is not always known or clear to the application developer. On the other hand, the application requirements and context are not always understood by the network security team assigned to implement the changes. This can result in back-and-forth communication between the developer and network security, making the entire process extremely long, error-prone, inefficient and frustrating to both teams.

The challenge here is building in the necessary checks and balances to ensure that DevOps processes are not rapidly introducing security problems, while equally ensuring that security is not stepping on the brakes and slowing down the organization’s agility. So, how can this be achieved?

The Solution: Connectivity as Code

The answer is connectivity as code, a methodology that supports not only automation and agility in the application delivery process but also bridges the gap between application developers and network security teams on an ongoing basis, even after the application is deployed into production.

Connectivity as code creates an abstraction layer that translates between the two worlds, giving application developers more control over their applications while helping network security understand the business impact of their day-to-day tasks, and thus ensuring business continuity.

Establishing Connectivity Requirements

In the first stage of this approach, the application developer describes the application connectivity requirements in a simple machine-readable text file, listing all the logical flows that represent these requirements, including those for the different application environments (e.g. test, development, production, etc.). It is presented as a list of abstract flows that typically include the source (who initiates the connection), destination (who accepts the connection), service and additional information.

There is no need to know where the servers are located or what the underlying network topology is—such as whether there is a firewall between them or cloud security controls—or even their IP addresses or subnets. Furthermore, there also is no need to know whether connectivity is already available, because it’s also required by other applications.

The Connectivity Phase

Once this list is created, a new phase in the DevOps process, the connectivity phase, will automatically take care of things with no further involvement required from the application developer. The connectivity phase is where the actual connectivity provisioning and validation happens, leveraging a network security policy management (NSPM) solution to automate the network connectivity provisioning as part of the DevOps process just like any other configuration or provisioning step in the DevOps pipeline.

The NSPM solution will read the file before automatically checking whether the connectivity requirements have changed. If no changes were made, the solution will verify that the required connectivity remains in place, and that the application will work correctly. This provides the application developer with the assurance that the necessary connectivity is in place and connectivity-related failures are not anticipated when moving into production.

Automating Connectivity Changes

If the connectivity requirements have changed, the NSPM solution will use the connectivity file to provision the required connectivity while retaining it for future reference, so that the application’s connectivity remains intact in the event of future network architecture changes or application or server migrations.

If one or more of the required flows is currently blocked, a security policy change request process is triggered. Through this process a NSPM will verify that the new flows comply with the organization’s preapproved security policy, as well as security best practices and industry regulations, and will then design and implement the required changes directly on the different security devices on the network, automatically and within minutes.

If, however, a change request is non-compliant, it will be escalated for approval and, once approved, it will be implemented automatically, thereby saving the developer the need to manually open an out-of-band change request.

Furthermore, whenever a new version of the application is developed, this list of connectivity requirements is updated by the developer and the connectivity phase will ensure that the new flows are provisioned. With the connectivity phase completed, the DevOps process can securely proceed onto the next phase.

The Benefits of Connectivity as Code

Introducing the connectivity-as-code methodology into the DevOps process achieves a range of benefits. These include:
- Seamless management of network connectivity: This makes DevOps process far faster and more agile, and enables problem-free application delivery, rather than as an external out-of-band issue that requires separate—and manual—handling.
- Continuous compliance and auditability: Throughout the application delivery process organizations will remain fully compliant with all relevant guidelines, while all changes are fully logged for auditing purposes.
- Business continuity: Application connectivity requirements are clearly documented and up to date, ensuring minimal disruption to the business even during network, infrastructure or architecture changes.
Ultimately, connectivity as code bridges the gap between application developers and network security throughout the entire application life cycle, from planning and development through to deployment, production and decommissioning. It ensures both developer and security teams get what they want: Dev teams don’t have to worry about security slowing them down, and security teams know that risk and compliance checks have been built in to the continuous delivery process, complete with a full audit trail of changes. By weaving network connectivity into the DevOps process, organizations can ensure fast delivery without any compromise in security or compliance.

— Anner Kushnir
July 13, 2018
Why Digital Transformation Can’t Happen without Experimentation

A sure sign of a company’s successful digital transformation is when it can leverage data to optimize every area within the enterprise. As Forrester puts it, for a company to realize the return on investment (ROI) of a digital investment, “benefits from digital investments need to be aligned with the business requirements for a specific value chain stage.” All along, data-driven insights need to be tied to measurable gains and real-world business key performance indicators (KPIs).

The reality, however, is that data-centric principles are easier to apply in some areas of the business than others. The ethos has certainly taken hold in the development world—with its whole culture around DevOps, continuous delivery (CD), continuous integration (CI) and related agile methodologies that emphasize constant, data-driven experimentation and frequent reassessment and adaptation of plans.

Unfortunately, too many companies drop the innovative mindset once things progress to production and customer rollout phases. In doing so, they fail to spread the ROI from digital investments more fully across the organization. In other words, they fail to mount a genuine digital transformation.

True digital transformation requires companies apply a stronger, more agile approach to product adjustments across the value chain. Nowhere is this more critical than the need to embrace “full-stack experimentation” to realize business value from the world of data-based product development and testing.

The Limits of A/B Testing

Especially in an era when the customer experience sits at the center of the value proposition for most industries, the ability to fine-tune everything about how we bring products and services to market—and how our customers use and feel about those products and services—is mission-critical, especially in the quest for true digital transformation.

Unfortunately, no matter how agile and data-driven the development phase may initially be, we lose visibility, insight and, most importantly, revenue whenever we fail to infuse those same values into the later stages of production, QA and analysis around how customers are actually using what they buy.

Perhaps the biggest sign of this is the over-reliance of marketing teams on signals from basic A/B testing.

Industrywide, A/B testing is the most common approach to experimentation, but it’s far from ideal. A/B testing is typically used by marketing teams to test front-end changes at the user interface-level. It’s limited in its efficacy, as it does not take into consideration the deeper statistical nuances and resultant effects across the application stack—for instance, swapping out code to roll out an entirely new feature to a small set of users and measuring the results of that single deployment across both business and system metrics.

Worse yet is the false sense of security that can come from the narrow scope of A/B testing. Any outcome can be easily misunderstood as the only improvement possible. That leaves countless other variables and opportunities for ROI unexamined. If Plato remarked, “The unexamined life is not worth living,” what does that say about the unexamined product development process? How do we know if it’s worth building?

A Better Way

While A/B testing is still appropriate for some limited uses, leading-edge companies are learning to expand inquiry across the “full stack” of their applications. Full-stack experimentation differs from traditional A/B testing in scope, purpose and the potential impact on a business. It examines not just user interfaces but also execution logic, technical services, APIs and the data infrastructure underlying it all. This fosters a market-driven approach that brings more insight and less guesswork to decision-making.

To be clear, full-stack experimentation does not optimize part of a product or feature; it is about optimizing the process around experimentation and allowing development to more directly support and inform the evolution of the business. In this way, full-stack experimentation helps gain deeper value from the organization’s use of agile, DevOps and CI/CD so we can garner critical insights into how a product or feature is actually used by customers. More importantly, it measures the impact of every release on important key business metrics—which may be radically different from the developer’s intent.

Full-stack experimentation unlocks value across the entire data ecosystem. It involves collaboration among the key stakeholders involved in the development of a product or application, from engineers to product managers, as opposed to primarily marketers. You can use full-stack experimentation to answer deeper questions and test deeper assumptions. For instance, does adding the ability to tag friends in a photo increase their likelihood of posting photos in the future? Does this result in more sessions per user and does that ultimately impact the ad revenue generated per user?

Delivering Digital Transformation to More Companies

The benefits of full-stack experimentation are well-known among organizations with the resources to build and manage this in-house. Microsoft, Etsy and Netflix, for instance, test everything from multiple angles, sometimes running 50 tests a day or more on a single feature. These companies have powerful instrumentation to capture and analyze the multitude of variables and metrics—including those that may have been unanticipated. This surfaces valuable insights to be quickly spotted and harnessed for value.

Fortunately, modern software-as-a-service (SaaS) solutions are increasingly saving companies from having to create their own in-house, custom-built systems to get the benefits of full-stack experimentation. Unified platforms now exist that help engineering and product teams accelerate the pace of product delivery and make data-driven decisions without investing valuable development resources to build the tools themselves. Critically, these platforms have robust feature flagging and extensive experimentation capabilities that allow testing to happen well into production and rollout to customers—with strategic and incremental tests to examine and adjust logic, APIs, UX or other elements across the entire application ecosystem.

On a tactical level, these new platforms make experimentation processes simpler and more accessible to more companies. But the bigger picture is that these new options for full-stack experimentation effectively remove the friction that exists in achieving true digital transformation—allowing more companies to completely revolutionize how they do business.

— Kavitha Mariappan

July 13, 2018
How Automated Testing Can Help the Latest C-Level Issue: Quality Software
Every year developers gather in San Jose for Apple’s annual WWDC, where software quality is typically a key point of discussion. This year, the company progressed from its 2017 buggy software debacle in full force—it’s a heads-on effort, marked by the industry leader naming 2018 its year of software quality.

This raises some larger questions around the software industry itself. Has modern IT become so complex that the concept of “bug-free” software is no longer realistic, particularly for a company the size and scale of Apple? Or did Apple recognize this pain point and shift its focus to software quality in an effort to be the harbinger of change?

Of course Apple isn’t alone when it comes to the struggle with quality. Buggy software from organizations across the board persists today, but it simply cannot be ignored. Poor quality software creates long-term effects of waning customer confidence, brand erosion and revenue loss. These are very real concerns for the entire C-suite. Quality is no longer an issue limited to Dev or Ops—it’s critical to every organization’s success.

For example, consider the fact that 88 percent of consumers say they’d abandon an app if they encountered bugs or glitches (source: QualiTest). Nearly half those people are likely to discontinue using an app entirely if they experienced a bug every day. And 32 percent say they would abandon an app the moment they encountered a glitch.

The cost of acquiring a new customer is 5 to 25 times higher than retaining an existing one. Getting software right the first time through a culture of continuous testing is critical to every business, as it translates directly to retaining customers.

Budgets for QA and testing are expected to rise to 40 percent in 2019 from 31 percent this year. However, one-third of developers still admit they don’t have enough time to properly test before software deployments. As everyone in this business knows, developers run fast. It is of the utmost importance to ensure software actually works well and will provide the expected customer experience. The garbage in, garbage out model is not an option. To fill the gap, more and more organizations are turning to automated testing.

Any repetitive, labor-intensive task is typically a good candidate for automation, including those associated with testing. Automated testing involves execution of prescripted tests on a web or mobile app, with test data accessible in detailed reports to help development teams quickly identify issues in the code, as well as any issues in the automated scripts themselves.

Performance of the automation framework and its scripts is crucial, and should be a high area of focus. Prescripted tests fall into different categories that comprise the test pyramid. They include:
1. Unit Testing
2. Code Coverage
3. Code Syntax Checks
4. Security Tests and Checks in Code
5. Integration Tests
6. UI Tests for Web, Hybrid and Mobile Apps – on multiple cross browsers on various platforms and OS versions
Once created, automated tests can be run again and again at no additional cost—which is particularly helpful when source code is changed. Since it is much faster compared to manual testing, automated testing translates into considerable time and cost savings.

Automated testing is especially valuable given the breadth of browsers, operating systems and devices in the marketplace. Consumers today run apps on a countless number of these, so developers must have access to every combination imaginable to deliver the highest level of performance. What good is your app if it doesn’t run (and, by extension, monetize) properly when it gains popularity in some part of the world with users on a different platform from the one you tested on?

Ensuring that software works on as many options as possible allows a business to reach a maximum number of users. This is where automated testing can help, by executing thousands of different complex test cases during every test run. Testing efficiency can be achieved by executing tests in parallel or grid, and executing CI/CD jobs in parallel. For example, one code is deployed in staging, teams can simultaneously start jobs to test against browsers and cross-browsers, apps and localizations, etc. The result is attaining coverage that would be impossible through manual tests. Teams can scale apps fast, maximize CI/CD workflows, and ship higher quality software faster.

Attaining CI/CD and automating all the tasks involved in creating a real-time test system is one of the arenas in which the next big software battle will be fought. More importantly, as software continues to eat the world, the C-Suite can no longer ignore the impact of poor quality software devouring your business. Apple’s bold step of identifying 2018 as the year of software quality is a call for other organizations to do the same.

Automated testing is a key piece of the solution. How is your organization meeting the software quality challenge, and by extension, the evolving standards of customer experience?
— Ritesh Arora
July 6, 2018
There’s No Such Thing as DevSecOps
Should DevSecOps be categorized separate from DevOps?

After years of debate, few doubts remain about the viability of the public cloud for enterprise workloads. Enterprises are shifting to cloud-first and cloud-only strategies, looking to take advantage of all the good the public cloud has to offer. For applications built and deployed in the public cloud, it is all about automation. The cloud enables you to be elastic, reduce costs and deliver products faster. To achieve these goals, you need to automate your processes—building, testing and deployment.

Security Challenges of DevOps

This need for faster innovation led to the rise of DevOps, a set of practices and tools that result in tighter collaboration between development and operations. In the past, controlling the deployment processes was the responsibility of dedicated teams that wrote scripts. However, with DevOps, development and deployment merge. Software engineers must do more than write their infrastructure and business logic code; they also need to write the automation that surrounds it and take responsible for deploying it through the different stages until production.

However, even as development and operations come closer together, security has remained a function apart. While DevOps workflows are heavily automated, security checks for application and infrastructure code often are manual. DevOps and security teams are measured differently—delivery versus protection—and often are at odds with each other. The siloed approaches to security hardening that worked in the past are incompatible with the holistic, iterative software development and deployment model of DevOps.

For a developer, adding the responsibilities associated with DevOps is not complicated, but securing the environment is a different level of challenge that requires mindset shifts in multiple areas:
1. Knowledge: Software engineers are great at writing business logic. They write efficient code and develop highly available environments that can handle millions of concurrent users. However, the vast majority of developers don’t know about the latest CVEs or the proper configurations of SSL/TLS in the load balancer.
2. Pace: Successful development organizations move fast, frequently adopting new development trends and frameworks. Public cloud vendors release new features every week, and it can be difficult to follow and keep up, as engineering is usually measured by the ability to deliver and the quality of output. DevOps enables rapid release cycles, and the security must not hold them back.
3. Scale: Modern applications make the transition to work with microservices. When each development team owns its cloud environment (and sometimes multiple environments), the security teams suddenly find themselves dealing with huge distributed deployments. They can’t manually configure security when new features are released a few times a day in different environments.
4. Heterogeneity: Multi-cloud deployments are another emerging trend. New product teams are often given a free hand choosing their tools and infrastructure, and they may pick a new cloud vendor. While security teams have a voice in this decision, they do not have the last word. It may have taken the security team years to learn to deal with the existing cloud solution, and suddenly the organizational policy needs to be applied across different infrastructures.
5. Consistency: Enforcing organizational policy across different development teams is challenging. Each team has its own style, environment and development frameworks, meaning it’s never easy to ensure the security is applied uniformly.
However you choose to address all these challenges, one thing must be clear: Your DevOps cycle is not complete without handling the security.

Choosing a Security Solution

Clearly, you can’t deploy services without properly securing them—you don’t want anyone to hack into your systems and steal your (and your customers’) information. Since cloud environments are dynamic and deployments tend to span across dozens of accounts, manual steps are not an option. Automation is necessary. Whether you acquire an enterprise security solution, adopt an open source security project or write your own tools, you need to make sure that they are:
1. CI/CD ready: The solution can easily integrate into the continuous integration/continuous compliance pipeline. User-friendly APIs are required.
2. Scalable: Assessing hundreds of cloud environments requires the solution to be scalable and robust. When covering more cloud vendor capabilities, performance becomes critical and increasingly challenging.
3. Flexible: Each organization is different, and security needs can vary with each deployment. The baseline configuration of your security solution will never meet your requirements by 100 percent, so it’s important that you be able to adapt it to your needs.
4. Continuously updated: Cloud vendors add new features at a rapid pace, which also means new potential misconfigurations are generated all the time. Whatever solution you choose, it needs to match the cloud vendors’ speed and agility. Security solutions that are not being updated soon become useless.
5. Conveniently Reportable: Aggregated reporting is critical. Consuming the security findings in a convenient manner can assure that issues are properly mitigated.
6. Automatic remediation-capable: Many of the findings are easy to remediate automatically. Applying automatic remediation would enable the security team to focus on complicated tasks and reduce alert fatigue.
DevOps and DevSecOps Are (or Should Be) the Same Thing

Security is clearly an integral part of DevOps. DevSecOps is emerging to bring security and compliance management practices in line with the rest of DevOps. DevSecOps serves to call out the temporary chasm that exists between security and DevOps.

As security checks and controls get automated and integrated, DevOps and DevSecOps are converging to the same thing. Just like adjusting the mirrors and putting on the seat belt are part of standard procedure before pulling your car out of the driveway, assessing your security posture before releasing a new version of your product must become habit. To remain relevant, your security team must integrate into the CI/CD and make sure the newly created environments meet organizational policies.

— Offir Zigelman
June 7, 2018
Complementary Technologies: Amazon RDS and PostgreSQL

For enterprises in search of a relational database service that will effectively and consistently meet their requirements, Amazon RDS is probably worth a hard look. By making cloud-based relational databases straightforward to set up, operate and scale, Amazon RDS liberates businesses from the time-sink of internal database administration duties—in turn allowing for a clearer focus on product and business development. When it comes to selecting a database engine, organizations on Amazon RDS have a variety of choices, from Amazon’s own Aurora to MySQL, MariaDB, Oracle, Microsoft SQL Server and PostgreSQL.

While each of these Amazon RDS database options have their pros and cons, I’m a fan of PostgreSQL for a number of reasons. PostgreSQL offers a feature-rich open source object-relational database engine that’s backed by 15 years of dedicated community development, a proven architecture and a well-earned reputation for reliability and data integrity. PostgreSQL more than matches MySQL’s speed and feature set, bringing to the database table than a standard SELECT, INSERT, UPDATE functionality. Driven by particularly strong community and third-party support, PostgreSQL regularly releases stable updates that enhance its performance and expand its features—which now include native JSON storage, native arrays and the ability to create custom data types, functions, and deeply customizable extensions to fulfill specific application needs.

Utilizing Amazon RDS and PostgreSQL in tandem provides synergistic benefits, due in no small part to a number of design features intended to provide simplicity. Amazon Web Services includes a one-click interface for creating a PostgreSQL instance in the cloud. Amazon also offers its AWS Database Migration Service (DMS) integrated within the Amazon RDS service, which makes it easy to migrate an existing database over to PostgreSQL and import it as an Amazon RDS instance with zero downtime (and schedule the migration for whatever time a business finds most convenient). Amazon RDS also allows you to scale PostgreSQL databases up and down in the cloud within minutes; provides automatic replication for high availability and redundancy using Amazon Availability Zones; integrates easily with other AWS services such as Amazon Virtual Private Clouds; and makes it simple to set up and restore database backups using automatic snapshots.

Scaling, In a Snap

The pairing of Amazon RDS and PostgreSQL is well-suited to smooth growth as the demands placed on an organizations’ application increase, as the Amazon RDS platform allows you to scale instances both vertically and horizontally as necessary.

Vertical scaling in Amazon RDS can be accomplished as follows:

From the RDS dashboard, select “Modify” from the list of options for your database instance:

Then, choose a new DB instance class:

The process is now complete. Vertical scaling in Amazon RDS is that simple. Vertical scaling also occurs with little or no downtime when running a database using multiple availability zones, since the standby database will be upgraded first and followed by a failover to this upscaled database.

Additionally, Amazon RDS provides an easy method for improving read performance by increasing scale horizontally, using read replicas as read-only copies synchronized to the master database.

From the dashboard, select “Create Read Replica” for the instance:

On the next screen, select the instance size and the database instance name. It’s also possible to deploy read replicas to any region and availability zone:

In just minutes, increasing horizontal scale will be achieved:

In my view, businesses can realize all the benefits of a simple, affordable, and versatile fully-managed database-as-a-service solution by teaming up Amazon RDS and PostgreSQL—one that can be set up in just minutes and scaled to any size a growing organization may come to require.

— Fabricio Mariani

May 21, 2018
How Python is Transforming the DevOps Landscape

DevOps is the combination of operations and software development which reduces or almost eliminates the disconnect between systems administrators who run the infrastructure and software developers who develop applications.

During software development, when the Agile methodology is utilized properly, a bottleneck often occurs during the frequent operations and deployment phases. New fixes and updates are produced so rapidly in every sprint that the infrastructure teams can be swamped with deployments. To eliminate few of these issues, the operations personnel and application developers are asked to work closely to automate the code delivery to production from deployment.

As DevOps is an approach for handling change and agility, engineers must master multiple languages. The Python programming language is one of the most crucial components of the DevOps toolchain. Many DevOps teams utilize it for building web applications for visualizing custom data, building custom utilities and more.

Ansible and other tools are written in Python, which means you can build custom scripts or modules to automate your tasks, for example, or do other things.

Reasons for Using Python for DevOps

The accessibility and flexibility of Python are reasons why it is a preferred language for the DevOps toolset. If you are working in the role of DevOps, you must require an adaptable skillset; you must know how to work with multiple languages. For anyone looking to explore new tools and languages, and are curious about technology, the language can form a solid foundation, as it doesn’t require the commitment levels that a specialist language requires.

In addition, it is a great scripting language, and scripting refers to automation. A few major configuration management tools, including SaltStack and Ansible, are written in Python underscores. This shows how useful the language is when it comes to orchestration and infrastructure automation.

Python and Ruby are compared often, and with good reason: Both are highly accessible and utilized in applications developed by many organizations. They also feature in the DevOps toolset. We can do almost all the things with Ruby that we do with Python. But, when it comes to syntax, Python gains the upper hand, as it is much more direct than the syntax of Ruby.

In the realm of learning to debug problems and code, Python has the advantage over Ruby, as it takes a much more direct approach to programming. Its directness and simplicity are invaluable if you are working in DevOps environment and agility is highly important.

Conclusion

As a growing number of engineers and developers take on decision-making responsibilities, Python can be the stabilizing language. And not only for DevOps—the language also can be used for developing blockchain, which is the underlying technology for various cryptocurrencies such as bitcoin wallet. As it can be utilized in various ways, it enables you to be open to broad range of technical possibilities. Python is the go-to programming language for the DevOps landscape.

— savaramravindra

April 24, 2018