Many argue that application security should be the responsibility of a security team. However, while security professionals can contribute, developers are usually the only ones with the technical ability to fix software security vulnerabilities. The same goes for software compliance. When it comes down to it, only developers are equipped to build applications in compliance with specific software standards.
Considering the tight deadlines for development teams, giving them additional duties can present challenges. To help developers deliver secure applications without slowing them down, many teams have adopted DevSecOps—which encourages integrating automated security testing into the DevOps workflows for every release.
DevSecOps helps teams deliver more secure software faster, but what about more compliant software? Many teams trying to achieve software compliance encounter similar challenges as those trying to achieve software security. Luckily for them, they can use similar strategies.
What Does DevSecOps Do?
The fast, iterative software release cycles supported by DevOps workflows present hurdles for software security.
In traditional approaches to securing software, testing occurs toward the end of the software development life cycle (SDLC), after an application’s construction. As a result, most (if not all) of the application is tested at once, and a long, daunting list of security issues is sent back to developers.
However, many teams find this methodology incompatible with DevOps. Development teams don’t have the time—not to mention the budget—to stop whatever they’re doing to sort through a massive spreadsheet of issues.
To solve this problem, organizations can implement DevSecOps by integrating security testing into the shorter, higher-frequency feedback loops of fast DevOps release cycles. Rather than running one big test after developers finish building an application, DevSecOps calls for automated security testing performed early and often. By helping developers check in more secure code, DevSecOps helps reduce the volume of issues QA must identify and send back to them.
If development teams can integrate security testing into DevOps workflows, why not compliance testing?
How to Apply DevSecOps Practices to Software Compliance
While testing for security usually requires a different analysis than testing for compliance, development teams practicing DevSecOps can use a similar approach to compliance testing.
Like traditional application security testing, compliance testing typically happens in the QA environment. A very rough application development and testing workflow looks something like this:
Developers write the code for an application.
QA tests the code while developers move on to another project.
QA sends a list of compliance violations back to developers, requiring them to pause their current work to fix those problems.
There is nothing technically wrong with this compliance testing strategy. However, this approach is frustrating for developers and costly if the list of issues is especially long. To resolve more compliance issues before an application is complete, teams can implement practices inspired by DevSecOps to help developers commit compliant code earlier in the SDLC.
One of these strategies is to integrate automated compliance testing into DevOps release cycles. By testing the application in increments rather than all at once, teams can reduce the number of compliance violations per testing cycle. While it obviously requires higher-frequency testing throughout the SDLC, this approach has proven to be faster than waiting to find issues in QA.
Another approach is to create a sandbox environment in the integrated development environment (IDE) where developers can run tests themselves. Testing their own code as they write it equips developers to check in cleaner, more compliant code, so compliance violations are less likely to show up in QA—or worse, production. This approach has the added benefit of familiarizing developers with code that leads to compliance violations.
These strategies for identifying and resolving compliance violations are not meant to replace QA testing. However, by integrating compliance testing into multiple stages in the SDLC, teams will see fewer issues in feedback loops between QA and development—which better supports DevOps workflows.
What Does This Look Like in the Real World?
As with security testing, integrating compliance testing into DevOps release cycles requires new technologies and workflows, each with a different learning curve.
Regarding technology, static analysis is growing as a popular option to help development teams build compliant applications. Coverage varies between static analysis tools, but in general, the technology can find issues in code quality standards, such as MISRA and CERT C/C++, as well as security standards, such as OWASP Top 10 and PCI DSS.
The static analysis tool or any other technology that teams choose to address, software compliance plays an important role in the workflows they adopt. As discussed earlier, those hoping to keep issues out of QA will want to find solutions that can:
Integrate into the quick feedback loops between testing and development.
Scan code in the IDE as developers write it.
Creating a more continuous cadence of feedback between testing and development could represent a significant shift in a developer’s day-to-day.
While “DevCompOps” doesn’t quite have the same ring, development teams in regulated industries will find cost-cutting benefits by integrating compliance testing into DevOps release cycles.
As enterprises have increased their reliance on applications over the years, there has been a significant rise in the use of reusable software components such as third-party libraries and open source code. This makes perfect sense, as this development method makes it possible to add value to applications and other software offerings quickly and easily. After all, why reinvent the wheel when supposedly well-tested and proven software components are readily available?
As a result, open source code has become a vital part of software development for most organizations. Today, nearly 70% of every application is comprised of reusable software components, including open source software. At one time, open source software was considered too risky for commercial use. That is not the case today, however, as even major corporations are making use of open source components to innovate and deliver applications to market rapidly.
While enterprises have gotten very good at using open source to speed the application development process, they have not yet figured out how to do so without inadvertently introducing vulnerabilities into their code. This is due in part to the fact that as more enterprises increase reliance on applications, they fail to implement application security into the software development life cycle. Consequently, as the adoption of open source software continues to grow, the likelihood of applications inheriting vulnerabilities is higher than ever before.
This is evidenced by the fact that, according to my company’s research, unpatched library vulnerabilities make up 21% of all application vulnerabilities, by far the most of any type. In fact, nearly 20% of the inquiries we answer are related to unpatched libraries, which is almost on par with injection vulnerabilities.
That is why it is imperative for development teams to stay up-to-date with open source library maintenance and updates.
Software Composition Analysis
The right way for organizations to track their open source components is through a set of technologies known collectively as software composition analysis (SCA). These technologies, which arose out of a need to automate the open source management process, provide organizations with visibility into their open source inventory. This allows them to identify third-party and open source components integrated into their applications.
Using SCA, organizations can determine easily if any of their applications are relying on a vulnerable library, if any libraries are out of date and whether licensing is legal/proper. SCA helps them identify multiple types of risk introduced by third-party and open source components. These can include known vulnerabilities, intellectual property risk and technical debt related to component age.
It’s important to remember the use of open source software code is not a problem in and of itself. On the contrary, it has been an essential part of the recent innovations in software development, and, in fact, it is no riskier than proprietary or commercial off-the-shelf offerings.
Problems arise only when organizations fail to be proactive in identifying and managing security and license risks associated with open source code. This may occur in part because it is assumed that, as with commercial software, updates, patches and information are pushed to the user. But that’s not the case in the open source world, where the onus is on the user to keep track of these things.
It takes diligence, but in the end, the only way to mitigate both the security and legal risks associated with your open source code—and to fully realize its benefits—is to track every piece of it. Today, SCA is the proven way to help you do that. By incorporating SCA into the development process, developers can capture inherited vulnerabilities early enough to prevent them from being introduced. Of course, it should go without saying, but it’s not enough to just find vulnerabilities; you also must fix them. The most typical remediation is replacing vulnerable components with the non-vulnerable ones.
Considering most application code is now open source, and attackers are going after vulnerabilities in open source more than ever before, SCA should not be an afterthought. It needs to be an integral part of any organization’s application security and secure DevOps processes.
I’m thrilled to announce that Dustin Smith has joined DORA as an author on this year’s “Accelerate State of DevOps Report.” I met Dustin last year, when he joined me in a brainstorming session to discuss research ideas and design considerations. (If you peek back at the 2018 report, you’ll see him in the acknowledgments!) This year, his research and statistics background made a huge impact to the rigor of the report, in addition to his insights as a top-notch researcher. I can’t wait for you to see it. Stay tuned—the report comes out late August! ~Nicole
It’s a pleasure to introduce myself to the community! After contributing to the research design of last year’s report as a subject matter expert, it’s exciting to join forces with the DORA team and be a co-author of this year’s “Accelerate State of DevOps Report”! I’m a huge supporter of DORA’s data-driven approach and focus on factors such as culture, organizational policy and well-being, as opposed to metrics such as number of PRs and lines of code changed, which easily can be misunderstood or taken out of context.
During my tenure at Google I’ve studied developer workflows and experiences to help inform product and design teams. My background is in psychology with an emphasis on considering the systematic factors that influence an individual’s or team’s judgment and decision-making. I believe that people are influenced by their environmental constraints, and to help them be most effective, we need to understand those constraints.
This year’s report is packed with trends and new exciting findings. Be sure to check out the section on “What Really Works in Scaling Transformations” — it’s one of my favorites this year, and I led that analysis. It presents patterns we see among the highest performers, allowing teams and organizations to take an evidence-based approach to building and scaling their own DevOps transformation.
I look forward to sharing the results with you all soon!
One of the biggest problems in modern distributed computing is data management. These days data are spread out all over the place. As a result, getting the data you need, exactly when you need it, becomes a problem. Wider distribution creates more network latency. If you have a system in which some machines are in California and others are in New York, logic dictates it’s going to take more time for data to travel coast to coast than from within the same geolocation. It’s a matter of common sense.
The data management problems created by distributed systems became more pronounced with the growth of the internet, which is, by nature, highly distributed. Eric Brewer, a computer scientist at the University of California, Berkeley, published a paper in 1999 that describes the dynamics of data management in a distributed system as a set of immutable constraints. Brewer published the work as the CAP principal. Three years later Seth Gilbert and Nancy Lynch at MIT proved the principal to be a theorem. Today, the CAP theorem is a central consideration when designing and implementing applications at web-scale.
Simply put, the CAP theorem asserts that when your data is distributed over a variety of machines and geolocations, if you want the latest, most accurate data, you’re going to have to wait. And, if you don’t want to wait, the data you get will probably be stale. However, if data resides on a single machine, then everything changes. In such a case, you can have the latest data on demand, at any time.
Of course, having a single data source implies having a machine that supports the demands of web-scale. Sadly, the x86 machine has yet to be made that can fit the bill. But a mainframe can. And, not only can it handle an enormous operational load, under the right set of circumstances it also can be a cost-effective way to meet the need at hand.
Allow me to elaborate.
The CAP Theorem Explained
Before we go into the details as to why a mainframe computer such as an IBM Z or LinuxONE can make the problems presented by the CAP theorem go away at a reasonable price, let’s look at the theorem itself.
Essentially, the CAP theorem asserts that of the three aspects of data management in a distributed system—consistency (C), availability (A) and partition tolerance (P)—you can have only two at any point in time.
Figure 1: The CAP theorem dictates that of the three aspects of data management in a distributed system, only two are available at any point in time.
In other words, if you want the latest information (consistency) from your data source immediately (availability), that information needs to be stored in a single, fail-safe location (partition tolerance). If you want to spread the data out over many machines (partition tolerance), either the data will be accurate at some point in time but not immediately (consistency) or you can have data immediately but it might be stale, awaiting an update to the most recent state (availability).
The CAP theorem is more than theoretical. It has an actual operational impact on companies. A telling example is a problem that Netflix experienced when implementing its Monet system.
Netflix Gets CAP-Ped
Back around 2018, Netflix decided to implement an internal ad management tool under design using GraphQL, an open source API specification originated at Facebook that was becoming popular in many companies that operate at web-scale. In fact, GraphQL is still growing in popularity. So Netflix decided to take the plunge toward adopting GraphQL with an internal project named Monet.
The way Netflix decided to implement Monet was to have the GraphQL API server act as a facade for existing RESTful APIs. The thinking was to reap the benefits provided by GraphQL, yet reuse as much existing code as possible. The idea was good, the actual deployment fell short. Once Monet was put into production, it ran dog slow.
It turns out the REST APIs were distributed among a variety of data centers (see Figure 2). Network latency was killing them.
Figure 2: Initially Netflix’s Monet system queried data from a variety of data centers.
The company’s solution was to bring all the REST APIs into a single data center (see Figure 3).
Figure 3: Netflix improved Monet’s performance by moving all RESTful APIs into a single data center.
The result? Once all data resided in the same data center, performance improved dramatically. It makes sense. Shorter distance to travel means less latency in the system overall.
Now, how this relates to the CAP theorem is interesting. Remember, according to CAP, as partitioning goes down (P), consistency (C) and availability (A) go up. So, while it’s true that Netflix was not using a single machine as a data source, moving everything into a single data center created what one could consider a single virtual data source. Yes, there might be latency issues within the data center. But, rack-to-rack latency is nowhere near as high as the latency between geolocations. We’re talking the difference of 1 to 10 milliseconds as opposed to hundreds, if not thousands, of milliseconds.
In Netflix’s case, the company addressed the constraints imposed by the CAP theorem by reducing the breadth of partition at the geolocation level. All the data lived in the same data center. Imagine if partition size could be reduced at the machine level. What if instead of having data distributed among many machines within the same data center, data actually resided within a single machine? Partition size would be reduced to one. Then, applications could have the latest data immediately. All that’s needed is a single machine that can handle the load. This is where using a mainframe comes into play.
Mainframes Are Built for Big Data and Agile DevOps
Mainframes since their inception were intended to be able to store and process enormous amounts of data. Today 80% of the world’s data still resides on mainframes. As mentioned in a previous article about this topic, one mainframe can process 2.5 billion transactions daily. That’s the equivalent of 100 Cyber Mondays in a single day. That’s a lot of computing power easily equivalent to more than a few racks of commodity hardware in a data center. In fact, in some situations, one mainframe can provide the computing power found in an entire data center.
So, let’s go back to the Netflix problem. What if instead refactoring those distributed RESTful APIs into a single data center, the time and money spent was used to refactor those RESTful APIs into a single mainframe? Yes, there are some technical hurdle to overcome, but considering the running Linux on a mainframe is standard these days, it’s not as if we’re talking transforming a bunch of Python code into COBOL. Databases from standard RDBMS such as DB2,Oracle to the NoSQL MongoDB can run perfectly well on a mainframe. There’s work to be done, but reinventing the wheel is not one of the tasks. Using a mainframe is a viable solution except for one sticky little problem: The conventional thinking is that using a mainframe is prohibitively expensive. Thus, the bias toward racks of commodity hardware or simply going with a cloud service.
Mainframes as a Cost-Effective Solution
The general perception in most IT shops is only big banks, insurance companies, governments and large-scale manufacturers use mainframe computers. They’re just too expensive for small to medium-size business. While many mainframes come with a price tag that can exceed $1 million, there are still surprisingly affordable models. For example, an entry-level IBM z114 will run about $75,000.
If you’re a company spending $50,000 a month on AWS or Google Cloud computing resources, then for the $600,000 a year cost you’re spending on a cloud provider and the latency that comes with it, your company can well afford not only an entry-level IBM Z system to handle a production load, but also a second one to serve as backup, should failure occur.
In terms of system administration, just about all the tools commonly used to support modern CI/CD processes today in the DevOps world have corresponding equivalents for mainframe implementation. As with any adoption of unfamiliar technology, there is a learning curve. But, again, we’re not reinventing the wheel. The tools, resources and expertise exist. Adoption is just a matter of following standard change management practices typical in medium-to-large-size companies that are well-positioned and use a mainframe to take advantage of implementing a cost-effective, no-partition data source strategy.
Integrating Mainframe Technology into a DevOps-Driven Ecosystem
Mainframes are capable of doing DevOps in an open, agile and collaborative fashion. The tools, resources and expertise required to support a mainframe’s participation in CI/CD processes exist today. For example, Zowe is an open source software framework that provides solutions that allow development and operations teams to securely manage, control and develop on the mainframe like any other platform. DB2 DevOps Experience for z/OS is one of the first commercial products built on Zowe. It allows IT the control over the mainframe database, its business rules and standards, while allowing development teams the ability to work independently using self-services and integration with modern CI/CD tools to speed their application release time with better quality and development practices.
When you think about it, in certain mission-critical situations, using a mainframe to have the latest data at any time not only makes sense from a technological point of view, but it’s also attractive in terms of return on investment. The trick is to be able to identify the right use case. Of course, if you have a situation in which 90% of your users are in California and you have a mainframe situated in Bangalore, the laws of physics will apply. Moving the bits from the mainframe to user will take time. But, if you have situations in which user and data are in continental proximity, using a mainframe as a standard data source begins to look very attractive.
Again, having the right use case is key. But, everything else being equal, using a mainframe for sensitive, data-driven applications warrants significant consideration.
Putting It All Together
The constraints on distributed computing imposed by the CAP theorem remain a challenge to those who design and implement large applications at web-scale. The “pick two” limitation CAP imposes can be an annoyance for simpler websites that provide content intended for the general consumer. But, for large scale, data-intensive applications, having accurate data immediately is not optional, it’s essential. Thus, using mainframe technology to create a no-partition data infrastructure becomes a viable way to address the limitations imposed by the CAP theorem. Under the right conditions, making a mainframe the central repository for your application’s data provides a significant return on investment hard to match when compared to commodity-based cloud providers.
The nice thing about using a mainframe is when it comes to the CAP theorem, you still only get to pick two, but the two you get to pick are consistency and availability. In the scheme of things, those are the two that count.
Staying competitive in today’s fast-moving digital economy is easier said than done. Operating in a digital environment requires a significant shift in company culture and an agile software environment that accelerates time to value. Creating a DevOps environment helps companies keep pace, but it requires a workforce that is empowered, excited and motivated.
Gamification offers a unique opportunity to continuously engage your workforce by applying game-design elements and game principles into a corporate setting, making work more fun. Companies can develop a team-oriented Pod model that incorporates gamification to improve employee engagement and productivity, enable team members to work more efficiently, accelerate the learning curve and deliver more sophisticated products to market faster.
When I joined Cognizant Softvision as CEO, I wanted to empower our teams and promote a continuous delivery mindset by developing a delivery model that was customer-centric, differentiated and unique to each client without being prescriptive. Thus, Game of Pods was born.
The global, Game of Pods gamification program is designed to provide timely and superior customer service. At a high level, Game of Pods inspires and measures team performance and delivery while encouraging and monitoring individual performance. During competition, the Pod and its members are recognized and rewarded for their collective and individual contributions with badges and trophies.
We typically focus on the output of each pod, rewarding collective behaviors and achievements that help the pod reach higher efficiency and quality targets. Collective badges are awarded to the pod while personal and peer-to-peer badges are awarded to the pod members. Pod badges are earned as a team and are focused on the health of the project, performance of the pod, quality of delivery and alignment with client objectives. A leader board displays the badges each pod earns, their experience points and their trophies. Pod and individual recognitions are celebrated over email and across Facebook Workplace, and shared with all employees.
While perhaps not as intense as “Game of Thrones,” Game of Pods is the epitome of gamification in action and enables organizations to reach a higher level of efficiency and professionalism while challenging employees to achieve personal goals in a fun, team-oriented process. In addition, Game of Pods offers other benefits such as:
A flexible model that can work in any client environment.
Team empowerment that incentivizes teams and individuals to achieve more.
An approach that encourages diverse thinking that recognizes and rewards teams and individuals for improvement.
Game of Pods is most effective in organizations that embrace an agile mindset and understand the need for flexibility in today’s fast-moving environment. While gamification has come a long way, successfully incorporating it into the workplace can be challenging. However, integrating gaming into your organization can empower and excite employees, motivating them to overcome obstacles that might otherwise stand in the way of success and sustainable innovation.
Is DevOps in the enterprise real? The short answer is yes, but you may assume it’s only relevant to the world’s leading cloud companies because they’re the ones that make the headlines. DevOps offers the promise of agility, quality and ultimately better-performing software. While DevOps processes can increase agility, and agile approaches are a great help in improving responsiveness to change, the overall process still needs a clear objective, measurable KPIs and a finite scope.
In other words, unlocking the promise of an agile (product) organization requires a few complementary approaches. If those are ignored or applied incorrectly, DevOps agility can become an impediment to adaptability and agility for your enterprise.
Yet, I’ve read so many blogs about how the cloud and DevOps processes solve problems. It’s true they can, but the nuance is that, while a few teams can coordinate change amongst themselves, when you attempt to deploy a DevOps practice at scale, challenges are quickly apparent. For example, if I’m working for a startup that has fewer than 50 developers and very few layers of management between the CTO and her team, the management and communication lines are likely short enough that strategy and direction can be conveyed directly to the teams. However, for larger organizations—where there are many often geographically distributed teams with silos of responsibility and layers of management—far more coordination and communication are required.
In these environments, it is more likely the business will achieve agile teams but not an agile culture. This is because it’s too simple and, therefore, commonplace for these environments to execute in silos not aware of changes to other silos and where environmental constraints and dependencies are not considered throughout the development process.
The best solution to these challenges is to focus on a cascading framework of objectives and goals. Every company has its mission-critical goals—some call them strategic imperatives, others call them (or include in them) board-level mandates—and those goals should cascade into a more refined product, program and team goals as you traverse the management tiers. This ensures the business leaders can set a strategic direction that will be measured in a certain way, and every subsequent layer below them understands how their initiatives align with supporting that direction.
The first critical element for building a successful, sustainable DevOps culture is understanding your goals and how you’ll define and measure success.
The next element is to establish which decisions each level of management can make autonomously. For example, risk tolerance is a strategic decision, architecture and security posture are typically defined for a product, platform-level technology decisions can be delegated to program teams and project teams can then focus on making the more granular technology decisions that will differentiate their product, service or feature.
Finally, communication is key. This sounds trite, but communicating this cascading decisioning framework, exposing the decisions each group makes that could impact the particulars of an environment and then discussing requirements and challenges of architecture and software choices can ensure that each team, at each level, can remain agile and adapt to change effectively.
The question is: Do you have the management rigor to dig into these challenges and own them? The answer likely lies in how important the pursuit of agility is to you and your business.
I arrived early to a meeting, and the pleasant security guard at AT&T suggested I take a look at their innovation museum while I wait. Among motion picture sound technology and life-size photos of switchboard operators on roller-skates stood a transistor prototype. The transistor was invented in Bell labs in 1947, and it changed the world. Steve Wozniak considered it the greatest innovation of all time in his 2014 article.
But almost nobody needs to think about transistors anymore. The transistor had to become transparent and abstracted, so the next generations can build great things in higher level. The transistor made a huge influence on our lives as an enabler, but if people had to keep talking about transistors it wouldn’t be possible.
Abstracting technologies is key to advancing to a higher level, making it possible to solve real business problems.
The language of a client-focused business is applications. If we want to make a business execute faster these days, we need to make the development and release of its applications execute faster. But one thing stands in our way: the infrastructure. Applications reside on infrastructure–they are installed on servers, bundled in containers, on cloud instances, in networks, using storage.
If we want to speak the language of applications, we need to abstract them from the infrastructure, and make the infrastructure transparent.
The concept of abstracting applications is what stands at the basis of every new approach to application development and release around us.
Containers are all about abstracting services from specific infrastructure and making them mobile.
Serverless takes it to the extreme.
Public cloud makes it easier not to worry about owning physical infrastructure.
Technology screams, “Think applications, soon you will not have to worry about infrastructure!”
And yet, most of us mix applications and infrastructure in an unhealthy way when we try to automate them. It is unhealthy because mixing applications and infrastructure prevents reuse of automation assets and makes it very difficult to connect infrastructure usage to a business context.
Understanding how mixing applications and infrastructure is manifested requires looking back into the evolution of infrastructure and application automation.
The Evolution of Infrastructure: Manual Deployment and the Inception of Its Automation
In the beginning there was manual. We got a server; it was sitting on a shelf. We got the application artifacts, maybe on a CD or a USB stick. We installed the application on the server. Our application was ready for use. When we needed to upgrade, we went back to the shelf and installed the update on our server. It was very hard to make this operation faster.
This is where automation of applications and infrastructure started.
Behind any automation tool or standard stands a model. This model reflects how its creators originally perceived the problem they were trying to solve. It consists of the entities they defined as first-class citizens, the operations they wanted to apply on these entities, their attributes.
“All models are wrong, some are useful,” said George E. P. Box in 1976. This is true for automation models too. Automation models are usually nice and clean when we start off. But as the problems we try to solve transform, our models need to grow and adapt. Until at some point they are not useful anymore and we need to replace them with next generation modelling approaches.
The Configuration Management Model
Configuration management tools such as Puppet and Chef were originally built to manage application deployment and configuration on static infrastructure. This new approach to Ops optimized manually updating applications on servers. Better network connectivity made it possible for this model to thrive, with the ability to configure applications on servers remotely.
But within a decade we witnessed the fast rise of immutable infrastructure. Immutable infrastructure is an approach to deploying applications where the infrastructure components are replaced instead of updated. This approach somewhat pulled the rug from under the configuration management model. With each update or change, the application must be re-deployed. This forced adding dynamic infrastructure provisioning to the configuration management model. Combining infrastructure automation and immutable infrastructure concepts into the configuration management model is a difficult task, since this model was never designed to accommodate something like that.
The Infrastructure-as-Code Model
Pure Infrastructure-as-Code tools were built to provision infrastructure dynamically. Terraform is an example for an open source tool that does this well. This model is created with infrastructure in mind.
Whether we look into Terraform, AWS, CloudFormation or ARM templates, we will see the infrastructure is the starting point (defining network connectivity, VM instances and services) and the applications are an implicit addition. They are artifacts that happen to be deployed on and are tightly coupled to the infrastructure. We may look at a component that says “App A” but it actually stands for “specific infrastructure with app A installed on it.”
This seems natural, because it’s the same way it worked in the good old physical-infrastructure world: We fetch a box, we install something on it. We focus on the infrastructure, because this is our starting point. The application is the artifacts we put on the infrastructure and scripts we run after we provision it. Presumably, the application cannot live without something to be on.
Infrastructure-as-Code tools evolved the way open tools do–their base model was extended by the community because there was a need to automate applications deployment and not just infrastructure. But it is becoming difficult to extend this model to reflect a world where applications are built to move between clustered servers, and the starting point is no longer the infrastructure, but rather the application’s requirements.
Mixing Infrastructure and Applications
Trying to fit infrastructure provisioning and application deployment in a single model that was originally designed for just one of them creates issues. Infrastructure is lower level than applications. Applications reside on the infrastructure. The same app can reside on different infrastructure configurations. This critical relationship is completely flattened if you mix them.
Moreover, from a business perspective it is important to make applications explicit. It’s our product. Our website. Our mobile app. The applications need to be the center of the attention and discussion. This kind of modelling puts applications in the shade of infrastructure.
Four Great Reasons to Model Applications as a First-Class Citizen Right Now
The models we use today are broken, and it is time to evolve. No matter if your applications reside in containers, or on cloud instances, or include native cloud services or a mix of them. Applications should be a first-class citizen, and not some artifact installed as an afterthought on the infrastructure we started from.
The notion that applications should be abstracted from infrastructure doesn’t eliminate all the challenges of infrastructure provisioning and application deployment, but it creates a foundation for better and more scalable automation.
1. Achieve Application Re-Use
Modelling applications as a first-class citizen is the only way to make applications a re-usable element throughout the development and release pipeline.
If there’s one thing I learned in my automation journey, it is that if automation doesn’t get frequently and consistently reused, you don’t get value for your money.
2. The Path to Multi-Cloud
Abstracting application is also the best approach for moving application workloads between cloud providers. If everything starts from infrastructure, we need to start from scratch when we move. If our applications are abstracted and defined separately—this definition stays the same when we move, only the infrastructure changes.
3. The Path to Breaking Silos
The vision of Dev, Sec, and Ops all dealing with the same language (code) is wonderful. But bringing everyone to the same code pool is not always easy. Applications and infrastructure are not the same thing, even though we can code both. Mixing them makes them unnecessarily hard to build and troubleshoot. It forces us to deal with fuzzy boundaries when it comes to responsibility for this code. Basing automation on a model that organizes the world in a way the different stakeholders can understand it, can help overcome this barrier.
4. Connect Infrastructure to Business Needs
Infrastructure-as-Code is a huge breakthrough, but the bigger story is about making application development and release faster. This doesn’t undermine the importance of infrastructure; infrastructure is the foundation of everything. But if we want to solve business problems, we need to talk the language of the business, and that’s the language of applications. Our automation should be built in a way that reflects this.
On December 6, 2018, approximately 32 million customers of the UK telecom provider O2 woke up to a frustrating reality: The most widely used mobile network in the country was experiencing a day-long outage of its 4G mobile network.
For some, not being able to check the weather forecast, traffic report or football scores was a minor inconvenience. For others, the impact was much more severe, as people throughout the country realized they couldn’t collect email, use their connected apps or perform any other task requiring data usage—no matter how critical to their business or personal lives. The world quickly came to realize that O2 subscribers in the UK were not alone: Other mobile providers around the globe, including Softbank in Japan, suffered similar outages in the same time frame.
What caused this global outage? Was it a software bug? A poorly tested update, shared too soon? Maybe even a cyberattack targeting mobile network infrastructure? In the end, the explanation was much simpler: A digital certificate expiration in a back-end mobile data service from Ericsson caused a cascading systems failure, ultimately resulting in data outage for these mobile services. This issue highlighted the potential impact of certificate errors in our modern digital enterprise and the value of automation in managing such a critical process.
Human Errors, Absences Lead to Outages
While the O2 and Softbank service outages had serious consequences in their home countries—the U.S. equivalent would be a provider like Verizon losing data service for an entire day—they are hardly the only ones to experience this issue. Earlier this year, during the shutdown of the U.S. Federal Government, a number of important sites went down or lost functionality because no one was there to manually renew expiring web certificates. Even popular agencies such as NASA and critical services such as those provided by the Department of Justice were not spared, and both agencies and the citizens they serve were forced to watch helplessly as their certificates expired and services went down.
Although it is of little comfort to the customers who woke up without mobile data or citizens who were unable to use government services, this scenario could easily have been prevented had automation been utilized to manage the certificates in question. When certificates are managed manually, it isn’t always easy to determine the cause of an outage like the one experienced by O2 and Softbank. An expiring certificate may not be the first answer that springs to mind, and may only be noticed after poring over code, looking at network monitoring and checking other potential culprits. Certificates are not always front of mind for IT teams, which underlines the value in simplifying their management.
The technology sector has made significant strides in this regard, and organizations have tools available to them that can help streamline certificate issuance, management, renewal and revocation. Gone are the days when certificates had to be manually issued one at a time, and organizations kept spreadsheets of certificate expiration dates hoping they remembered to renew them. While we’re not able to say specifically why or how the root-cause Ericsson certificate was allowed to lapse, unexpected certificate expirations can be eliminated through proper use of certificate automation—making such a disaster unlikely to repeat itself if the proper steps are taken.
Peace of Mind at Scale
Getting started is easier than you might expect. The rise of automation in managing digital certificates presents organizations with a simple and exciting solution that extends well beyond just outage protection. Perhaps best of all, many organizations operating in a Windows environment may already have access to auto-enrollment and auto-renewal tools using the Microsoft CA, which has been a pioneer in the field of automated certificate services. Chances are, you may already be familiar with using Microsoft CA as an easy way to take care of your internal (private) certificates.
But what about public certificates? True, Microsoft CA doesn’t handle those, so most organizations will need to find a third-party vendor to manage the SSL certificates for their web servers, load balancers, VPN devices and other networking gear. There are vendors capable of managing all certificate requirements under one roof, whether you operate in Windows, MacOS, Linux or other environments. The rise of standards like the Automation Certificate Management Environment (ACME) have made integrating different environments increasingly simple.
Designed by the Internet Security Research Group, ACME has become a popular certificate management tool. More than 150 million websites and 130 open-source client tools use ACME. In fact, ACME is supported by a number of tools familiar to those who use DevOps to manage deployments, including Kubernetes, Chef, Ansible, Salt Stack, Terraform, Istio, Docker and others.
As enterprises look to leverage DevOps with increasing regularity, effective certificate management can help facilitate container, micro-service and application security. Furthermore, orchestration tools such as Kubernetes support certificate management natively, leveraging ACME as the underlying protocol. The private keys may be stored in Kubernetes Secret or Hashicorp Vault so the certificate management system integrates seamlessly.
Digitally signing your DevOps containers with a private CA helps verify the identity of a given container as it communicates over TLS while also preventing unauthorized applications—and the tools available to ACME users add significant value. Of course, this is just the tip of the iceberg—using automation to manage these certificates means plugging into a vast ecosystem of resources, reducing deployment time and eliminating the possibility of human error. It really is that simple.
A New Automated Future
Certificate management is hardly the only area where human error can have an outsized impact, but the O2/Softbank outage clearly illustrates severe effects such errors can have. Service outages, damaged reputations and frustrated consumers lie in the wake of poorly managed certificates, and organizations are increasingly turning to automation to solve the problem. As protocols such as ACME become integrated into an increasing number of commonly used web tools and outside vendors expand their ability to shoulder the certificate management burden, it will only become easier for organizations to incorporate automated elements into their own environments.
Network operations traditionally has been the missing piece in DevOps processes. It’s not difficult to see why. Network management has lagged behind server and workload automation. Updating network policies and configurations typically requires a great deal of analysis, testing and time, undermining the goal of DevOps agility. But once we shift how network devices are administered and how changes are validated, there are emerging ways to incorporate network updates and policies into DevNetOps processes through automation.
Shift the Focus to Policies, Not Configurations
DevOps has taken off, thanks in part to the ability of virtual networking and software defined networks (SDN) to automate the placement of workloads, establish connectivity and deliver Layer 4 to 7 services in a few minutes. But virtually all network policy updates, particularly those addressed in the underlay, were constrained by having to reconfigure individual network devices through traditional management platforms or command line interfaces (CLI). New application deployments may require new network paths, access to cloud resources, punching holes in firewalls, etc. There was no way to automate these policy-related tasks through existing network management platforms, and DevOps processes were limited to simple changes that could be made in the overlay network.
Figure 1: The CLI management model and box-by-box management, in general, do not facilitate DevOps processes, which require higher level policy specifications and requirements. DevOps requires the automation of such policy analysis and verification.
What’s wrong with updating individual network devices through CLI? Network administration is hampered by the inherent nature of IP networks individual devices never have a holistic end-to-end view of network policies and behavior. They are usually configured to know what to do with traffic locally, when to drop it and when to forward it to which nearest neighbor. This is not how policy requirements are formed (see Figure 1).
If a new DevOps deployment needs to ensure a rack in Data Center 1 can reach a rack in Data Center 2 through a specific secure gateway, then to a cloud-hosted database and return traffic appropriately, then we need a much broader, path-based view than any individual device or CLI can give us. If we are going to automate any of this analysis or verification, we are going to need a solution that maps our intended policy requirements to network implementations across many devices.
Intent-Based Networking Aligns With Policies and Intent
This is where intent-based networking (IBN) can help. IBN can help bridge our application policy requirements with the network design and configuration. It can allow us to verify if a particular policy requirement is supported in the network currently or help with a path-oriented search to quickly identify which devices need to be updated to deliver the particular policy. Changes, such as updating a firewall rule, ultimately may still be made to a single device through a CLI, but we have accelerated the analysis if the policy is currently supported, what changes need to be made and how such a change may adversely affect other existing policies and requirements.
For example, it may be an enormous benefit to application development teams prior to deploying a new workload, to be able to determine if connectivity from our Rack 1 servers to a remote database is available for new traffic requirements. Typically, this might involve a request to the network team, which would have to analyze the policy requirements or arrange for a test scenario. A trouble ticket request such as this would take more than just a few minutes, even if the network team could jump right on it.
Application Team Self-Service
Imagine instead the application team could self-service their request by querying an IBN system that could immediately verify the implementation of the policy. Perhaps the application team could use a simple internal messaging platform such as Slack to specify their policy requirement and get an immediate answer without further analysis by the network team. They wouldn’t even need to learn the GUI or capabilities of the underlying IBN platform with a Slack messaging front-end.
An IBN solution gives teams the power to avoid thinking in terms of individual device rules and opens the door to end-to-end automated policy analysis for the first time. Application teams could never be expected to understand firewall configurations or be able to self-service their queries about those capabilities, but they can think in terms of high-level application policy requirements and end-to-end path behavior, which IBN platforms can deliver (see Figure 2).
Figure 2: A sample policy query is constructed in the top bar, as the IBN platform shows a number of available paths between data centers that map to the policy requirements.
Automating Network Verification
To further accelerate key network policy changes required for DevOps processes, imagine updates are required to individual devices in the underlay network. Not only can those changes be identified nearly immediately now through IBN, but also the verification that the changes were made correctly and didn’t adversely affect any other applications or policy requirements can be completely automated. These IT tasks of analysis and verification were usually the most manual and time-intensive in the network operations workflow. But now that much of the workflow can be automated, DevOps can benefit greatly.
The verification process would consist of a long list of existing policy requirements that were already in place and implemented. After a proposed change to network configurations, all of those policies could be analyzed quickly to determine if the new update adversely affected any prior requirements, preventing rollbacks or potential security or compliance breaches.
The question naturally arises, What is really enabling this shift in network operations from a box-by-box focus to an end-to-end, path-oriented policy focus? IBN platforms have generally replicated much of the analytical intelligence of network engineers to determine overall behavior from analyzing individual device configurations and state information. Much of this intelligence is based on mathematical models of the running network in software and much less so now on analyzing live traffic under limited scenarios. This analysis of the mathematical model in IBN platforms now can provide definitive verification of policy requirements for the first time, and even identify configuration errors long before they can cause loss of service or an outage.
The key to accelerating network analysis and updates in support of DevOps processes is to shift focus up from individual box-by-box device management and CLI interfaces to focus on end-to-end network behaviors that align with application policy requirements. Application team self-service for many policy-related network requests and automation of network verification tasks are just a few of the benefits for DevOps teams that IBN can deliver today and into the future.
Driven by advances in technologies such as artificial intelligence and deep learning, the mobile robots market is projected to grow to $54.1 billion by 2023 from $18.7 billion last year, according to Markets and Markets. But for robotics to progress, security for the internet of things (IoT) needs to be addressed fast, before the reputation of the entire industry suffers a permanent setback.
Every industry segment has a journey to maturity that involves ramping up how seriously it takes security. For robotics, that time has come. Failing to start robotics projects on the right software foundations will see confidence continue to drop, impeding the rate at which new solutions are created.
The future of robotics rests firmly on the shoulders of the IoT. Robots act as one part of intelligent ecosystems—they depend on IoT to link various sensors and smart meters, and pass data to and from third parties.
The self-driving car, for example, is a robot orchestrated by various smaller devices and smart sensors. In fact, robots are being created to tackle every conceivable problem. Take the Google-funded RangerBot, an underwater machine designed to track down a species of starfish responsible for coral reef destruction, or Small Robot Company, a startup tackling farming deficiencies with bots that autonomously feed, seed and weed arable crops.
However, while IoT is an established concept and connected devices are now mainstream, its full potential has not been realized in part because of security concerns. Ninety percent of consumers lack confidence in connected devices, according to a recent survey. These concerns are more than valid, as nearly half of companies are unable to detect when a breach occurs, and only 15% of budgets are earmarked for IoT security.
With IoT security a lingering weak spot, one major attack within robotics, could have a domino effect on the entire industry. Add to this the verticals that robots are beginning to infiltrate—health care and agriculture, for example—and the threat becomes immediate and very real.
Rising levels of sophistication within robotics go hand in hand with more targeted and damaging attacks. For example, telesurgery uses robotics to help surgeons perform procedures remotely. A malware bug in this scenario could mean the downing of tools, threatening the patient’s life.
Researchers at Brown University proved how easy it is to hack robots. The industry simply will not be sustainable without the backing of a secure, connected IoT network.
Robot manufacturers, therefore, must build with a security-by-design mindset. This begins by selecting a robust operating system from the outset—secure now, but also ready for future market demands.
Hackers are constantly evolving their activities and businesses must be flexible in their approach to security, shedding the old hardware-centric view of IoT security.
Software no longer can end when a device is shipped. It must align to the lifespan of a robot and be able to update whenever there is a potential flaw. The world of mobile took many years to get to grips with this. Robotics and IoT developers should learn from their mistakes.
One way developers can safely build and secure software is through snaps, containerized software packages on an open platform for building and publishing applications to an audience of millions.
If a security vulnerability is discovered in the libraries used by an application, the app publisher is notified so the app can be rebuilt quickly with the supplied fix and pushed out. This allowsr developers to stay focused on innovation while ensuring the longevity of the robotics hardware.
It remains unclear where the onus lies for IoT security, with nobody holding anyone else to account. Market constraints often prevent device makers from putting more budget than what is necessary into design security, when there is so much pressure to innovate ahead of competitors.
These issues need to be solved. Robotics has too much potential to let security concerns stunt their growth.
