devopsy.kinsta.cloud

Category: DevOps Practice

  • BMC Extends DevOps Reach to Db2 Databases

    BMC Extends DevOps Reach to Db2 Databases

    BMC Software is extending the reach of its campaign to bring DevOps to the mainframe to the IBM Db2 databases that run on the venerable platform.

    John McKenny, senior vice president, ZSolutions Strategy and Innovation at BMC, said BMC AMI DevOps for Db2 makes it possible for DevOps teams to automate changes to the Db2 schemas as part of a pipeline being employed on a Jenkins continuous integration/continuous development (CI/CD) platform.

    As part of its effort to bring DevOps processes to the mainframe, McKenny said BMC is trying to bridge the historical gap between DevOps teams and database administrators. Many DevOps teams have tried to bypass DBAs that they often perceive to be to slow to keep up with agile development processes. In mainframe environments, however, Db2 databases remain the primary source of data for applications. BMC AMI DevOps for Db2 automatically discovers and determines database schema change requirements, streamlines the review and approval process and implements those changes, he said.

    BMC AMI DevOps for Db2 is part of a portfolio of BMC automated mainframe intelligence (AMI) tools that leverage a policy engine to enforce best practices for database schemas. Those tools play a critical role in enabling organizations that continue to rely on mainframes to capture those best practices at a time when many of the IT professionals who defined those best practices are retiring, said McKenny, adding BMC will evaluate whether it will add support for other mainframe databases such as IMS based on demand.

    It’s unclear how much or even whether organizations that rely on mainframes are embracing best DevOps processes. Providers of tools for mainframe environments such as BMC are betting digital business transformation initiatives will require those organizations to replace their current application development and deployment tools and processes with DevOps processes, which are more agile. The challenge, however, is overcoming the perception that existing waterfall approaches to application development and deployment result in more reliable software. As a result, many organizations that rely on mainframes are trying to determine which methodology to employ based on the nature and criticality of the application workload.

    At the same time, BMC is investing in various forms of artificial intelligence (AI) to automate IT operations, otherwise known as AIOps. Naturally, BMC is not the only provider of IT management frameworks with similar ambitions, but it contends that after decades of providing tools to manage both mainframe and distributed computing environments, it’s in a better position than its rivals to aggregate all the data required to drive the models that are at the core of any AIOps platform.

    Of course, it remains to be seen to what degree IT organizations are comfortable with AIOps. Beyond the perceived threat to their jobs, many IT professionals are skeptical AI models are sophisticated enough to automate complex IT processes reliably. The one thing that is clear is that with the simultaneous rise of DevOps and AIOps, the way IBM platforms have been managed is about to change utterly.

    — Mike Vizard

  • Why Modernizing Security is Like Visiting a Fast-Food Restaurant

    Why Modernizing Security is Like Visiting a Fast-Food Restaurant

    Fast casual restaurants are taking over the food industry. Today’s consumers want quality and speed, and the brick-and-mortar model offering immediate service paired with quality ingredients perfectly fits the bill. It’s the new, modernized dining experience. You can also argue that the same modernization taking the food industry by storm is happening in the security sector.

    Simply put, fast-food restaurants know what their customers want: food that’s served hot, fresh and quickly–all while being made to order. And while the needs of a fast-food customer may seem far from the security needs of an organization, it isn’t too far off. Just like a fast-food customer, DevOps teams are hungry for a streamlined process. However, in their case, this doesn’t come in the form of a fried chicken sandwich; rather, an app infrastructure that is low-latency, secure and brings visibility.

    According to the 2018 Verizon Data Breach Investigations Report, web application attacks are the leading source of data breaches. This suggests security procedures are not keeping up with modern systems and vice versa, which means they’re not keeping up with the modern customer, who has increased security needs due to demands for fast, interactive, web-based experiences with vendors. From food service to security, many industries are learning that, to succeed today, you’ve got to combine the old with the new.

    As technology evolves, systems must as well. When it comes to security, it does not make sense to continue using a legacy WAF built for a decades-old line of defense. Today’s technology and applications require next-gen WAF systems that can keep up with the changing infrastructures and the evolving threat landscape. The risk is too great to sit back and do nothing.

    Security teams today often find themselves either supporting new infrastructure initiatives or inheriting legacy applications and systems. Determining where an application is running can be a challenge. Rather than simply pointing to a rack of servers to locate an app, security professionals now must look to the cloud. However, this presents another obstacle: container orchestration. Kubernetes or otherwise, the options are not always clear-cut.

    A modern infrastructure mix is made up of many parts (much like a layered chicken sandwich): cloud, containers, hardware, platforms—and sometimes serverless. Flexibility and deployment options are essential to defending applications and APIs across multiple components and delivery stacks. Typically, security teams end up in a balancing act of supporting new infrastructure plans while taking over existing legacy systems and applications.

    To be adequately protected, security teams must evaluate defensive tools, open source or commercial, for use in:

    • Major cloud providers (e.g. AWS, Azure, IBM, GCE)
    • Container platforms (e.g. Kubernetes, Docker)
    • Hardware and web servers (e.g. load balancers, NGINX+)
    • Serverless options (e.g. API gateways)
    • Platform services (e.g. Heroku, language plugins)
    • Cloud-based deployment models (e.g. hosted cloud WAF)

    The modern world is ever-evolving and the definition of “modern” changes as new technology is introduced. Fast-food chains have evolved over the years and, in turn, moved the food industry forward with the introduction of fast, casual and reliable options. The companies continue to evolve to meet the customers’ demands by introducing delivery services, unique rewards programs and leading mobile apps.

    The security world can learn a thing or two from the fast-food chains’ commitment to continued business evolution. Much like the way fast-food customers are quick to jump ship if they don’t see the selection they want, legacy WAF customers are learning they can make a change to other security solutions if features and infrastructure aren’t evolving.

    The simple truth is legacy WAF solutions can’t accommodate today’s technology and applications, exactly how traditional fast-food can’t keep up with customers’ growing demand for tasty yet quick meal options. This leaves proprietary data exposed and vulnerable to attacks. It’s time for security leaders to take action and upgrade their systems while getting their teams up to speed without sacrificing security of their vital assets.

    — Jonathan DiVincenzo

  • Embracing the Product Development Cycle

    Embracing the Product Development Cycle

    Nowadays, a truly successful product calls for purposeful investment in the people that will ultimately use it in their specific environments.

    We all know technological development is hyperactive and unpredictable. Ultimately, we want to be able to create cutting-edge tech, deploy it wherever it’s needed and have it operated by skilled staff that understand all the nuances. We need to be able to move fast, adapt on the fly and do so with security mechanisms that fire on all cylinders–irrespective of environment.

    The best way ahead is to actively engender a more informed user base that can apply experience and context. There are big benefits to be had by doing this. With the right investment and infrastructure, the aim is for informed and confident end users to form symbiotic relationships with the vendor, helping to guide product and solution development on an ongoing basis. Operational challenges are not static, so the technology in place to meet them certainly shouldn’t be.

    Vendors need to embrace the mindset that a product’s lifecycle needn’t be clunky, disrupted by diversions or constrained by boundaries. It is all about continually engaging and listening to every contributor to a product’s existence: customers, partners, engineers, support staff–the whole ecosystem. Collaborate, innovate, feedback, improve and repeat. The future is a dynamic continuum of constant innovation and improvement.

    As part of F5’s transformation to a multi-cloud application services company, we had to train hundreds of engineers around the world with a new set of foundational skills in cloud, automation, orchestration and DevOps methodologies. Essentially, the process entails developing training content to test with a selection of key users. Areas of improvement are identified, and these are then built into the product. The cycle is repeated.

    It may sound simple but you’re not going to get the substantive, actionable feedback you need without instilling the right culture. For this, you need a lot of training and documentation. You need to guide the way with strategic credibility and maintain confidence. Above all, you need to be patient and listen.

    At F5, we’re constantly building out the mechanisms for how we train in conjunction with our development efforts. A big part of that is integrating adequate feedback loops—both on the training and products front. If a customer hits a wall, what did we learn? How can we contribute via open source? How do we leverage what we learned to make our products and people better?

    We’re not just building automation and orchestration solutions. We’re aligning to the methodology around DevOps and the software process. We’re designing as if it were a giant distributed software project that happens to run on and integrate with the F5 platform. All of this gives us the means to start a new conversation with customers, one that is entirely based on how to achieve successful outcomes. Sometimes we may assent to, for example, the introduction of a new feature. Sometimes, we’ll push back. We will always be agile, adaptable and collaborative. It is never anything less than an eye-opening journey. It needs to be. We have no interest in directionless wheel-spinning, nor do our customers.

    Another major benefit of our approach is it explicitly and effectively helps NetOps become a true partner of DevOps. This means projects are no longer bounced around in silos, bottlenecks are eliminated and the results speak for themselves. Going through this process and making these shifts have fundamentally changed F5’s ethos and identity. We had to change in order to fully engage with the bleeding edge, with the way container ecosystems are built, the way microservices are built or the way software is developed in the modern age.

    — Hitesh Patel

  • A Culture-Driven Approach to DevOps Transformation

    A Culture-Driven Approach to DevOps Transformation

    One of the most challenging aspects of a DevOps transformation isn’t automating and overhauling technology. The biggest challenge of DevOps is changing an organization’s culture and purpose. You can’t “buy a DevOps.” DevOps demands the involvement of people, process and tools working together.

    Unfortunately, I’ve seen too many companies get carried away with the latest shiny object and neglect the people aspect. So, the end results they were expecting do not come to fruition. Research from Google Cloud and Harvard Business Review supports the importance of a people-first DevOps strategy. This research confirms DevOps transformation projects are worthless without addressing the people aspect.

    Of course, such large-scale transitions are difficult. People have to put their reputations on the line to sell the transformation. Employees may become stressed or resist the cultural changes that come with a DevOps transformation. Therefore, communication throughout the entire transformation must be your primary goal. Communication includes measuring the effectiveness of the transformation vision and goals and sharing that information with your team. Keeping them informed ensures they understand the critical role they play in this vision, as well as the value of their work. These measurements need to also show how participation in change translates to value for the business.

    Here are some best practices for prioritizing people and process when working through your DevOps transformation.

    Communicate Every Step of the Way

    It’s important to establish a level of empathy for your employees and understand exactly how changes will affect them. Employees’ professional identities develop over many years, and you can’t expect them to change overnight. Here are some questions your employees are wondering about:

    • What happens to my job when we have to start adhering to this new process?
    • Will I lose my job when all this automation is implemented?
    • How am I going to learn the skills to be effective and contribute to this new way of working?

    Expect their questions and listen for more areas of concern. Be proactive about sharing both the questions and the answers with everyone.

    It’s critical that you keep an open channel of communication with your employees to help them through this change. Don’t view this as an add-on exercise–it should be the cornerstone. Focus on communication, and devote time to your staff, so they are completely integrated into the new DevOps process.

    Your communication strategy needs to include all the teams involved with application delivery. Hold regular meetings across all departments, not only development and operations teams. Think about how the DevOps transformation impacts departments such as marketing, human resources and finance. What challenges have they run into? Have there been any negative impacts thus far? Listening and addressing each issue in real time will ensure the transformation is moving in the right direction.

    Measure What Matters

    When it comes to measuring your success, don’t worry about what other companies may be doing. The metrics that matter are the ones that support your ultimate business goal, which in this case is a better software delivery process.

    Identifying metrics for your business demands you understand how your company’s process drives behavior. Ask questions such as: What corners were cut to get from development to production on time? Did meeting this deadline impact the timelines and performance of other teams and the quality of other applications?

    Meaningful metrics allow you better insight into your processes in the real world. And meaningful metrics help determine whether your pipeline is moving in the right direction to deliver high-quality, high-value software.

    Remember those regular meetings I mentioned earlier? These are also a great time to focus on metrics so employees understand how their work impacts the business. It’s also an opportunity to hear what’s happening on the ground in the delivery pipeline. This real-world information can determine progress, as well as test if the right metrics are being tracked.

    Conclusion

    A DevOps transformation requires changes to staffing, organizational structure, performance and culture. As a result, the success of your transformation involves moving the entire organization forward—building a strong culture ensures all people at all levels are invested in it. Measuring and communicating results must be the standard for every release you do.

    DevOps can fundamentally change any organization for the good. By continually communicating a clear vision of the goals and expectations of your transformation, people will feel confident they are on a winning track.

    — Tj Randall

  • Principles Systems and Tools – What Shingo Can Teach Us About DevOps

    Principles Systems and Tools – What Shingo Can Teach Us About DevOps

    In my last post, The Power of a Game: What I Learned from Facilitating the Phoenix Project Simulation, we explored how DevOps and Agile practices are built on underlying Lean principles that go back decades to a time before the Toyota Production System. When you look into the history of Lean, it is almost impossible not to encounter the great thinker Shigeo Shingo. Shingo worked closely with Taiichi Ohno (the father of the Toyota Production System) to advance Lean thinking while building the Lean production system at Toyota.

    As someone who attempted to apply Lean to IT long before DevOps came into existence, I have always said DevOps is about creating an environment where it is safe to look deeply, question what is happening, think crazy things, experiment and learn based on outcomes. Ten years ago, I was invited by The Shingo Institute to serve as a teacher, coach and assessor. I had recently been awarded the Shingo prize for my book, Lean IT – Enabling and Sustaining Your Lean Transformation. What I learned from Shigeo Shingo’s writings heavily deepened my understanding of the role of Lean and flow in IT. These ideas are reflected in the Lean Enterprise Pyramid below and it’s amazing how effective they are with Agile and DevOps.

    Principles and IT Excellence

    A principle is a timeless universal truth that controls outcomes whether you believe in it or now. Think of it as a fundamental truth or proposition that serves as the foundation for a system of beliefs and behaviors. Principles govern consequences and outcomes–both positive and negative. “Create customer value” is a core principle in business success. If you don’t believe in customer value, or don’t deliver it, your prognosis is pretty bleak.

    In Lean IT, I introduced some key principles that drive IT success:

    Let’s begin with a working definition of DevOps—a mindset, tool set and set of practices that combine software development and IT operations to shorten the development/operations life cycle to deliver small iterative features, fixes and updates often and in close alignment with business objectives and value creation.

    Consider the Lean principles pyramid from a DevOps perspective:

    • Foundation principles build an intentional culture of trust, respect, focus and learning.
    • Behavior is how we show up and engage.
    • Perspective is a fresh new way of looking at IT work.
    • Flow is the outcome of effective DevOps in action.
    • Capstone of the pyramid fosters a regenerative culture of trust, safety, experimentation, collaboration and innovation.
    • Outcomes are higher quality IT solutions and services, in less time, with less cost with healthier teams.

    For DevOps, Agile and High-Velocity IT of any flavor to be successful, each of these principles must be addressed. You can’t address them all at once, so I suggest beginning at the base while taking steps to enhance flow. If you focus on these two elements, respect for people and flow, you will always be progressing in the right direction.

    Systems and IT Excellence

    Shingo defined a system as a collection of tools and tasks highly integrated to accomplish a specific outcome. The Shingo Model captures the important relationship between principles, systems, tools and results. It also supports three key insights which clarify how to achieve excellence in IT (or any organization). IT needs to get good and then continuously get better with its systems, tools and results (no kidding). Understanding how these elements are informed and reinforced by principles is essential to making this happen.

    The Insights

    1. Ideal Results Require Ideal Behavior
    2. Beliefs and Systems Drive Behavior
    3. Principles Inform Ideal Behavior

    For DevOps and Agile, the Shingo Model provides a logical sequence for change. Put another way, what are the key questions we need to answer?

    1. What do the ideal behaviors of DevOps look like?
    2. What beliefs do our people have today and what beliefs do they need to embrace DevOps? What systems (think: work systems, procedures, policies and technology) do we have in place today, what behaviors do they drive and what needs to change?
    3. How can our understanding of lean principles clarify our understanding of question #1?

    Take a moment and reflect on the role of principles and systems in your transformation. The more you ground your approach in the stability of principles, the greater your chances of success. If you or your organization does not truly believe in these principles, a conversation around which principles you do believe in would be a good place to start.

    — Mike Orzen

  • Fujitsu: State of Digital Transformation Is…Mixed

    Fujitsu: State of Digital Transformation Is…Mixed

    According to the Fujitsu Future Insights Global Digital Transformation Survey Report 2019, nearly 90% of companies in a recent survey are already deep into their digital transformation efforts. While the survey did find broad movement toward digital transformation, there existed wide differences in the progress of transformation across industries.

    If digital transformation progress were a horse race across industries, financial services and transportation would be running neck and neck. While, overall, companies reported 36% progress in their digital transformations, 47% of financial services firms and 45% of transformation companies said they have moved forward with digital transformations and have delivered positive outcomes. From there, the government reported 35%, manufacturing 32%, healthcare 31% and wholesale/retail 29%.

    Fujitsu’s findings are based on 900 respondents from nine countries. Interestingly, only 13% of those surveyed said their companies have yet to get started with their digital transformations.

    Not surprisingly, online businesses are moving along more quickly with their digital transformations than brick-and-mortar businesses. While online businesses comprised 33% of the survey respondents and brick-and-mortar 67%, online businesses have made more progress in digital transformation. Of the online companies, 91% are implementing a digital transformation, with more than half achieving positive outcomes.

    One of the more concerning findings from the Future Insights Global Digital Transformation Survey Report was the lack of trust toward online data. A considerable 70% of respondents found it difficult to judge if online information is trustworthy, and 59% were concerned that the data they use may have been falsified.

    “Concern about personal data is also high, with 72% of respondents saying that they are worried organizations exploit personal data without their permission,” the report said. “However, more than half of respondents are content to provide personal data to an organization if they can trust the organization and in cases where personal data is used to enhance products and services.”

    In addition to a lack of trust toward data, respondents are also unsure about AI-made decisions. “Sixty-three percent of respondents think that decisions made by AI are fair because AI is not subject to bias. However, 52% of respondents claimed they couldn’t trust AI because data can be incorrect or biased.”

    However, 60% of respondents said they would trust decisions made by AI if the AI shows substantial reasons for reaching the decisions, and 66% indicated they would trust a company that published a code of ethics governing the use of AI.

    “Companies in which management places an emphasis on long-term perspectives, practices empathic leadership by sharing their messages and passion with employees, and empowers their staff tend to achieve greater success in their journeys toward digital transformation,” the survey found.

    While respondents to this survey indicate moving forward toward their digital transformation objectives, as we recently covered in Digital Transformations Fall Short, Survey Claims, the Kony Digital Experience 2019 Index Survey concluded that businesses are “still missing the mark” with consumers despite nearly $5 trillion in digital investment. The report didn’t distinguish how the $5 trillion was being spent and in what areas of the business the investments were made.

    That report found that said organizations in banking (54%), retail (59%) and utilities (51%) are keeping pace with their expectations but are not delivering differentiated digital experiences. The big laggard was health care, with only 48% keeping pace and 20% being behind respondents’ expectations.

    — George V. Hulme

  • Relentless Automation: The Roadmap to Success

    Relentless Automation: The Roadmap to Success

    As digital transformation continues to shape organization-wide initiatives, CIOs are recognizing the benefits and value of automation in reducing errors and process variability, leading to lower operating risks and faster execution and innovation. In fact, according to Gartner’s recent report, by 2023, 40% of I&O teams will use AI-augmented automation in large enterprises, resulting in higher IT productivity.

    Despite this future uptick, many organizations are currently automation deficient as a result of complex infrastructure, fragmented task automation and tool sprawl. A renewed focus on automation will put I&O teams in the driver’s seat as cloud-like capabilities become the norm. To achieve the highest ROI, CIOs should audit their organization first in order to determine the level of need for automation as well as which specific areas to automate.

    Having a clear mandate is essential for any organization to be able to implement automation at scale. CIOs need to ask the right questions of their organization to determine a baseline for automation. These core questions include: Where is the organization experiencing business issues as a result of a lack of automation? What is preventing the organization from automating more? Does the organization have the right foundation of standards, people and processes on which to automate?

    From our experience, on the journey to mature automation, companies typically fall into one of following five stages:

    Stage Zero: Reactive

    Organizations in the reactive phase do not understand the demand for IT services and the importance of measuring IT service results. Their customers don’t have a well-established way of requesting services nor an understanding of the status of their request. Requests are obtained from various sources, completed inconsistently and process documentation is typically piecemeal. Difficult requests are often directed to individuals who have assisted with similar requests in the past, and tooling is procured by teams without strategy or direction.

    Stage One: Focused

    Focused organizations know the limitations of existing IT automation and actively create initiatives to improve service delivery on a small scale. In this stage, customers have a formal method of requesting services, but it isn’t fully followed or enforced. These organizations have limited processes and tools, and are still highly reactive, relying heavily on a specific individual’s knowledge. Automation is defined and repeatable with possible consistent results, documentation exists for all common services and there is awareness that tooling and coding is inconsistent.

    Stage Two: Established

    Organizations in the established stage are more proactive, and provide IT services with a formal approach to processing service requests with an automated triage capability. Workflow is measured for completeness, and tasks-to-complete services are well-understood, with a gap analysis on which automation has not been completed. In this stage, at least one primary IT service has been automated. There is an automation backlog with prioritization in place. Tooling is standardized and the process for code management is uniform across all teams. Though there is an agreed-upon vision for automation, integration into broader services, such as DevOps, are still incomplete or inconsistent.

    Stage Three: Accelerated

    Accelerated organizations actively deliver new services based on input from the business. Service levels and quality metrics are captured and improved upon, and IT has a catalogue of services available to all business units. Over half of the major IT services have been automated, and developers have integrated standard tooling into pipelines with defined outcomes. In this stage, organizations are focused on process execution and excellence, but KPIs are centered around IT, instead of the business as a whole.

    Stage Four: Optimized

    Optimized organizations have a mature and continuous service improvement program with an established cadence for enhancements. The entire suite of IT services has been automated and aligned with business initiatives, directly affecting business outcomes. Service offerings have various options to ensure automation is being utilized, there are teams in place to deliver applications and software, and IT services have comprehensive documentation and adoption of standardized processes with seamless integration.

    Achieving Relentless Automation

    A focused, cross-functional team should be assembled to effectively assess and define the automation roadmap for an organization. The cross-functional team will bring relevant perspectives to IT service offerings with the intent to automate. Once a baseline for automation is established, CIOs should monitor the progress of their automation efforts, including potential for growth and barriers the organization is experiencing within the zones of infrastructure, operations, application development, service delivery and enterprise service management.

    As CIOs begin to implement automation at scale, they will see reduced operational costs and time to deliver on critical, strategic products and offerings. As a result, organizations could recognize increased revenue, boosting their organization’s bottom line and driving them closer to total digital transformation.

    — Derek Wise

  • 3 Ways IoT Developers Can Make Their Applications More Secure

    3 Ways IoT Developers Can Make Their Applications More Secure

    When the IoT was still young, IoT application developers got away with making security an afterthought, as they built prototypes and minimum viable products designed to demonstrate the different ways the IoT could be used to transform the way we work, play and live. But the IoT has matured and grown to the point where today, Gartner estimates there were 8.4 billion connected things in 2017 and forecasts there will be 20.4 billion IoT devices deployed by 2020. With this growth has come a massive increase in number and sophistication of IoT security attacks, and the need for developers to make security not just a priority, but a top priority.

    As the IoT hacks of Trendnet’s SecurView camera, St. Jude’s cardiac device and the Jeep Cherokee have demonstrated, when security is not a top priority, successful IoT attacks can cause significant real-world financial, reputational and even human damage.

    While it is impossible to make IoT applications (or any type of software) 100% secure, there are several best practices that IoT application developers can adopt to strengthen the security of their IoT applications. These best practices include:

    1. Following a set of approved cryptographic standards when building applications.
    2. Using static code analysis programs that analyze application code to identify common coding mistakes that can lead to security vulnerabilities.
    3. Instituting a vulnerability management strategy that identifies and repairs known vulnerabilities in IoT applications on a scheduled, ongoing basis.

    These three best practices will not make IoT applications perfectly protected from malicious actors. However, they can significantly reduce an IoT application’s vulnerabilities and attack surface, making it much less likely a developer’s applications will be breached, generating angry customers and negative headlines.

    Follow Approved Cryptographic Standards

    Most developers understand that, if they hope to keep IoT data safe, they should encrypt this data, especially while it is in transit from device to cloud or vice versa. Yet, faced with tight development deadlines, as well as the need to make the most of their IoT devices’ limited bandwidth and processing capabilities, some developers choose to not encrypt their data or use weak cryptographic algorithms that promise to be faster or more lightweight than others but do not provide proper levels of security.

    Experience shows taking encryption shortcuts can result in customers’ data being stolen, an application disabled or other problems. For this reason, developers need to work closely with their security teams to encrypt IoT application data appropriately, following standards and guidelines approved by internationally recognized experts such as the National Institute of Standards and Technology (NIST).

    Choosing to use such approved standards does not have a major impact on IoT application development time, performance or functionality. NIST provides developers with guidelines on how to streamline integration of encryption standards into their applications. They also offer many different standards to choose from, so developers can find one that will provide their application with an appropriate level of security without sacrificing too much in terms of performance or functionality. Yes, it does require some additional work and effort, but if IoT developers want to make security a top priority, they need to make sure they are following approved cryptographic standards.

    Use Static Code Analysis Programs

    Many developers are familiar with static code analysis programs and their ability to discover common coding errors that can lead to bugs that affect software quality. However, these programs also can identify unsafe coding practices such as the use of improperly terminated strings, pointers to freed memory and buffer indexes allowed to go past the end of the buffer. These common coding mistakes create security vulnerabilities malicious actors can exploit in otherwise functional applications.

    There are a variety of static code analysis tools available, both proprietary and open source that developers can use to alert them to coding errors that could create security vulnerabilities. Developers should research which ones best support their security and other code analysis needs, and then ensure that throughout the application development processes—and especially before deploying an application—they use these static code analysis programs to discover any coding errors that might lead to a security breach down the road.

    Again, many developers might be tempted to skip this step, since the problems revealed by the program might prevent them from deploying and commercializing an otherwise functional application as fast as they would like. But, as the examples of IoT application hacks mentioned above demonstrate, cutting corners with security can often result in major problems down the road. If developers are dedicated to delivering their customers IoT applications with robust security, they need to commit themselves to using static code analysis programs to fix coding errors that might make their applications vulnerable to cyberattacks.

    Monitor For and Repair Known Vulnerabilities

    The media pays a lot of attention to so-called zero-day vulnerabilities—those that a software vendor has no prior knowledge about. However, the low-hanging fruit for most attackers is the ever-growing number of known vulnerabilities present in the millions of software components used in IoT applications that haven’t been fixed, either because the user of the component is unaware of the vulnerability or because they have not taken action to update their application to repair the vulnerability.

    All IoT application developers should be aware of the known vulnerabilities their applications might have as result of incorporating third-party software components into their applications, be they open source components (such as a Linux operating systems and associated software) or proprietary components. However, keeping up to date on all the known vulnerabilities in an IoT application’s device firmware, operating system and other third-party software components is not simple. The enterprises that have built these software components, government agencies and other watchdogs often identify as many as 1,000 different vulnerabilities in these types of software components per month.

    Fortunately, there are resources that can help developers stay up-to-date on new known vulnerabilities. For example, the National Vulnerability Database (NVD) is a public resource operated by NIST that catalogues known vulnerabilities in all types of software components, both proprietary and open source. To ensure their IoT applications do not have any known vulnerabilities, IoT developers can compile a security bill of materials (BOM) for their IoT application—a list of all the software components and versions used in the application—and then monitor the NVD for new known vulnerabilities with these components. Alternatively, there are a number of proprietary and open source tools that leverage free APIs to query the NVD database and then scan applications for vulnerabilities either by evaluating their components’ codebases or by discerning versions of exposed services (such as DNS, HTTP/S, NTP, SSH) on network interfaces.

    However, simply being aware of known vulnerabilities does not fix those vulnerabilities. Which is why developers need to not just check for new known vulnerabilities on an ongoing basis, but also have a plan to regularly repair new known vulnerabilities with security updates. Ideally such a plan should allow these updates to be performed remotely (over the air) and automatically. Today, many consumer IoT applications already include an option to enable remote, automatic updates. The commercial IoT applications built by many developers sometimes need more coordination to enable such remote, automatic updates—but this should be the goal. In any event, even if the updates can’t be done remotely or automatically, it is essential that IoT developers have some sort of plan to deploy security updates for known vulnerabilities in their applications in a timely fashion.

    Robust Security Will Enable Future Growth of IoT

    IoT is at an important inflection point. The market continues to rapidly expand, with exciting new agricultural, supply chain, industrial, public safety, consumer and other IoT applications being launched every day. Yet, additional high-profile IoT application breaches could threaten the tentative level of trust businesses and consumers have placed in the IoT, especially as it increasingly is used to collect sensitive data or complete critical tasks.

    To strengthen trust in IoT, developers need to dedicate themselves to making IoT security a top priority. Lip service is not enough–developers need to implement IoT security best practices, including the adoption of approved cryptographic standards, use of static code analysis programs to discover coding mistakes that can create security vulnerabilities and implementation of vulnerability management strategies that regularly identify and repair known vulnerabilities in their applications. If IoT developers make security a real top priority by implementing these and other IoT security best practices, the world’s trust in the IoT will grow and so will their business.

    — Larry LeBlanc

  • Aim to Be a Top 12% Global Testing Team

    Aim to Be a Top 12% Global Testing Team

    We know that you’re thinking, “Really? Another fluff post about the top people in my profession and what I need to do to be just like them?” Listen, we hear you, but this is not some fluff piece with vague ideas of what might (or might not) work.

    In case you missed it, on May 27th, the 2019 State of Automated Visual Testing was released. Based on independent research sourced from over 350 testing teams around the world, we learned 12% of you are getting much better results than the other 88%. We’re talking four times more successful as measured by the things you (and your boss) really care about–test coverage, release velocity, application quality, overall R&D teamwork and cold hard cash. This is not our opinion. It’s not subjective. It’s objective data and information. Data and information that came from you and your peers.

    In other words, we’re not asking you to take our word for it; we’re asking you to take your word for it.

    What Separates the Top 12% from the Other 88%?

    Digital Transformation. Two words that have been baked into our world over the past 20 years. To such an extent it may be fair to call it fluff? Fluff or not, IDC forecasts worldwide spending on technologies and services that enable digital transformation will reach $1.97 trillion in 2022, per the (IDC) Worldwide Semiannual Digital Transformation Spending Guide.

    That’s a lot of fluff! When your bosses’ boss is spending that kind of cash, it’s always worth paying attention. It could be good for your career. As it turns out, 12% of the world’s testing teams did pay attention, and it’s their approach to managing the challenges of digital transformation that have set them apart.

    The Testing World’s Digital Divide – Digital Transformation Quantified.

    You can quantify it for yourself with some simple math and see how you compare to other R&D and testing teams around the world. Got that calculator ready? Here we go…

    How many applications do you have in production today? (Don’t forget those native mobile apps, they really add up).

    How many pages or screens do you have in production on average for each of these applications? (Single page applications can be tricky but give it your best guess).

    How many viewport breaks do you support? (The market average is six if you are not sure).

    How many human languages do you support? (We’re talking about localization here–English, German, Spanish, Chinese, Japanese–not coding languages in case you’re wondering).

    Now, multiply those four numbers together. Congrats! You have just quantified the digital footprint of your business. People can write about digital transformation all they want, but you’ve just transformed all that fluff into something quite real. It’s probably a big number. Over 90,000 for a typical company and over 624,000 for the largest 30% of companies in the world. Here’s the kicker. You and your R&D team are responsible for managing the visual and functional quality of every single one of those pages and screens.

    When functional test automation first emerged way back in 1989 with the launch of Mercury Interactive XRunner, it was a much different time. Browsers didn’t even exist. Since then, the browser wars have come and gone, and standards are now in place. Applications have grown far more complex with native mobile, dynamic content, single page applications (SPA) and responsive design now an everyday reality. Digital transformation for any size company is officially past tense. We’re not transforming, we’re transformed. And now you have to deal with it, but how?

    What Defines a Top 12% Global Testing Team?

    Top 12% teams have overcome the massive challenge posed by digital transformation. Like any technical challenge we have ever faced, it started with someone on the team who felt the pain and set out to solve for it. And when they did, good things happened:

    • Test coverage increased by 60%
    • Release velocity became 2.8x faster (even though coverage increased 60%)
    • Visual and functional quality improved 3x with far fewer escape bugs (even though they release 2.8x times faster)
    • R&D teams are 4x more satisfied with their visual and functional quality outcome

    All of this despite the 12% of testing teams were managing applications 2.2x larger than the other 88%. In all likelihood, they felt the pain before most of us, and have now led the way for the rest of us. We just need to follow.

    Today, successful continuous management of application visual quality creates competitive advantage. Business leaders know this and are paying attention. As a result, testers are in a better position than ever to be heroes respected by their R&D teams for driving huge value for the companies they work for.

    — James Lamberti

  • Top 5 AWS Security Mistakes: Leaky S3 Buckets

    Top 5 AWS Security Mistakes: Leaky S3 Buckets

    As we get ready to discuss our list of the Top 5 AWS Security Mistakes in the upcoming devopsy.kinsta.cloud webinar, we wanted to provide a preview of the type and depth of information we’ll be discussing. Since the most-talked about, and likely the most vulnerable, aspect of AWS security is inevitably those leaky S3 buckets, it’s a given we’ll tackle S3 first.

    In this article, we’ll dig into S3 buckets, explain how they become vulnerable, cover eight different ways they can inherently become open to the public and how to find and protect any public facing S3 data.

    The S3 Bucket

    S3 is one of the oldest services in AWS—so old that parts of it still support XML-based policies instead of the JSON you see everywhere else. S3 also has a lot of features that aren’t as commonly known or used anymore, such as allowing someone else to host put objects in your bucket yet still maintain ownership of them.

    Think of an S3 bucket as a server that can have subdirectories and files. Because S3 buckets can have a different root url (e.g. “dops.s3.amazonaws.com”) and have their own life cycle and settings, we tend to think of buckets as servers, not directories. Also, directories in S3 allow you to organize objects, but don’t have any distinct settings of their own, other than a name.

    AWS has a wide range of use cases it needs to support, which is why we have all these mechanisms. Sharing publicly, within an account, between accounts, hosting websites and so on, creates complexity. The good news is S3 always defaults to secure and private. The bad news is AWS allows human beings to use it (and therefore weaken security) and can be confusing to manage. For example, S3 supports both read and write permissions (and list, for buckets). It’s possible to have no public read but public write permissions, which could lead to people placing bad files in your directories.

    8 Ways AWS Data Becomes Public

    There are at least eight different ways an AWS S3 bucket can inadvertently become open to the public and be exposed to a data breach.

    • Bucket ACLs (Access Control Lists): This is an XML document that defines the first layer of access. By default, only the account owner has access, but this can be opened up to other AWS accounts or the public at large. Amazon recommends against using these at all, but it’s also the easiest way to make something public quickly, so we see it all the time.
    • Bucket Policies: These are super-flexible JSON policies that allow you to set things such as IP-based and other conditional permissions on a bucket. While this should be the primary way for managing public access, ACLs are the first tab and about three clicks to make something public. It’s this flexibility that leads to mistakes such as opening up to a wider range of IP addresses than intended or using a negative to block access to some IPs but inadvertently granting access to everyone else.
    • IAM Policies: These are the normal IAM permissions you use to control access throughout your AWS account. You can’t make a bucket public with them, since they only govern AWS users, but you can open up access to the bucket by authorizing another AWS service access, and then that service exposes the content.
    • Object ACLs: These are the primary object-level controls. It’s just like a bucket ACL and uses XML to allow access from other accounts or the public. Objects do not necessarily inherit bucket ACLs. You can totally make an object public even if the bucket is private.
    • Explicit IAM or Bucket Policy Statements: IAM policies and bucket policies can have explicit statements referencing the object, which will override the object ACL (if you own the object), since they are evaluated first.
    • Pre-Signed URLs: These object-level policies must be created using code (not the console) and provide temporary access for anyone with the URL. It’s used, for example, to share files for a few minutes or an hour for someone using your app to download a media file.
    • CloudFront Origin Access Identity: CloudFront is the AWS content delivery network and can serve as the front end to S3. You can create something called a CloudFront origin access identity to write an IAM policy that allows CloudFront to access the S3 content. If the CloudFront allows public access, it can access the S3 content and that won’t show in a bucket policy or ACL.
    • Cross Origin Resource Sharing (CORS) Policies: CORS is required if you use S3 in a website and don’t want the browser to break due to same site security settings. CORS in S3 won’t override an ACL or bucket policy but could mask public access in limited situations where the data is exposed in the web code through the authorized site.

    All put together, AWS evaluates IAM permissions first. Within those, the only one to make a bucket public over the web is the CloudFront Origin Access Identity. AWS then turns to the bucket ACLs and policies for any explicit deny statements. Then it looks at the object ACLs for public access. If you make an object public and the bucket policy doesn’t have an explicit deny, it is still public, but otherwise a good bucket policy will block everything. Besides, a lot of the problems are open bucket ACLs, not object ACLs.

    4 Ways to Find Your Public S3 Buckets and Protect Your Data

    What’s the best way to find public buckets? It’s relatively easy.

    1. Log into the console, click on S3 and look for the Public tag. AWS uses some advanced back-end math to evaluate all the bucket policies to figure out if something is public, which catches most of the fringe cases, but it does not show if the bucket is private and objects in it are public.
    2. Check CloudFront to see if you use any origin access identities. It’s easier to look here than in all your bucket policies (usually). This saves you from looking through object ACLs as well.
    3. Check any CORS policies, if you use them. This is the least common situation and I even hesitated to complicate this post by mentioning them.
    4. The bad news … there is no easy way to find public objects. Even finding them programmatically can be difficult if you have a large number. Also, changing a bucket ACL doesn’t cascade to the object ACLs so you need to run through and fix them one by one. However, if the bucket is locked down the attacker would need to know the full URL/name of the object to find it. It’s not perfect, but it’s better than nothing.

    That’s it. Although, if you know of any other fringe cases, please don’t hesitate to let us know. While conducting assessments, we find well over 90% of the problems are just basic bucket ACL and policy mistakes. Object ACLs are also an issue, but slightly lower risk since you need the exact URL to find them if your bucket is otherwise secured properly.

    If you are curious about the other 4 Top AWS Security Mistakes, sign up for the Aug 14 webinar.

    — Mike Rothman