devopsy.kinsta.cloud

Category: DevOps Practice

  • Bringing 5G-Enabled Digital Services to Life Calls for Cloud-Native Operations

    Bringing 5G-Enabled Digital Services to Life Calls for Cloud-Native Operations

    Spotify, Netflix, IoT connectivity and cloud storage are a few examples of the digital services providing new options for fun and function to consumers and businesses alike, while representing a growing boon for communications service providers (CSPs).

    Gartner predicts by 2020, nearly one quarter (23%) of CSPs’ large enterprise business communication revenues will be driven by digital services—up a staggering 10 percentage points from 13% of revenues in 2017. This gain should come as no shock to developers who favor building in the cloud or using tools such as Microsoft Teams to collaborate, or who stream video from HBO Go or skim stories in Apple News during their commutes. Today’s customers (consumers or businesses) expect value-driven, digitally enabled services that inspire personalized and meaningful experiences with their brands of choice.

    The good news is companies are eager to meet demand: CSPs want to deploy services in the cloud to address the challenges of web-scale companies. In fact, 85% of companies surveyed say launching new digital services is critical to their business strategy.

    What’s the catch? To do it, they’ll need to be agile. Although telecom is not an industry known for its agility, and is often thought of as the opposite, it has been transforming—like many industries—through adopting more agile approaches in business strategy. CSPs are striving to maximize resources and reduce inefficiencies, while at the same time sparking innovation. Nothing has been more of a stimulus for agility than the cloud, especially for the web-scale companies whose services pose the greatest threat to the telco industry.

    A key requirement will be cloud-native applications that improve the agility of network functions. However, being cloud-native involves a lot more than just shoehorning code into a container previously ported from an appliance to a virtual machine.

    Rather, at Oracle, cloud-native means faster development and deployment through DevOps and continuous integration and continuous delivery (CI/CD) pipeline. CSPs can only achieve an agile cloud with an environment enabling developers to innovate true cloud-native solutions if they implement an operational framework that embraces a key set of principles. Specifically, we at Oracle have identified the principles for successful cloud-native operations.

    System Immutability

    Everything, including software and configuration, is code. All changes are made through CI/CD, where they are deployed as immutable artifacts. No manual configurations or customizations are allowed. This makes it easier to implement the principle of least privilege as there is no need to run scripts in the production environment. Also, any changes not coming though the delivery pipeline can be considered malicious.

    Automate Everything

    All aspects of build, test, verification and deployment are automated. This includes activities such as backup, recovery, password/key rotation, etc. Fully automating the DevOps pipeline, including verification and testing, removes much of the potential for human error. This automation also allows changes to be applied to the environment with confidence and provides for rapid repair.

    Disposability

    All services are transient and treated as short-lived. Instead of focusing on never failing, services are designed to go up and down quickly without service interruption. Regular repaving (re-deployment) of the environment ensures failed or failing services are removed and new ones deployed.

    Externalized Configuration

    Configuration (including passwords, credentials, location of backing stores, etc.) is decoupled from the software image, and like software, can be treated as a build artifact in a controlled and versioned manner. Versioned configuration enables development and production parity as an artifact and can eliminate costly operational errors.

    Logs as Event Streams and Constant Telemetry

    Everything needed to debug or diagnose any functional, operational or security issue is found in logs, traces or metrics data. These are treated as a stream of time-ordered events and stored in a centralized collector outside the system, enabling better threat monitoring, forensics and diagnostics through event correlation or analyzing the aggregated and holistic view. Constant telemetry is fundamentally enabled through logs as event streams and, thus, the two should always be considered together.

    Delegated Governance

    Some shared aspects of the environment are centrally managed like networking, identity management or infrastructure, but in a true DevOps fashion, teams delivering a service are responsible for operating the service. This is allowed only with strict governance, enforced through checks in the continuous delivery pipelines, giving greater control over rate of change. Business agility is improved as applications have better visibility into the operations of their service with tighter feedback loops that ultimately improve quality.

    Independent Lifecycle

    Independently upgrading, scaling and deploying each microservice is paramount for supporting other cloud-native principles as well as minimizing the amount of change in the system at a given time. Furthermore, such decoupling makes other principles, such as repaving, easier and promotes easier isolation of issues.

    As 5G introduces increased connectivity and speed, along with the need for more security and reliability, service providers will need the agility, flexibility and security of the cloud and cloud-native principles. These principles are enablers for better cloud operations. Together, they provide the ideal context in which to implement a broader view of the Cloud Native Computing Foundation (CNCF) landscape, giving developers a robust Platform-as-a-Service of backing services beyond just containers.

    As 5G moves from a fabled future to a reality, the market for digital services that reap its benefits is likely to continue to grow. By leveraging these principles, along with DevOps and the CI/CD pipeline, organizations will be able to evolve and prepare for 5G’s promise for innovative business models and new revenue streams—be they digital services or beyond.

    — Shirin Esfandiari

  • Autonomous Security in Containers

    Autonomous Security in Containers

    With the advent of DevOps, the development world has quickly moved to agile development practices and containerized applications. At Forum Systems, we have responded to this trend by putting our API security software, Forum Sentry, into virtual form factors such as Amazon Machine Image, Azure Image, VMware Image, Linux, Windows and Docker.

    Why do we feel it’s necessary to containerize API security in this way, and what benefits does it deliver over traditional software or even hardware-based API security?

    What Is API Security?

    Before we explain why we containerized our API security software, it is first necessary to define exactly what we mean by API security. As a result of the major IT trends of the last decade–in particular cloud, mobile and IoT–more people and applications are connecting to IT assets than ever before.

    What’s more, the majority of these interactions are from untrusted entities outside the organization’s network perimeter (and as you will read below, across container boundaries). Securing your organization’s systems, data and business-critical processes is harder than ever.

    Almost every interaction relies on an API to communicate to an application or system somewhere in the world. The simplicity of APIs makes it easy for developers to connect their projects to other systems to enhance their functionality, with data being easily shared between a myriad of external partners, cloud providers, virtualized data centers and on-premise applications.

    As a result, APIs have become the primary channel for business transactions and the traditional boundaries of data exchanges have become blurred. APIs provide many benefits but also present risks, as wherever there is innovation, there is also the dark side of threat and attack, and always someone who will aim to exploit weaknesses.

    In many environments, APIs and their underlying technologies are designed primarily to share data; they are not designed to thwart threat and attack. This is where API security gateways are necessary to validate both the data and the identity of users, systems and devices interacting with the API.

    API security goes far beyond just access control, it requires specialized technology to perform dynamic data security to inspect and ensure the specific and unique characteristics of each API communication are correct. The API security policies are designed to prevent intrusion, data leakage and other forms of data loss as a seamless and transparent part of the technology architecture.

    API security gateways differ greatly from API gateways; the key word “security” immediately distinguishes this technology from simple API gateways, which serve only to provide proxy and simplified access control points. API security gateways are purpose-built to protect against API threats, and the dynamic data security capabilities they offer provide centralized protection of bi-directional API communications. API security gateway technology provides essential risk mitigation at each layer of the API architecture, and as rising adoption of containers and micro-service architectures arise, it provide zero-trust concepts within and across the container boundaries to protect the assets and technologies therein.

    Why Containerize API Security?

    Container technology such as Docker has become a popular means to deploy API micro-services, a collection of loosely coupled services which are fine-grained and lightweight. The move toward virtual and cloud was initially driven by fully virtualized images with their own operating system, but the adoption of lightweight services and on-demand environments has led to widespread adoption of container technologies run on a shared operating system. These container architectures provide flexibility and ease of deployment, but come with the same set of API risks.

    The concept of containers also brings into focus the trust model that needs to be considered for communication among technology components within and across containers. Containers become a new API boundary layer and thus represent the same type of risk paradigms, where a rogue container application can gain access or otherwise wreak havoc if the proper security controls are not put into place (commonly known as an insider attack).

    By deploying API security directly into a container, organizations can automate their API security into their existing workflows and provide zero-trust security capability that ensures real-time enforcement and protection of container-based APIs. This means API security becomes autonomous and baked into the architecture, rather than tacked on at a later stage.

    As mentioned earlier, dynamic data security is essential wherever information is traversing. The traditional cyber umbrella approach no longer applies in the interconnected API world. It’s like using an umbrella in a water park, it may stop the water from the top, but not from the sides. By having automated API security at the container and virtual layers, the ability to visualize, connect and secure information becomes an integral (and necessary) aspect of the container security strategy and a fundamental baseline for secure API enablement and trusted data exchanges.

    API security gateways deployed as container nodes allow DevOps teams to harness the power of virtualization, cloud computing and containers while at the same time protecting the organization against API threats from both the inside and the outside. With API security baked into the container, developers are free to focus their time on building the best functionality in their apps.

    — Jason Macy

  • Three Insights from the World’s Leading IT Organizations

    Three Insights from the World’s Leading IT Organizations

    IT is in a state of change on a consistent basis, but never more so than now. With the increasing importance of the digital customer experience across all industries, technology is at a new inflection point in how it can fundamentally impact executive priorities. And while nobody can speak for the direction of IT as a whole, there are a few resources so comprehensive that they’re worth reviewing when they’re released.

    Take, for example, the 2019 Harvey Nash/KPMG CIO Survey. It’s the largest global survey of CIOs with data from 3,645 respondents across 108 countries. IT leaders who participated in the survey controlled a whopping $250+ billion in annual IT budgets. The CIO survey examines how board priorities, technology adoption, skills gaps and IT leadership is changing in the age of digital business.

    Here are three insights that explain the tectonic shifts occurring in enterprise IT teams today.

    Digital Leadership: Three Steps to Becoming a Digital Native

    The 2019 Harvey Nash/KPMG CIO Survey shows only 30% of IT organizations are digital leaders. These organizations are unusually adept at “using digital technologies to advance their business strategy,” and can launch innovative solutions that generate new revenue streams, enable faster time to market and cut down operational costs. Here are three defining characteristics of what it takes to be a digital leader in 2019:

    • It’s All About Data. Data management and analytics is a core competency of leading IT organizations. Digital leaders are in a strong position to maximize the value of their organizational data by extracting, aggregating and curating the right information. This data is then used to train machine learning models, create high-value assets and deliver customer value on an ongoing basis.
    • Speed of Execution. Digital leaders are willing to either quickly double down on successful experiments or pull the plug on products that don’t deliver expected value. They are also more likely to leverage agile and DevOps practices to increase software release velocity and iterate rapidly on solutions using fast feedback loops.
    • Expansive Mindset. Digital leaders have transitioned from project thinking to product-centric innovation for creating enduring value from their software investments. They also build cross-functional teams across their business and IT silos for better collaboration and faster decision making for digital transformation initiatives.  

    What Does This Mean? CIOs will need to embrace big data analytics, agile delivery models and product-centric teams to create the right foundations for digital business. Without proper foundational investments, IT leaders will struggle to meet business demands, gain customer trust and withstand competitive pressures in the face of escalating customer expectations.

    Business-Managed IT: How Shadow IT Finally Became Respectable

    In the last decade, the adoption of cloud services led to the phenomenon of shadow IT where business units would directly buy cloud solutions without consulting IT teams. CIOs resented their growing irrelevance and tried their best to bring shadow IT under their control.

    The 2019 CIO survey confirms 64% of enterprises have made peace with business-managed IT investments. In fact, 10% of IT leaders actively endorse this approach. What factors have led IT teams to change their mind on shadow IT?

    The survey shows organizations with business-led IT scored better on key metrics than organizations without any mandate for business-managed technology spend. Business-led IT organizations delivered “better customer experiences and faster time to market for new products (52% more likely) as well as positive employee experiences (38% more likely).”

    For business-managed IT spending to be truly successful, CIOs will require the right guardrails in the form of central IT oversight. The 2019 CIO survey makes the case that organizations without IT team involvement are twice as likely to have multiple security issues and are also less likely to build customer trust with their technology investments.

    What Does This Mean? Instead of actively opposing shadow IT investments, CIOs should partner with their business counterparts to create effective governance and operating models for business-managed IT spending. Central IT oversight will ensure the right support for digital business initiatives, reduce the risk of unauthorized breaches and minimize wasted spend.

    Influencing Without Authority: What It Takes to Be an Effective Technology Leader

    Board membership for CIOs dropped this year, with only 58% of IT leaders being part of the executive team in 2019 as compared to 71% in 2017. Does this mean CIOs no longer have executive backing for orchestrating and executing digital business strategies in their enterprise?

    The 2019 CIO Survey indicates board membership, headcount or annual budgets are not the right indicators of the power and influence of technology leaders. Instead, CIOs should focus on building trust across organizational boundaries, creating partnerships with the right executive stakeholders and ensuring the success of critical initiatives. CIOs working for digital leaders are comfortable exerting influence without authority and are driving organizational growth, by being “three times more likely to collaborate strongly with business leaders.”

    What Does This Mean? Formal board membership or the size of the IT team is not the best way to assess CIO influence. The best technology leaders ensure successful enterprise transformation by garnering strong commitment from the board, building digital capabilities with cross-functional teams and creating a sense of urgency for rapid innovation.

    — Deepak Jannu

  • The Impact of DevOps on Database Monitoring and Management

    The Impact of DevOps on Database Monitoring and Management

    The rise of DevOps has broken down silos and brought database administrators (DBAs) and application developers closer together to deliver software faster and in a more agile manner. This near-constant change driven by DevOps is reflected in the results of the Redgate 2019 State of SQL Server Monitoring Report.

    It reveals that 73% of organizations now deploy database changes multiple times per month, and 15% deploy multiple times per day, driven by DevOps. Yet, this is just one of the challenges DBAs face today. The research also identified the size of database estates is growing–45% said they will have more to manage over the next year, with 43% remaining the same.

    Additionally, they are increasingly looking to migrate their databases to the cloud (cited as their biggest challenge by 22%) and are seeking to solve performance issues. Perhaps unsurprisingly, given these greater workloads and more frequent changes, human error is the largest cause of issues (22%), ahead of capacity problems and bad deployments.

    Managing in a DevOps World

    The database is now a core part of DevOps for many organizations and DBAs need to work more closely with their developer colleagues, adopting new technologies and tools to ensure business success.

    This essentially involves DBAs focusing on the following distinct areas.

    Standardizing Approaches Across DevOps Processes

    The monitoring report highlighted human error as the biggest cause of problems for DBAs–and with larger DBA and developer teams now working on databases together, adopting standardized processes is vital to flagging issues early and preventing later problems. Otherwise, what starts as a minor issue can swiftly escalate to become a significant business crisis.

    As well as standardizing processes, organizations should look at rolling out industry-standard coding tools across the entire team. This will ensure a consistent approach, not matter who is writing, reviewing or editing code. Introducing a team style for database code, rather than having the code written in many different ways, for example, will make the code base consistent and clear and minimize the confusion that lead to errors. At the same time, merging code early and often with version control tools will ensure you have a known, shared codebase on which to work, and provide a single source of truth that can always be relied on.

    Automating for Speed and Consistency

    Developers who have adopted DevOps already see the value automated processes such as continuous integration, continuous delivery and continuous deployment provide. Automation not only speeds up development, avoids manual errors and reduces laborious work, but also provides a full audit trail for regulatory compliance.

    Extending this to the database can deliver major benefits for hard-pressed DBAs. Every time a change is committed to version control, for example, a continuous integration process can be triggered to test the change and flag up any errors in the database code. The errors can be fixed immediately and tested again, before the change is then passed along to a release management tool where it can be reviewed before being deployed to production.

    Overall, automation dramatically brings down time to market, but organizations need to ensure they have the right tools and processes in place if it is to be truly effective.

    Stepping Up Monitoring to Drive Continual Improvement

    One of the mantras of DevOps is continual improvement, and that applies equally to database DevOps. Monitoring is a key part of this. On one level, monitoring tools highlight potentially disruptive performance issues (cited by DBAs in the research as a major challenge), enabling them to be fixed quickly.

    Beyond this, they also deliver the insights and trends companies need to improve, helping them optimize operations moving forward. Thanks to a combination of DevOps and the growing importance of the database to business success, many organizations are already sharing this information. For example, 40% give developers access to their monitoring data, according to the research, which helps to enhance performance by highlighting opportunities for improvement.

    Guarding Personal Data and Ensuring Regulatory Compliance

    We’ve all seen the spread of data privacy regulations across the globe. Over 60% of the world’s population is likely to be covered by more stringent legislation in the near future, with California joining the EU in enforcing new regulations from January 2020. Organizations need to realize they are now data guardians, not data owners, when it comes to personal information.

    At the same time, developers like to have realistic information in order to successfully create and test new code. That means organizations need to adopt tools for data masking and anonymization that can protect confidential and personal information, while still providing realistic and usable data for development and testing. This also gives a full audit trail for regulatory requirements, balancing speed and compliance.

    DevOps has broken down the traditional silos between DBAs and developers and the importance of information has put the database center stage. As the latest Redgate research shows, DBA workloads are increasing, meaning they need to adopt new technology and collaborate more closely with developers if they are to successfully manage in a DevOps world.

    — Matt Hilbert

  • Why DevOps Implementation is Often Unsuccessful – and How to Fix It

    Why DevOps Implementation is Often Unsuccessful – and How to Fix It

    Much like “blockchain” and “big data,” the term “DevOps” is another buzzword currently being thrown around the IT departments of large organizations.

    Many have identified the need for faster software development life cycles; a more precise process closely aligned with business objectives, allowing for clearer workflow and collaboration between the development and operations teams. DevOps is essentially agile, all grown up and ready to take on the constantly innovating and rapidly deploying needs of the modern business.

    For security professionals, it’s a fantastic initiative: We can inject security into the process far earlier, reducing the cost of fixing bugs and avoiding potential catastrophe down the track.

    The problem is, few companies are truly successful in their DevOps implementation. Without the right support, nurturing and understanding across the business, it can quickly become a white elephant, you know, one of those “don’t mention the war” projects.

    So, what’s the problem? There are a few ways to approach DevOps that I believe will make for much smoother sailing. An effective program goes beyond a few fancy new tools, titles and team meetings. It’s not always going to be easy but taking the time to fix a broken strategy (or implement it the right way from start) is going to be far less painful in the long run. Ultimately, it’s going to result in higher quality and more secure software.

    Let’s break it down.

    Let Go of the Agile Apron Strings

    There is somewhat of a misconception that an organization must choose between agile or DevOps, setting down one path or the other, never to look back.

    The thing is, the development process works best when both are being considered and implemented as one. DevOps is not a reinvention of agile development; rather, it is an extension of it. The wheels tend to fall off when there is an expectation the process will be exactly agile, or completely different from agile.

    Agile supports the principle of cross-functional teams, bringing designers, testers and developers together from the beginning and committing to open communication lines throughout a project. Its aim is to stop siloed delivery and reduce double-handling, both of which are benefits of the DevOps process as well. However, DevOps goes a step further, introducing systems, security and operations into the mix to offer a robust, end-to-end skillset that had the ultimate goal of full, functional software delivery to the customer.

    During the inevitable pain-points of moving to a more DevOps-centric process, the risk of siloed development can crop up again. You can often have the original agile team working together, with the security and operations additions still finding their way in the machine. No one is quite sure how to include them, what they should be doing and their overall objectives.

    DevOps does not work without clearly defined objectives, cross-functional onboarding and direct communication with all parties. There will be an adjustment period requiring careful change management but getting everyone on the same page with the enhancements DevOps functionality will bring is half the battle.

    Increasingly, DevOps is placing emphasis on security best practice as part of the process as well, demystifying that step and bridging the gap between the security team and everyone else. As I have said before, we still have a long way to go in empowering developers to code securely from the start, but the successful implementation of DevOps methodologies is an excellent foundation on which security skills can be built within the development team.

    Automation Isn’t Everything (and It’s Not the Most Secure)

    Another feature of DevOps methodology is, to a certain extent, the automation of the software development process. Continuous integration and continuous delivery (CI/CD) principles are the cornerstones of this concept, and as you can likely guess, very reliant on tools.

    Tools are awesome. They can bring unprecedented speed to the software delivery process, managing the code repository, testing, maintenance and storage elements with relatively seamless ease.

    However, while robots might take all our jobs and imprison us someday, they are definitely not there yet. Heavy reliance on tools and automation leaves a window wide open for errors. Scans and tests may not pick up everything, code may go unchecked and that presents enormous quality—not to mention security—issues down the track. An attacker only needs one back door to exploit to steal data, and forgoing the human element in quality and security control can have disastrous consequences.

    The happy medium is to ensure you have a balance of people and tools. Tools should serve as the assistants to a team you trust to deliver on project goals. You should:

    • Allocate enough time for people to become familiar with the chosen DevOps toolchain.
    • Focus on effective collaboration (and how the tools can support that).
    • Address any gaps in the process, whether they are skill/knowledge or tool-based.

    In short, don’t just tool up and hope for the best.

    DevOps Isn’t a Buzzword, It’s a Culture – Are you Growing Yours?

    Change management is tough at the best of times. Fear of the unknown can stop even the most brilliant team members from growing their skills and expanding their horizons.

    You see, merely saying “let’s do DevOps” and making the operations team move desks isn’t going to magically implement a successful process. Many will be confused, and long-serving members of the team will be left feeling disgruntled. Communication of expectations is crucial, as is walking the walk. DevOps represents a cultural movement just as much as a development methodology, and a team should live and breathe a cross-functional, collaborative mindset.

    What does a great DevOps culture look like?

    • Individuals are empowered to lend their expertise to a process, not just leaders.
    • Open, honest and respectful communication between teams.
    • Each person takes responsibility for the overall objective of building quality and security into the development process.
    • Everyone is on the same page with the definition of DevOps in the business, the roadmap and how/what/why of each person’s role.

    For years I have emphasized the importance of building positive security cultures in development teams, and DevOps is no different.

    The right tools, knowledge and support are imperative to achieving security best practice, seeing a downturn in discovered vulnerabilities and opening the team’s eyes to the importance of protecting our data. With DevOps, you must lay the cultural groundwork for positive change: Ensure everyone understands their role, value and expectations, the overall project goals and steps in the process.

    Have you mastered that? Great. Now, let’s shift the needle, dial up the security aspect and make DevSecOps the ultimate plan for software excellence.

    — Pieter Danhieux

  • How to Transition to Become a DevOps Engineer

    How to Transition to Become a DevOps Engineer

    You’ve either heard the buzzword or you’ve seen the extensive list of rewarding aspects that come with being a DevOps engineer. Whatever the reason you’re here, this guide should help with taking the right steps toward becoming a DevOps engineer, whether this is your first job after graduating or an opportunity to reskill leveraging your prior industry experience.

    Immerse Yourself

    Learn the fundamentals, practices and methodologies. Understand the why behind DevOps before jumping into the tools. I believe the key goal of any DevOps engineer is to increase speed and maintain or improve quality across the entire software development life cycle, in order to provide maximum business value. Read articles, watch some YouTube videos and go to one of your local Meetup groups or conferences–it’s useful to become a part of the welcoming DevOps community where you’ll learn from the mistakes and successes of those that have been here before you.

    Your Background

    There are likely two key times where you’ll be transitioning into the role of DevOps engineer; either reskilling during your career or as your first job after graduating. You may have prior experience in technology, such as working as a software developer, systems engineer, systems administrator, network operations engineer or database administrator, all of which will bring broad insights and useful experience for your future role as a DevOps engineer. If you’re starting your career, maybe you’ve finished your degree in computer science or any other STEM field which will give you some of the basic stepping stones you’ll need in this transition.

    I think it’s important to breakdown the broad spectrum of what DevOps engineer roles entail. Most enterprise companies today will use DevOps engineers in three different positions.

    • DevOps Engineers, with a Dev Bias: Work in a software development role, building software applications. The key point is they leverage CI/CD, shared repositories, cloud and containers as part of their everyday work, but they are not necessarily responsible for building or implementing tooling. They will understand infrastructure and in a mature environment, will be able to push their own code into production.
    • DevOps Engineers, with an Ops Bias: Could be compared to systems engineers or systems administrators. They understand software development but do not spend the core parts of their day building software applications. Instead, they are more likely to be supporting software development teams to automate manual processes and increase efficiencies across the human and technology systems. This could mean breaking down legacy code and using less cumbersome automation scripts to run the same commands or it could mean installing, configuring or maintaining infrastructure and tooling. They ensure the right tools are installed and available for any teams that need them. They enable teams with learning how to leverage CI/CD and other DevOps practices.
    • SREs (Site Reliability Engineers): Think of site reliability engineering as having software engineers solving operations and infrastructure problems. SREs focus on creating scalable, highly available and reliable software systems.

    In the ideal world, DevOps engineers will understand all the above, which is the norm at some mature technology companies. However, if you knock on the door of top tier banks or many Fortune 500 companies you’ll realize oftentimes these roles do usually have biases towards dev or ops.

    Operating Systems

    Whichever background you’re coming from, you should start with the fundamental technologies you’ll need to use and understand as a DevOps engineer. Linux is most likely the operating system you’ll be using daily, although some of you might be using Windows. This is where everything runs, and fundamental knowledge is important. You can install Linux at home where you’ll be able to break as much as you want and continue learning along the way.

    Scripting

    Next up you’ll need to pick a scripting language. There are many to choose from, such as Python, Go, Java, Bash, PowerShell, Ruby and C/C++. I suggest starting with Python, it’s one of the most popular for a reason, it’s easy to learn and interpret. Python follows the fundamentals of object-oriented programming and can be used for web development, software development, creating desktop GUI apps and even business applications.

    Cloud

    After Linux and Python, my next pick would be cloud, where you could choose from Amazon Web Services, Azure or Google Cloud Platform. I’d start with AWS, as they have an extensive collection of free learning tools available to  take you down any track from learning about AWS as a developer to operations and even the business facing components. In fact, you might even be overwhelmed by how much is on offer when you first get setup with AWS. Maybe start with EC2, S3 and VPC and see where you want to go from there.

    Programming languages

    For those of you with a passion for software development. You should keep on improving your programming skills. Some good languages commonly used in this role include Python, Go, Java, Bash, PowerShell, Ruby and C/C++. You should also become familiar with tools such as Jenkins and Git/GitHub which you’ll be using frequently as part of the CI/CD process. Infrastructure is no longer left to the operations guys so you’ll need to have some exposure to a cloud platform such as AWS, Azure or GCP.

    Containers

    Finally, start learning about containerizing code using tools such as Docker and orchestration platforms such as Kubernetes. All the above have extensive online learning resources for free and most cities will have local Meetup groups where you can turn up and learn from experienced people in a friendly environment.

    For those of you with less experience in development, you can still get involved using your passion for automation, increasing efficiency, collaborating with others and a never-ending desire to improve on your own work. I would still suggest learning the tooling outlined above, but with less emphasis on the coding/scripting languages. It will be useful to learn about Infrastructure as a Service, Platform as a Service, cloud platforms and Linux. You will likely be setting up the tools and learning how to build systems that are resilient and fault-tolerant while writing your own code.

    Reskilling

    If you’re transitioning into a DevOps role during your career, you should start within your company to explore opportunities. Can you reskill by working with an existing team? There might be an option for you to shadow other team members, ask for advice and reskill without even leaving your current job. This might not always be possible and some of you will need to move into another company.

    If you can learn some of the practices, tools and technologies listed above, you’ll be in a good position to demonstrate relevant knowledge during interviews. The key is to be honest and to not set yourself up for failure. Most hiring managers understand you don’t know all the answers but if you can show what you’ve been learning and explain you’re open to learning more, you should be in with a good chance.

    Starting Your Career

    You can apply to direct open opportunities at companies who are hiring junior DevOps engineers. You will often find these companies say they’re looking for more experience and you should apply back in future when you’ve gained some commercial experience. It’s the typical, frustrating scenario of “we want more experience” but nobody is willing to give you the first chance.

    It’s not all gloomy though, there are a few companies focused on training and upskilling graduates directly out of the university. MThree, where I currently work, delivers exactly that. We hire fresh graduates and train them for eight weeks in our Alumni Academy. Training covers everything from theory through to hands-on projects with Python, Jenkins, Git, Ansible, Docker, Kubernetes and AWS. By the end of the training you’ll have some solid exposure to the entire software development life cycle and a good understanding of how this applies in a Fortune 500 environment. The best part is once qualified, our teams then leverage their extensive list of clients to provide job opportunities for junior DevOps engineers. MThree pays these associates on a full-time salary with benefits for the first 18-24 months before they join the client as direct employees at the end of the agreement. This is a great way to bridge the gap from university into your technology career.

    In summary, there are many ways to transition to become a DevOps engineer. This is a very rewarding career route which will likely keep you engaged and challenged as well as increasing your earning potential.

    — Conor Delanbanque

  • Computer Vision is Building the Future

    Computer Vision is Building the Future

    Developing an accurate, regulated and purpose-driven facial recognition software is the challenge of computer vision engineering. It is also the future of how people do their everyday tasks: from unlocking a phone to signing into a bank app or logging the time they get to the office. Facial recognition amazes more and more with its new applications. Today, people can go to a concert without the tickets they bought online, they just have to bring…their face and show it at the gate.

    Computer vision science has been working on facial and object recognition software for some decades now. From the beginning, the main goal of this engineering, framed in the AI field, has been to give computers a visual understanding of the world.

    The goal gets fulfilled through a three-step process: image acquisition, image processing and image understanding. Each step can be done thanks to different kinds of technology. For example, image acquisition can be done using a web camera, an embedded phone camera or a 3D camera, among other types of lenses.

    With the help of algorithms computer vision is currently capable of recognizing and tracking objects. But it can go much further than that. Now, neural networks are at such a high level that they are capable of restoring and even creating new images. All of these features that seemed to be available only in futuristic or science fiction movies are now in the market thanks to AI algorithms, such as Generative Adversarial Networks (GAN).

    Today, big cloud providers have released their own AI and vision models including facial recognition APIs. Nevertheless, these models are conceived with rather general purpose algorithms that might be sensitive to specific environmental details, but don’t offer the possibility to retrain models with data gathered in a productive environment.

    The good news is some companies can offer AI models retraining in customers’ environment to accommodate any need or requirement, including usage of depth cameras or ToF sensors. But of course, these developments bring along some challenges. The way to succeed is to be ready and thoughtful in every step. This way, any surprises along the process will be controlled and won’t destabilize the project offered to a client.

    Last year at intive, we built a cognitive department with our internal AI experts as well as the best people we could find in the market. Eleven people shaped the core team that developed advanced facial recognition software in only one year of work. Our objective was to create an AI model fast suited to specific environmental conditions of small, medium-sized and big companies. We created AI models capable of using RGB, depth cameras and ToF sensors.

    Depth cameras enable the API to distinguish a real human face from flat images such as screens, tablets and paper. The new intive’s API can estimate age, detect sex and recognize a fraudulent face. It identifies eyes, nose and the ratio of length and distance of individual facial elements. It can even discern the differences between identical twins. The only face characteristics not taken into account are hair, mustache and beard; this is to make sure shaving or changing the hair style won’t impact the recognition capabilities.

    Here are the three major challenges for computer vision development.

    Legal and Compliance

    First, at any computer vision development, it is crucial to have the General Data Protection Regulation (GDPR) in mind. You must ensure you are allowed to manage or process certain biometric data. Imagine the city government installs cameras over the city parks, these are useful not only for recording but they are also capable of tracking people, thus enhancing the streets security. In this case, people’s identity has to be somehow recognized, and the way of doing this is by processing biometric data. If the person tracked does not know this is being done, a GDPR violation happens.

    Anytime a system needs to process people’s biometric data that can identify them, they need to give consent to process this data. To solve this, intive installed a button on our API that releases the biometric calculations and the person’s identification. By pressing this button, the person automatically gives consent to start the process. Additionally, while enrolling to the system, employees need to sign the consent.

    Same Environment, Always

    In every computer vision case, environment is an aspect you will always have to deal with. This refers to the technology used for acquiring images during the development, testing and productive stages of the project: the type of camera and its calibration, the drivers, libraries, settings and the light conditions.

    To ensure the API’s best accuracy, all stages must be developed with the same environmental conditions. Otherwise, the system loses its best accuracy and could come-up with worse results in the productive environment and in the development one. When environments change, the error rate rises.

    To ensure success, it is crucial to collect all possible data from the final environment and retrain the developed model with this data. For our computer vision project, we created several different production environments and we gathered all the data that came up. There will always be differences, for example, the lighting conditions are hard to control in every stage. But the more data you have, the fewer errors will appear.

    Set Acceptance Criteria and Meet It with the Data You Have

    From the beginning of the project, it is very important to set the accuracy you are aiming for. This means you need to decide how often the system recognizes an enrolled user correctly. To understand this better, if you have 10 users, and you set a classification accuracy of 90%, this means nine people are going to be recognized accurately.

    False positive rate (FPR) is linked to this and it needs to be decided too. This value represents how often the system recognizes a person that enters the system with different characteristics than the enrolled one. For example, if you are using the system to control employees’ entrance to an office, how often will a person who is not an employee (and not enrolled in the system) gain access to the building?

    Both values, accuracy and FPR, are strictly bound. Better accuracy is directly related to an increase in FPR. You need to decide if you want better accuracy and higher FPR or lower accuracy and lower FPR. This can be controlled smoothly along the course of the development if you set it according to the available data.

    Accuracy and FPR will change based on the quality of data–the photos used. The more pictures (data) per enrolled person, the more accurate the system is. Pictures from different angles allow the system to have a better chance of being effective. In our case, we took a minimum of five pictures for every enrolled person.

    Working on a computer vision project with facial recognition software can seem intimidating. But it is just a matter of being well organized. Take into account the three challenges highlighted above and you will be on the right track to success.

    — Grzegorz Tymiński

  • Progressive Web Apps are Taking Over the Mobile Web

    Progressive Web Apps are Taking Over the Mobile Web

    Consumers now spend most of their time on their mobiles, that’s no secret. And with 4G networks now the norm in both developed and developing markets, people have every reason to expect their mobile experiences will be as good as PC-based ones.

    But they aren’t for the most part. Mobile web is 49% of all online traffic, but mobile conversions remain painfully low at only 1.7%. This is a sobering statistic, particularly for companies who are under growing pressure to stay relevant as customer behavior continues to evolve.

    Part of the issue comes down to limitations in the user experience on mobile (i.e. smaller screens), but the bigger factor is speed. According to Google, 53% of users will bounce off a web page that takes more than three seconds to load. And yet, the average homepage still takes 15 seconds to load on a mobile phone.

    Unless a customer feels an extreme sense of urgency to make their purchase, it’s unlikely they will endure the pain of waiting for page after page to load as they try to get through a purchase selection and checkout process on their phone. Neither are they likely to immediately put down their phone and dig out their laptop to complete the transaction.

    Welcome to the Progressive Web

    Enter progressive web apps (PWAs), the future of mobile web experiences.

    While responsive web design (RWD) made most websites usable on a phone sized screen, the actual experience of interaction has proven too slow and clunky when compared to the same website on a PC. In response, many brands have developed their own mobile apps, hoping to win customers over with a smoother and more convenient service. However, aside from niche businesses with a high-touch, loyal customer-base, such as Starbucks and Facebook, most brands have spent a great deal of time making apps that are moderately successful at best, not to mention the hundreds of thousands, or even millions of dollars in development costs.

    PWAs represent a significant leap forward because they allow companies to deliver the user-friendliness and page load speeds of a native app, but directly from the company’s mobile website. PWAs also make it possible for developers to create online experiences that don’t just adjust to different screen sizes but can interact with device features (such as Camera, GPS and push-notifications) creating streamlined experiences that are optimized based on which device a customer is using. The result is a more consistent experience across devices, with no features lost.

    Most importantly, at a time when the war for attention has never been more intense, with PWAs, web pages load in a fraction of the time, keeping customers engaged rather than testing their patience and driving them to competitor sites.

    Take the Financial Times, a company proving PWAs can have a major impact on conversion. The FT recently went the PWA route to keep pace with surging web traffic.

    “One of our key metrics was speed. We determined if we made the site just one second faster, it increases engagement by 5%, and would ultimately add millions of dollars to our bottom line due to increased ad views,” said Cait O’Riordan, chief product and information officer at the FT.

    Analyst houses Forrester and Gartner are also bullish on PWAs. Forrester went as far as to say there is no longer a point in arguing over which kind of mobile app is best–native or hybrid–because with PWAs taking off, nobody will bother using apps soon. Even among airlines, whose apps are some of the most useful and commonly downloaded, progressive web is seen as the future.

    “Apps are a temporary necessity in the development of better mobile experiences,” admitted Scott Wilson, vice president of eCommerce merchandising at United.

    Democratizing Mobile Access

    In addition to supporting faster and more responsive mobile experiences, PWAs also make access to these experiences more democratic. This is particularly relevant in emerging markets such as India and Indonesia, where consumers have limited data allowances on their phones due to high costs.

    Until now, this has stopped customers from downloading multiple apps. It has also limited people’s ability to engage with brands because, for many, mobile is their only portal to the internet. This is all set to change with PWAs.

    PWAs demand a fraction of the data compared with traditional apps while offering nearly the same quality of experience, eliminating the access barrier and delivering a win-win for merchants and customers alike.

    Why Now?

    It’s no surprise the world’s leading brands, from BMW to Starbucks, have already deployed PWAs or are on the verge of doing so. Starbucks’s mobile app has long been viewed as a gold standard for customer experience and loyalty programs, but even the world’s biggest coffee chain understands PWAs are the way forward.

    The wider move toward progressive web will begin in earnest over the next year. More organizations have come to appreciate the revenue risk of getting left behind and have begun to prepare for a PWAs push between now and 2025. Five years from now, mobile will easily account for 60-70% of web traffic, and customers will not hesitate to funnel towards mobile sites that work quickly and reliably. As the disruption of the past decade has shown us, heritage is no guarantee of loyalty when it comes to digital experiences.

    Of course, the transition to PWAs can’t be accomplished overnight. It is a major strategic transformation that requires brands to take on a front-end rebuild of their websites. It’s also worth noting PWAs development costs can be on-par with those for a mobile app. However, unlike apps that need multiple versions for iOS and Android and require frequent updates on their respective app stores, PWAs are simply an evolution of existing open web technologies that make the web a better place both on mobile and desktop. The opportunity cost of becoming a laggard on the web is much higher than the investment cost of building a PWA.

    PWAs will soon be the foundation of all digital commerce globally, and the rollouts planned over the next few years will see leading brands set a new minimum standard for customer experience. Google already added a mobile speed score to its search result rankings algorithm in 2018, and there is widespread consensus that page load times and PWA compliance will only carry more weight in Google’s rankings as more people abandon their PCs for mobile devices.

    Customers around the world are also ready for better mobile experiences. With PWAs, brands will gain access to technology that will help them satisfy their customers’ appetite for speedy web experiences–and the race to capitalize has begun.

    — Peter Sheldon

  • How Product Managers Can Take Back Control with Feature Flags

    How Product Managers Can Take Back Control with Feature Flags

    Product managers live in a high-speed world and are always focused on releasing the most impactful features in the shortest amount of time. Since users have grown accustomed to regular UI updates and bug fixes completed in near real time, this frenetic pace at which product managers work will only increase.

    The process of pushing out new features or software fixes can be very stressful on product managers, and the one thing they crave to keep their sanity and ensure a successful release is control. Luckily, that’s exactly what feature flags provide.

    Dialing Up Feature Flags

    At their core, feature flags give development teams the ability to turn features on and off without deploying new code. Think of them as dials in the cloud. When the dial is attached to any feature in a product–whether a new one or an old one–it controls who sees it, who has access to it and how quickly it is delivered to an entire customer base.

    Feature flags let organizations break the tyranny of the Thursday night release by segmenting it into smaller bits to control exposure and enable a continuous release motion. You might start by turning on a feature to only internal users or beta testers so you can test in production to make sure nothing has been missed. From there, you have the freedom to ramp up the feature to your customers gradually before going to 100% availability.

    With this controlled exposure, development teams can lessen the panic and anxiety around the weekly release schedule. Having the ability to completely turn off features means feature flags also increase application stability and remove the need for hotfixes. It’s less often necessary to assemble a war room if there is a problem when you can simply turn off the feature in seconds. By working feature flags into the release process, teams take control of the who, what and when in terms of releases. They can also dictate types of users, demographics, locations and scale the release of the new feature.

    Feature Flags and Product Managers

    Outside of the DevOps team, feature flags can also be really powerful for product managers. Historically, the release of a new feature meant product managers had to coordinate with cross-functional teams to ensure the release was executed successfully. This meant making sure marketing was ready to publish a blog post about the feature, the sales team knew how to use and sell the new feature, and the product marketing team had documentation prepared for how the feature set works. Of course, all of this had to be perfectly synchronized with the engineers’ timeline for releasing the feature.

    With feature flags, product managers now essentially have the keys to the kingdom. It gives them the power to turn the feature on for some or all of their customers without waiting for engineers to complete their next deploy. Not having to coordinate that extra piece of the schedule gives a lot of power and time back to the product manager.

    A Deeper Look at Customer Feedback

    Another aspect of a product manager’s job is collecting feedback. They need to be able to answer questions related to business KPIs, such as “How is this feature performing for my customers?,” “Are customers able to do the action the feature is intended to perform?” and “Are we generating more revenue from the feature?” Feature flags can also help in this area.

    As product managers ramp up a feature, they are simultaneously shortening the feedback cycle. Since feature flags allow for segmentation, where one customer group sees one set of features and another customer group sees a different set of features, and since key metrics can be captured into separate buckets for each group, product managers can now run controlled experiments as part of the release cycle. Seeing KPIs aligned with the different customer groups helps them understand which features are performing the most successfully.

    Creating the Ideal Product Roadmap

    As product managers embrace usage measurement tied to feature rollouts, entire teams can better align with agile best practices to continuously push out the smallest incremental set of new features. By understanding the performance of every feature, you’ll be better equipped to iterate and refine your ideal product roadmap.

    — Dave Karow

  • FaaS is Key to DevOps Efficiency

    FaaS is Key to DevOps Efficiency

    When AWS announced Lambda–a serverless Function-as-a-Service (FaaS) technology offering–the idea of encapsulating a specific task into a function that resides in the cloud was introduced into mainstream access.

    Now, with the rising popularity of microservices, DevOps is waking up the benefits of FaaS where it is used to build and maintain enterprise applications in agile environments. The move to serverless architecture using FaaS is one organizations are now increasingly likely to investigate in 2019.

    Given these industry realities, we need to ask ourselves what place will FaaS take in contemporary computing environments and how will its core efficiency proposition enable DevOps to flourish and improve experiences for users at every level?

    Back to Focus

    First let’s remind ourselves what any FaaS platform really does. It allows us to focus on developing and executing business logic because server provisioning and management is handled automatically at the backend. Developers can deploy applications into a space with zero server administration, yet with the ability to enjoy high availability and high scalability.

    The impact for DevOps is immediately apparent.

    Developers are able to work in a zone where they are essentially abstracted away from the servers their applications live and breath on. The services developers draw upon are event-driven and eminently scalable. Even if they have to scale by some massively parallel level of magnitude, server-side service optimization is handled at the backend by the FaaS vendor.

    At the same time, the cost structure is shifted and operations staff are able to oversee compute billing models based on actual execution of functions, as opposed to being dependent on the physical size of a pre-provisioned cloud server established at best guess size and core optimization parameters for compute, I/O, storage, analytics and so on.

    Speedier Implementation

    If we have so far looked at the FaaS efficiencies that reside at the backend server-side management element of the equation, let us also turn our attention toward the way FaaS allows developers to reduce the amount of time it takes to deploy code in a new environment.

    Adopting a FaaS-centric approach to application architectures allows the developer team to encapsulate specific software tasks into functions, hence the name. For a mortgage calculation application, for example, we can encapsulate functions such as interest rate calculations, payments, length of contract etc., and then place those functions in the cloud on a FaaS platform. The FaaS provider is responsible for the hosting of that function and also looks after the heavy lifting associated with availability and redundancy.

    The benefit to developers is a very rapid environment for deploying new code, getting feedback and iterating. The operations benefit is one of hosted happiness as the FaaS provider shoulders a significant administrative burden.

    FaaS and microservices are now increasingly viable options in any modern technology architecture and it’s important to distinguish between the two.

    Microservices are made up of self-contained elements of business logic designed to perform a very specific job. They are optimized for their single responsibility and can communicate with other services via application programming interfaces (APIs). Further, they can be developed, deployed and scaled independently without impacting other services.

    A FaaS-based approach is well-suited for transactional workloads where code is run periodically. This is also a technology that is quicker to establish, as and when needed. A FaaS function might also make a call to other application logic that (in the case of our mortgage example) calculates interest rate and so on.

    FaaS Challenges

    Like any still-nascent technology, FaaS has its functionality flaws and challenges. The automated nature of foundational provisioning in FaaS is great news, but it’s not a panacea.

    Why? For one, DevOps teams will have limits on the debugging capabilities for applications that run on FaaS DNA. Also, functions are instantiated and then de-instantiated as needed, which requires time, especially for a cold start. This process can cause unwanted delays in time-sensitive operations. The transient nature of FaaS also impedes state preservation among dependent functions, which requires careful design considerations.

    FaaS is a place where the burden of server resources control has been shifted. But there’s no such thing as a free lunch and this means DevOps won’t be able to interject some level of application monitoring function at the server level itself. You’ll need to know more about your FaaS-based application before you FaaS it up.

    We must also remember for any distributed software architecture, there’s an up-front responsibility to design to accommodate the limitations and increasing options available in modern services.

    The FaaS Road Ahead

    Examples of FaaS now extend beyond AWS Lambda, we can also look to Microsoft for Azure Functions, Google for Cloud Functions, IBM OpenWhisk, Oracle Functions, Fn Project and IronFunctions with its Iron.io platform.

    This sector of the software landscape is guaranteed to grow over the coming years. FaaS can be an appropriate option in an overall software architecture, but there are specific limitations, so choose wisely. Observing the evolution of FaaS over the coming years will be a fascinating process.

    — Mike Duensing