devopsy.kinsta.cloud

Tag: engineers

Avoiding the Pain of a ‘Resume-Driven Architecture’
Successful new technologies follow a predictable arc, from initial innovation to technical gold rush and, ultimately, to mass market adoption. As a consequence, they achieve strong standardization and productization. Being highly attuned to where a specific field lies on the maturity arc is how you yield the greatest benefits to a business. But what happens if your developers are not thinking about this, too? You end up with what I call a “resume-driven architecture,” and it can be highly costly to your business.

A resume-driven architecture occurs when the interests of developers lead them to designs that no longer align with maximized impacts and outcomes for the organization. Often, the developer clings to a technology that provides them a greater level of control and, at least initially, a higher salary. Meanwhile, the organization gets an architecture that only a handful of people know how to manage and maintain, limiting the available talent pool and hindering future innovation.

To understand why this happens and how to avoid it, it’s helpful to look at this technology arc in more detail. It’s a pattern that has played out up and down the stack, with technologies like big data platforms (now dominated by Snowflake and Databricks), Kubernetes and even the broader move to the cloud. Failing to grasp this concept of a resume-driven architecture can increase costs and hold back innovation.

Here’s how this technology arc typically plays out, using Apache Spark as an example:
- Stage One: A technology breakthrough like Spark emerges, providing a significant advantage to early movers.
- Stage Two: Other companies see the benefits, and the technology receives widespread interest. Specialists are in high demand and command big salaries. Many adjacent professions naturally invest in becoming Spark experts.
- Stage Three: The technology matures and reaches a level of standardization with an ecosystem of tools and services around it. Specialized knowledge, such as how to tune your Spark job parameters, becomes less valuable.
- Stage Four: Mass convergence, productization and simplification occurs. As examples, Databricks becomes the primary Spark vendor and launches SQL warehouses for easier data processing. Snowflake launches Snowpark for Python as a simpler model than classic PySpark jobs. Nobody worries about tuning most parameters anymore as the underlying technology becomes part of the stack: Powerful but almost invisible.
It’s important for businesses to take advantage of this productization stage when it comes along. There’s no sense in investing resources in a bespoke architecture if it’s not providing you with any differentiation—especially when competitors are achieving the same outcome with fewer resources.

Moreover, getting stuck in a Stage Two mindset when the field moves on to Stage Three (or, worse, Stage Four) and cuts you off from the next wave of innovation. Subsequent technology breakthroughs often build on top of—and interoperate with—the previous technology layers. If you’re stuck with a custom architecture when the industry has moved on, you can miss out on the next wave of innovation and fall further behind competitors.

The problem is, developers who’ve made a great career from implementing custom architectures are often reluctant to move on. After all, mastery of that previous innovation is what got them to where they are. They may not have realized that the world has progressed around them, or they may worry a standardized architecture with managed services and off-the-shelf products undermines their power and autonomy. This is how organizations fall victim to a resume-driven architecture. I’ve observed this several times when speaking with engineering leaders, and it’s become more common with the rapid innovation happening around data.

As engineers, our value is in building things and, ideally, in solving innovative, artisanal problems. But this identity and desire can confuse what we want from our own jobs with what our team needs from us. When standard patterns and standard products emerge, we need to adopt them because that’s the way to maximize our impact and ultimately free our time and energy to move further “up stack” and have an even greater impact.

Developers who would rather stick to a custom architecture often argue that they can squeeze out greater performance, but this is rarely a justification. You may end up with an app that’s 10% faster, but rarely does that come close to making up for the time and energy invested, the loss of flexibility and, ultimately, being held hostage by your own technology stack.

So how does an organization avoid the trap of a resume-driven architecture? Strong leadership is key. Look critically at the business and ask which areas of your stack are truly differentiators because it’s only those areas that may—just may—justify a unique architecture. If you do go that route, it needs to provide huge benefits to the business to be worthwhile.

For example, “data” is not in itself strategic to you; every company is now a data company, and the vast majority of things you do with it—how you source, store, process and distribute data—is no different from what everyone else is doing. What does matter are the data products that fuel your business and, most importantly, how fast you can build them.

That’s why, as a developer or engineer, bespoke stacks are not your friend. Instead, architect your platform in a way that allows other developers to onboard quickly; that allows you to upgrade and swap out components easily over time and that doesn’t leave you held hostage to a developer whose main interest is retaining their value by sticking with an unnecessarily complex architecture.
June 13, 2023
Solving the Culture Conundrum in Software Engineering

The one crucial ingredient underpinning our transition to a digital-first economy, more than any other, is software. That’s why software engineering has become one of the world’s most in-demand professions. In the UK alone, the median salary for the role has increased 13% year-over-year as businesses look to stay ahead of the competition and attract the very best IT talent. But the high salary often isn’t consistent with where software developers stand in the business; many organizations still view them as suppliers rather than as crucial players in their success. Based on the current speed of transformation, we know that demand for software engineering is only going to increase. So business and technology leaders need to urgently consider vital cultural changes to help better integrate software development teams within the wider organization. But what exactly should they be?

The Development Dilemma

Agility has quickly emerged as a competitive advantage as businesses strive to deliver on customer expectations quickly and effectively. That’s heralded the rise of DevOps, which aims to write high-quality digital solutions more efficiently and have the ability to continuously update in a secure and low-risk way.

The practice is already accelerating feedback loops in the software development of digital solutions on a wide scale, meaning new software is brought into production as fast as possible and with reduced risk. As a result, today’s successful businesses are now releasing software updates multiple times a day. The figure even runs well into the tens of thousands for tech giants like Amazon and Netflix. However, other businesses are struggling to keep up with the speed the market demands, largely due to a lack of clear communication, ways of working and integration between software developers and the wider business. It’s a dilemma that needs immediate attention if businesses are to remain at the competitive edge.

Developers are too often left to deal with huge monolithic pieces of work, which require them to write code for months on end without it being road-tested. This inevitably leads to mistakes that are found only once the updates have already been released to market creating a ripple effect throughout the organization. It’s an all-too-common scenario that demonstrates business leaders’ clear lack of clarity and understanding about the colossal value adopting a DevOps culture can deliver for the wider business, including its customers.

Take financial institutions looking to introduce a new offering, for example, such as a new mortgage product. Without a collective approach to development, getting the backend systems up and running for a product can take up to six months. But modularising the options and breaking down the functionality can shorten timelines to weeks.

Business leaders must look to create unified, cohesive and cross-functional teams that work together on projects that are broken down into simplified and localized components that can be easily integrated together without a negative impact on each other. It might seem like a small strategic change in working practices, but its impact on the speed and efficacy of software development can be enormous.

Revisiting Personalities and Processes

The role of the software engineer has changed; it is no longer about writing code in isolation without much regard for or knowledge of how it benefits the business. Developers work better when they have clarity about the direct impact their work will have on achieving business goals and on the bottom line. It’s down to business leaders to communicate these challenges and goals (in other words, understand the “why”) to help software developers understand what they’re trying to achieve. But doing so in a way that moves towards a better working culture requires a new approach to building and managing software development teams.

The first step is casting aside the negative stereotypes many have of software engineers and celebrating the intellectual and cultural diversity within their teams. Diversity of personnel brings a diversity of personalities, which is crucial to creating more inclusive cultures that accept and welcome all characters with open arms. While this may seem obvious, what is often overlooked is the impact diversity can have on stimulating and increasing innovation.

Improving the diversity of development teams helps unlock multiple ways of responding to and analyzing the challenges brought forward by business leaders. After all, each individual has an array of different soft and hard skills based on their learning, development and experience. So, encouraging individuals to bring their characters and personalities to their roles is crucial; it ensures each and every stakeholder in the development process is empowered to contribute, feels inherently valued, and is able to challenge existing assumptions, helping deliver a more robust end product.

Recognizing and harnessing this spectrum of skills also helps bridge many of the barriers to development that exist within businesses. Some software engineers, for example, might struggle to communicate technical updates in a way the wider business can understand, but tapping into another individual’s strong communication skills can help unite teams–and entire businesses-around one, holistic digital strategy.

Channeling Change to Deliver Better Outcomes

Clearly, cultural change is needed to embed software engineering at the core of modern business. But revisiting project management practices is also imperative to continually deliver software that provides value to the business, the world of scientific management and Waterfall delivery methods is no longer appropriate for the development of digital solutions. With such a heavy reliance on digital services, businesses—and, consequently, developers—need to be able to make changes with minimal effect. So, eschewing monolithic projects and encouraging cross-functional teams is essential. Not only does it reduce the need for sweeping software updates, but it negates the risk of a project failing to meet the initial brief, which could ultimately impact the entire business.

By reviewing organizational processes, and bringing the wider business and software developers together, we can create teams that deliver digital solutions effectively and quickly, so that every user, whether internal or external, reaps the benefits of a more sophisticated approach to software engineering.

November 3, 2022
Why the Linux Kernel is Still Important

Last year, the Linux kernel turned 30. If you’re someone who’s been immersed in the Linux world since Y2K like I am, it may feel a bit surreal that so much time has passed since the kernel’s inception.

As a training architect at A Cloud Guru (ACG), I teach courses about all things Linux and specialize in hands-on lab-based learning. Before joining ACG, I worked as a Unix systems engineer at GE and IBM as well as a technical account manager and customer advocate for Red Hat. I’m hugely passionate about Linux, just as so many other engineers and enthusiasts are, not just because of its importance to our careers but its impact overall.

In its 30 years of existence, the Linux kernel has had a massive impact on the modern computing landscape—revolutionizing what’s possible for operating systems and allowing countless tinkerers to get their hands dirty in the process. Linux has also become the foundation of paradigm-shifting innovations over the years due to its ever-evolving nature.

Here are some of the key reasons why the kernel is so valuable and why it will continue to be a major player in the computing landscape for years to come.

My Own Linux Journey

My first exposure to Linux was in college in 1999. I took an introductory Unix Shell Scripting class and Fedora was installed on the lab servers because it was free and easily scalable. I view Linux as my entry point into customizable operating systems. After learning basic Unix commands on Linux, I ultimately went on to pursue a career supporting multiple commercial Unix vendors.

As Linux made its way into larger enterprise-level companies, I quickly returned to working with Linux distributions. Something I’ve always loved about Linux is the capacity to fine-tune your system to support the applications and distros that work best for your projects. Linux improves the functionality of whatever applications you’re running. The Linux kernel has literally changed how the world processes information, which is why I’m so invested in the software.

Now, it’s my job to share my knowledge and passion for Linux with other technologists. At ACG, I develop courses to help aspiring Linux experts learn how to optimize their systems. Additionally, I contribute to Linux this Month, an ACG-hosted web series that provides monthly updates from the global Linux community.

The fact that I am able to build a career around teaching Linux and staying up-to-date with Linux news shows the vast use cases and applications of the kernel. What makes Linux so unique and evergreen is its open source nature—Linux innovations are only limited by the creativity of the technologists who use and adapt it.

The Open Source Effect

When Linux first arrived, it was mostly a hobby for enthusiastic engineers and computer science students who could contribute by developing code. The steep learning curve associated with fitting Linux to your machine was a barrier to entry for more novice programmers.

Over time, this has changed considerably. Online forums, workshops and classes have made Linux more accessible to the average internet user. The free sharing of ideas has come to epitomize the open-source community, and for software engineers, Linux is at the heart of this community. This democratization of Linux has had incredibly positive impacts on the computing world.

Now, Linux is everywhere. Enterprise-level companies use Linux distributions to process the biggest production workloads in the world. It has replaced proprietary commercial Unix operating systems in very large companies with better stability and less downtime. Because Linux systems can be as small or as large as you want, it’s also now being used in our homes for smart and mobile devices as well.

The open source nature of Linux is incredibly beneficial for these enterprise-level companies. Linux distributors leverage the contributions of the entire open source community. This wide range of contributors produces a more stable product with more features but also ensures that the OS keeps growing and solving real-world problems that are beneficial to a wide range of users.

Linux Kernel Runs the Cloud

Potentially the most impactful outcome of the kernel is its importance to the infrastructure of modern cloud computing. Linux’s scalability has paved the way for supercomputers and server farms to function efficiently while requiring relatively lightweight computing resources. In fact, Linux supports about 90% of the public cloud workload.

Without Linux, the cloud as we know it would not exist. This is, in part, because Linux has become so ubiquitous—its use cases are nearly limitless. Because it has been time-tested, many engineers and IT professionals have a solid grasp of Linux fundamentals, making it an attractive choice for enterprise companies dealing with the cloud.

Cloud-based software and products are increasingly becoming the norm in the engineering world. Unsurprisingly, major cloud providers such as AWS, Azure and Google Cloud are all supported by Linux as well. Linux is unique because it is a shape-shifter that can conform to the needs of any given engineering environment, and it’s incredibly stable because of the army of contributors that fortify weak points in the software.

Linux Forever

The concept of Linux is just as important as the Linux product. The concept allows a free and open source operating system to be refined, reinforced and replicated across an endless web of contributors. Thirty years is a long time for software to be relevant, especially with the ever-shortening tech product cycles. Because it was designed with the intent to be changed and updated by an open source community, Linux has no foreseeable expiration date.

Had Linux not achieved the prominence it has today, we would see more commercial Unix vendors attempting to solve some of the problems that Linux addresses, but none would address them all. Additionally, customers would have to choose which OS to invest in based on which addresses some of their use cases, but none would be as beneficial as the Linux OS.

Linux is always growing and will become even more popular within the next few years. As more people become familiar with Linux and learn to use it, I see major potential for growth in the mobile computing space, within personal computers, and across small and large companies. In fact, we are already seeing it filter into home gaming systems and Raspberry Pi projects. With Linux, the sky’s the limit!

March 17, 2022
DataOps: The Key for Real-Time Data Application Development

Data engineers and developers face challenges every day to help their organizations digitally transform. To do this, they must deliver real-time data applications faster, better and cheaper. With businesses in every industry now data-driven, data professionals must work more efficiently and accelerate time to market for their products. That’s where DataOps comes in.

Development of real-time data applications slows down when strategic business project teams are dependent on small, centralized engineering or IT teams for basic data operations such as exploring data or even building most data processing applications. Needlessly, engineering teams are notorious for being a bottleneck for strategic digital transformation initiatives. Rather, they can become known as strategic project enablers that provide the best tooling to their customers—instead of building it and being known as gatekeepers.

Business users, outside of IT and engineering, need access to data but lack the necessary deep technical knowledge to work with the latest data technologies. DataOps brings people closer to data, helping organizations accelerate the delivery of their strategic projects, reduce costs and minimize the need for hard-to-source engineering skills.

To accelerate digital transformation initiatives, teams and organizations should onboard new users directly onto a data platform quickly with the right tooling and resources required to get to work on their projects. DataOps empowers these business-focused users to work with data tools on their own to self-serve. A DataOps approach makes it easier for more people across an organization to work with its data, resulting in better alignment for data engineers and developers with their business and technical partners and improved data access.

Serious new security, compliance and governance risks are introduced when less-technical users increase their interaction with technology, such as onboarding to a data platform. More people now understand that personal data has tremendous value. Initiatives such as the General Data Protection Regulation (GDPR) and California Consumer Privacy Act (CCPA) have brought the need to respect data privacy to the forefront. The stakes for safeguarding data are increasingly high and regulators are unforgiving—costing some organizations several billion dollars in fines, lawsuits and loss of market cap for some data breaches.

When providing access to data, stringent management, control and real-time monitoring must be applied from a governance perspective. A surprise compliance auditor visit can result in questions such as which applications are reading credit card data, how an organization is keeping customer data safe, who has access to data and what are they doing with it. While locking down for compliance, you must help your business stay agile and avoid any slow down in data initiatives.

Build vs. Buy to Streamline Operations

In addition to adopting a DataOps approach, a key step is to onboard users to a data platform. This will optimize your data strategy and will help take the worry out of managing data projects, putting you on a path to faster execution and accelerated time to market.

When onboarding users to a data platform, you need to choose between building your own or buying. It takes time and resources to develop your own system to onboard users and clients and can take years to build your own technology in-house. If your engineers are spending all their time working with open source technology to build tooling for onboarding, they’re not focusing on better ways to use data and get solutions to market fast—which is an inefficient approach.

When your users are onboarded, the cycle begins again, as users request access to more data sets. More time at the command-line console means less time for your engineering teams to focus on higher-value work and strategic initiatives.

Get a Leg Up on Competition With a DataOps Approach

Buying a solution requires an upfront investment to support onboarding and, when combined with DataOps principles, is formidable to remain competitive. Together they enable support and manage the onboarding of clients to a data platform at scale and to maintain security and compliance. A DataOps approach fosters improved communication between data and business professionals to share data more effectively and to build a data mesh architecture.

A Swedish digital bank, Avanza, needed to ensure the continuous improvement of the customer experience of its digital services to remain the leader in the Swedish market. Doing so required giving developers direct and constant access to real-time business data in Apache Kafka.

The organization applied a DataOps approach and intelligent data masking rules to meet GDPR and protect customer confidentiality, featuring a large single data platform to address how it delivered data access and applied data governance to support more than 20 strategic projects. Using the platform, tenants who build streaming applications gained visibility into the health of their Kafka clusters and improved insight into data flows. This self-service approach means fewer support calls to data engineers, minimizing bottlenecks and freeing them up to work on more strategic initiatives.

Data is the fuel that propels insights by tying back to analytics, decision-making and business systems. The better the shared data, the more it can be applied to supporting better business decisions, better customer experiences and execution of strategic business objectives and goals.

DataOps also builds compliance and governance into your processes. It provides the visibility needed to understand who is accessing data and how to maintain governance and ensure people are acting ethically with the data they are enabled to access.

The daily challenges faced in data-driven organizations only continue to grow. A DataOps approach helps data engineers and developers meet these challenges to digitally transform and build real-time data applications that help organizations gain a competitive business edge in their respective marketplaces.

November 5, 2020
Managing Remotely: 6 Ways to Support Engineers

Managing remote engineers effectively comes with a host of considerations. Here are six to keep in mind

Dear 2020, we’d like our deep work back.

Our shift to remote work in 2020 moved the majority of face-to-face interactions into screens. Although initially we assumed this shift was a temporary adjustment, most of us now realize it’s just the opposite. Twitter, Facebook, Shopify, Slack, Box and more have publicly declared remote work is here to stay and they are not alone—84% of businesses now report they will increase remote work post-COVID-19 pandemic.

With our new reality came next-level distractions—homeschoolers, playful pets, social unrest and, oh yes, a scary global pandemic. Moreover, the lockdowns rewired our collaboration habits, stretched our bandwidth, and threatened our ability to focus on our best work.

As a result, managers—the forgotten middle of the workforce—are emerging as quiet heroes fighting to protect team effectiveness. At Uplevel, we’ve worked with engineers nationwide and have learned the top ways that managers can help their engineers empower their best work:

Mind the Meetings

Engineers want their managers to protect their time for deep work. They are constantly bombarded with changing priorities that require endless meetings, crushing their ability to dive in, solve problems and innovate. How bad is it? Of the 250 engineers and managers we interviewed, 50% of individual contributors reported that more than twice a week they don’t have enough time to do their work, with 19% of engineers reporting it’s an issue every day. Bottom line: If a meeting “coulda been an email,” make it so.

Say No to Micromanagement

Nobody likes the helicopter parents of work. The constant hover erodes trust, destroys morale and slows down otherwise efficient, effective people. Top performers will tell you, autonomy contributes to their overall happiness at work, and research continually tells us that happy employees are more productive and they stick around longer. Instead of spinning on the minutia, make it your mission to block distractions and tackle bottlenecks for them so they are empowered and available to do their best work. More than one-third—39%—of engineers are blocked on issues they can’t unblock themselves at least once a week. That’s a weekly chance to free your team to do their best work.

Embrace Flexibility

For the sake of protecting “the flow,” it’s a good time to go with the flow, too. Overly structured, old-school project management must give way to the fluid, flexible tools empowering us to get work done anytime, anywhere. Embrace a project management style that is as fluid as your team’s changing priorities and consider light-touch micro check-ins that leave room to revisit the game plan weekly.

More Comms, Not Less

Right now, it’s easy for individual contributors to feel both socially and professionally isolated. As a result, their confidence waivers and paranoid flares as they question whether their contributions make an impact or get noticed at all. Now is not the time to back off of communications. Don’t confuse this with “respecting personal boundaries.” If you’re worried that communicating will somehow exert unintended pressure, simply establish clear expectations and model them by example. A simple way to lean into communications is to make them more frequent, much shorter and less formal. By acting more like a coach than a manager, you’ll not only keep the team focused on the right tasks but you’ll also show them you care about their well-being beyond “the work.”

Lead By Example

The quickest way to zap trust and kill morale is by ignoring your own rules. Engineers want their managers to lead by example and consistently exhibit the same behaviors they expect from their teams. If it’s respecting work/life balance (and refraining from midnight emails), don’t hit the send button until it’s time. If it’s a commitment to stay present during group meetings, stop multitasking. If it’s a promise to wear pants on Zoom calls, put your pants on. You get the point: Don’t create rules for remote teams that you’re unwilling to follow yourself.

Create Opportunities To Connect

You’ve heard it time and again since March: We’re in this together. It’s true. No one rolled into the pandemic with a playbook; managers and their teams are winging it. Forced isolation is not a normal human state and it shows. Across the country, doctors report more cases of depression, domestic violence and substance abuse. In fact, a poll by the Kaiser Family Foundation shows the pandemic is taking its toll—45% of adults say this pandemic has affected their mental health. Another study by mental health provider Ginger found 69% of employees admit this pandemic has been the most stressful time of their career. How can you help? Use this collective behavioral beta test as an opportunity to show empathy and connect. Are team happy hours gone? Replace them with the dogs and cats of Zoom at the beginning of your weekly meeting. Are team lunches no longer doable? Have virtual outdoor picnics and encourage your team to get some fresh air. Be creative—it’s a great time to rally around the things that connect us as humans, not just teammates.

Thanks to the COVID pandemic, 64% of us now commute from our beds to our home office (or the kitchen table, or sofa). That’s a ninefold increase in four months. As it turns out, many of us don’t mind the change. A full 60% of us would actually prefer to keep it this way. If our engineering teams stay virtual, it’s our job to empower them so that remote work can still bring about their best work. As we embrace this ever-changing world, now is the time to tap into our empathy and communication and leverage the data and technology available to us to ensure we turn hardship into innovation.

August 26, 2020
How to Reduce Engineer Burnout During COVID-19

Burnout is increasingly common during the COVID-19 pandemic. Here’s how to cope effectively

The COVID-19 pandemic has changed the way we all work, connect and live. Many teams have been forced to transition their work into their homes and to learn ways to organize and execute that they aren’t used to. For product and engineering teams, development now occurs in isolation; meetings are held via email, messaging app or calls; and decisions are made asynchronously. The feedback loop we are used to has slowed and often links in the chain are broken. Additionally, our working environments have changed dramatically—many of us are working from unfamiliar environments unsuited to focused work, while juggling the needs of those we live with who also need space to work or who may be learning to operate without a job or school to go to.

This is a new kind of stress that most of us have never before experienced. Expecting to complete projects in the same manner as before while working in a different environment and moving at the same pace is inevitably going to cause burnout. On top of that, the COVID-19 pandemic has caused many businesses—especially online retailers—to ramp up their project development and even start brand new projects to adjust to the new normal.

Burnout is more common within organizations than many like to believe. It can’t be measured easily by the number of hours worked or the amount of sleep one gets. It is more often related to prolonged stretches of elevated pressure and to operating from a place where you start each day feeling behind, fighting against quicksand. Burnout can cause exhaustion, trouble sleeping and an inability to focus. It can also have an impact on a person’s mental and emotional well-being, creating a state of fragility born from overtiredness and frustration.

These symptoms can lead to a loss of empathy, a rise in conflicts and a decline in performance. Burnout can also cause people to feel unsatisfied with their job and question their role and purpose within an organization. Not only does this affect employees, but it also has a negative impact on the entire business. A survey from Gallup showed that burned-out employees are 60% more likely to take a sick day, and more than twice as likely to be actively seeking a different job.

Now more than ever, it is important for leaders to be watchful for burnout among their teams and to take proactive steps to protect team members. When performance goals are missed, it is far more effective to readjust expectations and set new timelines than it is to count each day that slips past the old one.

We are all going through an unprecedented time, which takes its toll on employees in different ways. If a teammate is struggling with their work or difficult in their communication, there very well could be factors going on outside of the current work situation that are contributing. Old processes should be reviewed and new ones invested in to make workflows purposeful and more effective in the current climate. It is common for employees who are experiencing burnout to view their performance as a personal failure, which leads to demotivation in their job.

Engineering teams can alleviate the stress that leads to burnout by actively and visibly reinforcing the value that the team is delivering and by ensuring progress is achievable, measurable and frequent. An agile workflow can be a great way to do this. By incrementally releasing features and achieving milestones, the team can avoid an endless push. By measuring the results of those changes, the team will see the value that they are delivering and gain greater confidence in themselves. For engineering teams who are responsible for production systems, finding ways to minimize the noise in your monitoring can help reduce context switching and give everyone more all-important sleep.

One way to achieve these results within your organization is to make feature flags a core part of your workflow. Feature flags allow for engineering teams to operate with greater autonomy, as code developed by one engineer can be released before its dependencies are completed, safe in the knowledge that the code will not be exposed to users. They allow for enabling the code safely and incrementally, minimizing the stress of releases and giving teams an escape hatch should an issue occur.

The benefits of feature flags also extend to measurement, as they are the enabling tool for conducting experiments as a team no matter where you are. Experimenting, especially in the form of A/B testing, allows teams to get a better understanding of customer behavior.

As people are isolating themselves from physical spaces, our digital spaces are changing rapidly. The measurement of that change is driving the success of businesses around the world, and teams that understand that can gain greater motivation for the work that needs to be done. For companies that are struggling in these uncertain times, effective measurement and control of your development can help focus efforts toward the areas of your product that are best addressing your customers’ new needs and behaviors.

As our teams have spread apart, it is more important than ever to work together. Validate projects early, deliver work incrementally, measure accomplishments and progress and empathize with your teammates. The risk of burnout is greater now than ever.

June 25, 2020
Important DevOps Roles Every Organization Needs to Succeed

There is no agreement between the scholars and the practitioners of what the DevOps concept necessitates. There are many definitions available for the term “DevOps,” but it is the term used for emphasizing the collaboration between operations and development.

Development (Dev): This is a process for creating a software product consisting of interrelated activities of software requirements, analysis, coding, architectural design, integration, testing and acceptance testing.

Operations (Ops): This is a process of putting into use, and supporting end users in the use of, software product in the operational environment. Its activities include upgrade, installation, operation control, migration, configuration management, monitoring, alerting, support and availability.

DevOps Roles Required for an Organization

Many IT organizations are looking to embrace DevOps to increase their pace of software development and realize the business agility that stems from accelerating and streamlining the communications between operations and development.

However, there is no fast or simple way to achieve DevOps. Many organizations have approached it as a pure technology play, but the successful path begins with the best people in the right DevOps roles with the right DevOps skills and an interest to collaborate.

Experts in IT need to empower and employ evolving professional roles. Working as a lead change agent to supervise the transition, these DevOps roles include automation architects, security engineers, release managers, experience assurance experts, developer-testers and utility technology players who understand operations and development.

DevOps Evangelist

This individual needs to promote the benefits of DevOps by quantifying and identifying the business benefits that come from the higher agility that DevOps delivers. Reporting as a change agent, the DevOps evangelist ensures buy-in from operational and development teams, recognizes the important roles to handle DevOps delivery methods and ensures IT professionals are empowered and trained to make those changes.

Release Manager

The release manager is responsible for addressing the coordination and management of the product from development through production. Primarily they work largely on technical details and obstacles project managers are not involved with. Release managers supervise the integration, coordination, development, testing and deployment to support continuous delivery.

Automation Architect

The role of automation architect becomes important in DevOps, as this technology depends largely on automated systems. They are responsible for designing, analyzing and executing strategies for persistent integration while conforming high availability on pre-production and production systems.

Software Developer/Tester

The heart of the DevOps organization is the role of a software developer. Under this technology, the title of a software developer will stay the same, but the job of a tester/developer slowly raises the scope of responsibilities. These developers are not only responsible to convert new requirements into code, but also deploy, unit test and handle the ongoing monitoring.

Experience Assurance Professional

While the quality assurance function is frequently part of the software development, a new type of control becomes necessary when organizations adopt DevOps. The need for QA testers is being replaced with a need for XA experts charged with making sure that new functions are launched with the end user experience in mind.

Security Engineer

Considering the conventional waterfall development, system security is immensely an afterthought. It is a nonfunctional condition that, such as QA, is frequently tacked on toward the end of the system development. DevOps-minded stores have security engineers working side by side with programmers, incorporating their recommendations much earlier on in the process.

Utility Technology Player

Traditional IT operations or systems management professionals concentrate on maintaining the servers running. The primary cause of service interruption is likely to be the applications being run on these servers, so the administrators introduce very strict controls on what’s allowed to operate on their servers. They need extensive QA in a staging environment, large transfer and operations documentation and very rare issues. Traditional programmers have been heads down-coders without any involvement in post production systems.

Conclusion

A successful DevOps implementation is one that begins with the best people in the right DevOps roles to address all aspects and promote the speed and agility afforded by DevOps.

— Saikumar Talari

August 21, 2018
Does the DevOps Engineer Exist? How Beliefs Can Influence Hiring

Does a DevOps engineer actually exist? That’s a risky subject to broach for a person who specializes in DevOps hiring. To the untrained eye, the easy answer is, Yes. However, to all the so-called DevOps engineers out there, the answer isn’t so cut and dry.

While valid stances around this vary, here are some things to consider when hiring a DevOps engineer and the effects your mindset will have on determining who you choose.

DevOps is not merely a set of tools but a philosophy that brings people together (cue two verses of “Kumbaya”) and structures a set of thinking that promotes personal ownership. This starts by bringing awareness to the consequences of individuals’ actions leading to a shared sense of responsibility toward creating successful products.

Therefore, just hiring one or multiple DevOps engineers is seldom enough. When thinking of these hire(s) as a cure to your lack of DevOps, you are inadvertently hiring for the exact opposite of what DevOps aims to achieve. DevOps is a mindset rather than a set of linear workflows.

It’s a methodology that must flow through cross-functional teams. It starts with your DevOps engineers, that’s for sure, but it does not end there, since the goal is for multiple teams to collaborate effectively toward the same end goal. By keeping your DevOps engineer(s) siloed in a “DevOps Department,” you’re solidifying the silo effect that DevOps was created to eliminate in the first place.

What Exact Change Can We Expect?

So now that you’ve hired a DevOps engineer through your trusty DevOps recruiter, what’s next? We know that just hiring someone with some DevOps skills won’t do the trick and neither will more hiring. The goal is not hire for experience with flashy tools but for technical expertise and enough interpersonal skills to spread a collaborative attitude. You’re looking for help restructuring how your teams collaborate, where everyone is now responsible for the finished product.

When a group of people share this responsibility, a culture of healthy pushback and questioning is formed. Sure, everyone still concentrates on their specialties; however, restructuring everyone’s priorities toward continuous improvement brings a newly shared goal that is usually new to almost all parties involved who have not experienced this before.

Whether you’re a Dev, Admin, QA or even a manager, this healthy questioning helps teams better understand the value everyone contributes. People begin analyzing their own work through a more critical lens and later do so when analyzing their colleague’s work (not always to everyone’s immediate delight). What were once separate departments can now gain a true understanding of what each group contributes.

People begin questioning less on how much weight their colleagues are pulling and more about how their own actions affect the shared end goal which seldom exists before a DevOps mindset is introduced.

Interviewing for the Right Fit

So, can you really categorize this hire as simply a DevOps engineer while they are championing your company’s new work philosophy?

A DevOps hire brings value that is not always tangible but helps mold companies and move colleagues’ priorities in the same direction. It begins with thinking about processes and aggregating everyone toward the greater good.

Technical expertise is much easier to hire. The greater challenge is getting everyone on board with this whole idea of DevOps. Sure, maybe C-level management is bought in, but how do you get your lifelong employees to embrace such a seemingly radical change? This is where an effective interview process can make all the difference in finding the right DevOps engineer.

Considering how communicative a DevOps engineer needs be to bring this change to tech and non-technical groups, a role-play method could help, in which your candidate explains DevOps to a nontechnical team member. It’s not enough to simply grill someone on technical acumen when you can more effectively see how they can make the idea of DevOps tangible to everyone in your company.

If they can only communicate with fellow technical experts, they probably are not the right hire. However, if they can put the effects of DevOps in layman’s terms for other employees, it’s a safe bet they’re right for the job. There’s nothing quite like a real-world situation that could realistically be one of their first orders of business once on the job anyway.

So, What’s Next?

Again, this is not just about a toolset and whether someone has leveraged Kubernetes or AWS. This is about hiring someone who can restructure teams and implement philosophies. There is no such thing as buying DevOps or hiring someone to handle your “DevOpsing”—from setting mutually targeted goals and creating a dynamic of collective responsibility, a DevOps engineer will impact far beyond leveraging Jenkins, especially if this is your company’s first time around the DevOps block.

So, after role-play and trusting your gut feeling on a person, can you genuinely call this an engineering hire? Some may still argue yes; however, this person is doing far more than just bringing technical knowledge. They have seen and helped companies digitally transform and can do the same for you. The best way to start is knowing what you are looking for far beyond a glossy skills matrix on a resume.

— Michael Race

July 5, 2018
Is AWS Cloud Per-Second Billing Boon or Bane for DevOps Engineers?

Amazon’s introduction of its AWS services in 2006 with a per-hour billing model for compute resources was a game-changer for the IT world. Since then, AWS has taken over the cloud computing market as a leader, while broadening the range of cloud services offered every year. However, the per-hour billing business model was one of the major issues for many customers—in particular, those who had applications using EC2 for only minutes at a time.

Recently, AWS announced support for per-second billing of its EC2 service, addressing the aforementioned concerns. Of course, this also created controversy among the per-hour supporters. While I won’t get into the pros and cons across the whole enterprise, one group in particular is affected from the new per-second model: DevOps engineers.

Here are the top five implications for DevOps teams as a result of per-second billing:

Autoscale without worrying about cost

Most engineers were vary of autoscaling at will, mostly because of EC2’s hourly charges even though they might need additional compute capacity for just 15 minutes. With per-second billing, DevOps engineers don’t have to worry—it’s as simple as scaling your EC2 instance as you need to, and when you need to.

No more custom logic for hourly batches

Many customers use EC2 for jobs and batch processing purposes. Due to the hourly pricing, engineers have written custom logic to benchmark and constantly tune the number of nodes in their batch clusters to ensure they are cost-optimized. Per-second billing frees up headaches for DevOps engineers that are using EC2 for batch processing and job clusters.

The need to speed up EC2 launches

With per-second billing, it’s important to take advantage of prebaked Amazon Machine Images (AMI) for launching your EC2 instances, instead of complete configuration setup on every instance launch. This will reduce the costs for you, by saving anywhere from five to 15 minutes. It’s time to create prebaked AMIs for faster instance launches, ultimately saving costs.

Dilemma of Lambda or EC2

Be prepared to compare whether to use Lambda or EC2 for new workloads and applications more often than you are used to. Your benchmarking between Lambda and EC2 has become a little more dicey. The serverless lambda just got competition from EC2 again.

Your big data just got easier

With EMR also supporting per-second billing, your job has become a lot easier. As a , you no longer need to constantly benchmark your EMR clusters for number of nodes and job processing times to be optimized for per-hour billing.

DevOps engineers have a lot to benefit from per-hour billing, but also much to consider when it comes to time-to-deployment and choosing a compute service. Now is the the time for DevOps engineers engineers to bring these concerns to their CIOs to make a smooth transition to the new AWS business model.

About the Author / Vijay Rayapati

Vijay Rayapati, CEO of cloud management platform company Botmetric, identifies himself as a technology entrepreneur, passionate about creating products, building high-performance teams, and solutions that solve real-world business issues. Vijay is a popular speaker in India and the US, and has co-authored a popular book, “Learning Big Data.” Connect with him on LinkedIn and Twitter.

October 4, 2017
In Product Development, Transparency is the New Agility

The rise of DevOps has shown that agility and speed are key drivers for turning a business into a high-growth company. But as product managers, that’s not all we can take away from the DevOps culture. Those same methodologies that enable DevOps to function, such as having cross-functional teams and embracing a culture that embraces transparency, can also help product teams adopt experimentation.

The product cycle should be an ongoing opportunity to try new ideas, gather data, make changes and celebrate failure. Teams with an experimentation mindset are better-suited to accept these goals, knowing that they lead to learning and succeeding quickly. Yet, experimentation and developing innovative products will succeed only when there’s a foundation of transparency—and not just about financial data and company successes, though that certainly helps. Everyone on the team needs the freedom, insight and tooling to make big bets, as well as the accountability to see them through.

This is where DevOps comes in. Leaders at the C-suite and VP level need to apply DevOps systems to product development to stay agile. Here are three ways to do so:

Make Development a Collaborative Process

DevOps brings IT and engineers together to meet customer and business needs. Product development teams should adopt a similar cross-functional team culture to remove silos. When product managers, sales managers, success managers and developers have regular and open conversations about industry trends, customer requests and big successes, they’ll have more insights in the customer experience and product road map that can help each team be more effective. For example, sales managers interact with customers daily and bring a broad perspective about customer problems, while product managers can gauge whether the ideas align with the business goals and developers can assess whether ideas are feasible to implement.

At Optimizely, I encourage my team to brainstorm several big bet ideas on a quarterly basis and discuss whether these ideas are new offerings we need to build or buy or whether we can build on into our current offerings. This collaborative process gives the product, sales and developer teams insight into what other teams are thinking about and encourages all of the teams to put their heads together to experiment with new ideas. Recurring brainstorm sessions that incorporate recent trends will ensure the team stays agile.

We also have a collaborative product feedback process that allows go-to-market teams to suggest features to the product team at any time and have a conversation around the business benefits of feature delivery. This is accomplished with pods of product areas comprised of product, design, engineering, product marketing, success and sales. The members of the pod meet weekly at standups and biweekly at product reviews to ensure alignment between what is being built and what the go-to-market teams are hearing from customers and prospects.

Shop Around Your Proof of Concept and Be Ready to Course-Correct

Product development isn’t a one-and-done process. It requires building minimum viable product, collecting feedback and revising. At Optimizely, we try new ideas and first get feedback from our internal teams and around five trusted customers to start. We implement their feedback and updates before rolling the feature or product out with a slightly larger subset of beta customers, around 20 or 30. With even more feedback, we continue to improve and build on the product, rolling it out to larger and larger customer groups under beta before making it generally available.

As feedback is collected, it’s important to bring our cross-functional teams together to analyze the comments and bring their different perspectives, whether it be customer-centric or developer-focused. We have to be willing to accept the feedback we get and make changes based on it. A recent example is when we rolled out Optimizely X Experimentation, which was built with reusability in mind to overcome some of the issues of Optimizely Classic. While this was a worthwhile goal and customers wanted reusability, we ended up overcorrecting and having to build new workflows after the fact. These ended up looking more like Classic and were more user-friendly but less reusable across experiments.

Hire Product Managers who Prioritize Trust and Transparency

People underestimate the importance of getting along with your colleagues and building trust with them. That doesn’t mean you can’t disagree or have different approaches. It simply means you’ve established trust and open communication—and you’re open to hearing and considering these differing opinions. These two elements are key for staying agile and quickly adapting to industry trends. You often heard the phrase “strong opinion, weakly held” at Optimizely, which encourages lively debate.

At Optimizely, we evaluate trust and transparency by having engineers and marketers interview prospective product managers to make sure the candidate can build trust with different teams and see pain points and priorities from various perspectives. During the interview process, it’s equally important that people acknowledge their mistakes and are able to learn from them. If a candidate can’t think of a mistake or explain what they learned from one, they likely aren’t taking enough risks or reflecting on their failures. Opening up about failures creates trust that leads to better teamwork and ultimately better business decisions.

Ultimately, we can all take a page from the DevOps playbook to make trust and transparency a priority on our teams. Those who do will see new business ideas flourish and team members work in a more cohesive way.

About the Author / William Gradin

William Gradin is the VP of product management at Optimizely and is responsible for building the Experimentation Platform to help businesses boldly test their assumptions and respond more quickly to ever-changing market dynamics and customer expectations. Prior to joining Optimizely in 2015, Will worked in Product Management at Salesforce, where he owned the Platform teams for the Salesforce Communities product. Connect with him on LinkedIn.

May 26, 2017