devopsy.kinsta.cloud

Tag: cross-functional teams

  • Visually Staffing Your Developer Teams

    Visually Staffing Your Developer Teams

    Effective collaboration within developer teams is key to creating and improving any product, so who you choose to put on the team—and how it’s structured—can make a huge difference. There are many considerations that go into these decisions, and the challenge is even greater thanks to the widespread movement to a hybrid work environment. To make the process a bit easier and more transparent and to set teams up for success from the outset of each project, it can be helpful to use a visual approach to staffing. 

    Considerations When Assembling Your Team

    When a general manager is assembling a team in professional sports, they must take into account personalities, abilities and leadership styles to make sure the team functions smoothly and can win games. There are similar considerations necessary when putting together a developer team.

    For example, a team must have the right balance of expertise. Having a team that is loaded with frontend experience but little to no backend experience will likely prove problematic. Also, a team full of interns or entry-level engineers with few or no senior engineers won’t have the leadership and expertise needed to succeed.

    Coding styles are also a major consideration when assembling teams. Some engineers may be “cowboy coders”—coders who aren’t afraid to take risks when developing a product. Other engineers, however, may suffer from “analysis paralysis,” spending too much time making sure the code is as perfect as possible. It’s important for teams to have a mix of these styles so the development process doesn’t lean too far in either direction and so the overall team has the right balance for the project at hand.

    Using Visual Collaboration to Assemble Teams 

    As a development team leader, I’ve found staffing our teams using a visual approach—and visually collaborating within those teams—is invaluable. It makes it much easier to ensure we have the right expertise and coding style on the right teams while managing the fast-paced changes that are a reality in today’s business environment. And with the staffing for each project visualized, leaders and team members can easily see who leads the projects and how many people are on each team within those.

    Managers or project leads should be able to see all of the relevant factors of staffing before making an informed decision on who should be on the team or if they need additional support. To begin visually staffing your teams, start with a diagram that—in addition to displaying which team each person is on—clearly shows each individual’s responsibility and flags if they’re new to the team. When decision-makers are able to see each of these factors at a glance, it helps them quickly determine if a team has a gap and needs additional support. 

    Caption: Here, team leaders can see team breakdowns, responsibilities and if someone is new to the team (Lucidchart)

    To make visual staffing even more effective, you may want to get more granular in detail when viewing team structures. This could include labeling team members with relevant attributes such as title, seniority, location, etc. Therefore, if a team needs a senior engineer or someone who’s worked on similar projects, it’s possible to quickly drag their name on your diagram and place them on another team. With this method, teams can see how a new assembly of talent can work together and how this adjustment affects other teams. 

    Caption: Team leaders can identify personnel, job title and the reporting structure (Lucidchart)

    We’re also able to diagram reporting structures. It’s important to know to whom each person reports and how many direct reports each leader has. This helps prevent burnout and ensures each manager has the bandwidth to lead their direct reports and be in touch with the needs of the team.

    Once teams are created, work sessions, such as program increment (PI) planning meetings, will also benefit from a visual collaboration solution such as a virtual whiteboard. Before the recent predominance of hybrid and remote work, many planning meetings included in-person discussions in front of a physical whiteboard. Also, voting on decisions included physical, color-coated sticky notes which each person would place on the whiteboard to indicate their vote. 

    Caption: Developer teams can participate in virtual PI planning meetings for more seamless collaboration. (Lucidspark)

    Using visual collaboration will help not only recreate this experience in a hybrid environment but also make it more effective and efficient. For example, working with a virtual whiteboard solution could improve a team’s experience with common project management tools, such as Jira. The right visual collaboration solution can allow teams to pull stories and display the latest project information for everyone to provide input directly within a collaborative space. The result is an ability for team leaders to create high-level plans with both real-time collaboration and asynchronous input from teams around the world—all while massively reducing the slow, manual steps that are so common with collaboration in front of a physical whiteboard. Those slow manual steps could include transferring stories from project management software onto physical cards, moving those cards around as discussions continue and then—once decisions are final—manually uploading new information back into Jira or another project management tool. With the amount of dependencies on developer teams today, removing these steps in favor of more efficient processes is invaluable. 

    Visually Staffing Your Teams 

    Staffing teams and enabling collaboration can be a smooth experience, even in a hybrid environment. I’ve found that visualizing how teams work together provides the clarity needed to make the best decisions. It also allows easy changes to team composition should it be necessary. Visually staffing teams allows team leaders to start developers off on the right foot and consistently work toward project goals from start to finish.

  • The Pros and Cons of Embedded SREs

    The Pros and Cons of Embedded SREs

    To embed or not to embed: That is the question.

    At least, that’s one of the questions that companies have to answer as they decide how to implement site reliability engineering. They can either embed SREs into existing teams, or they can build a new, separate team.

    Both approaches have their pros and cons. The right strategy for your company or team depends, of course, on your needs and priorities.

    What is an Embedded SRE?

    An embedded SRE works as part of a non-SRE team. They might join development or IT operations teams, for example—although they could also be part of quality assurance teams, security teams or any other unit within the organization that can benefit from their expertise.

    Embedded SREs are the opposite of creating a dedicated, general-purpose SRE team. In the latter approach, an organization hires talent that collaborates with non-SRE teams but who aren’t integrated directly into them.

    There are several other organizational models that fall somewhere between these two poles. But typically, the two main types of structures are embedded or dedicated teams.

    Advantages of Embedded SREs

    Embedding SREs into other teams offers several benefits.

    Arguably the most important advantage is that embedded SREs ensure the highest level of collaboration between site reliability engineers and other stakeholders. When they work directly alongside other engineers on a continuous basis, it’s a pretty safe bet that their perspective will inform every decision that the other engineers make. Separate teams will still generally collaborate with other teams, but not necessarily with the same level of dedication as embedded SREs.

    Embedded SREs are also beneficial for organizations where the site reliability engineering concept remains new, and where stakeholders still need to learn what SREs do and the value they can bring. It may be a bit challenging to get buy-in when you propose creating an entirely separate team; indeed, developers or IT engineers may even perceive such a team as a threat to their own jobs (even though in reality, of course, they complement other teams instead of competing with them). But when you add SREs to larger teams, it’s likely to be easier to get everyone on board with the concept and for other types of engineers to appreciate how SREs make their jobs easier.

    Finally, embedded SREs work well for organizations that are on the small side and don’t need an entire team. If you have just several dozen engineers on staff instead of several hundred, hiring a few SREs to embed into your existing teams probably makes more sense than investing in a brand-new team.

    The Challenges of Embedded SRE

    The embedded model has its drawbacks, too.

    One major risk is that the SRE’s influence within a larger team may not be strong enough to have a major impact on how that team operates. A single SRE working alongside a dozen developers or IT engineers, for example, may struggle to integrate site reliability engineering techniques and tools into the existing team’s workflow—especially if it’s a team that already has strong opinions about how it should operate.

    Along similar lines, a single SRE working within a team using the embedded model could end up being stretched too thin to do his or her job well. That’s especially true if other team members expect them to “own” functions that should really be a collective, team responsibility—like responding to incidents or using chaos engineering to test reliability. The latter processes may be led by SREs, but they shouldn’t be their responsibility alone, and the function can start to break down when other team members think of them as the person on whom they can simply “dump” all of their reliability-related tasks.

    Finally, the embedded model can be difficult to scale, which is one reason why it doesn’t work as well at large organizations. If you have dozens of different engineering teams, embedding an SRE into each one requires a lot of management effort. It also deprives the various SREs within your organization of the ability to work closely with each other. In that type of scenario, it probably makes more sense to create a dedicated team and have those SREs interface with other teams as needed, rather than trying to distribute site reliability engineers across the business by embedding them directly into other teams.

    Conclusion

    In general, then, embedded SREs make most sense for companies that have relatively small engineering organizations, and in which the concept of the function has not yet received complete buy-in. For large businesses that have numerous engineering teams, it will likely be easier to manage SREs by creating a separate team just for them.

    That said, your mileage may vary from the norm, and it may be worth experimenting a bit to determine which model—embedded or not—makes the most sense for your company. Keep in mind that you can also use a hybrid approach wherein you embed SREs into some teams, while simultaneously maintaining a dedicated team that leads site reliability initiatives for the organization as a whole.

    After all, site reliability engineering is all about thinking creatively and coming up with innovative solutions to challenges—even the challenge of figuring out how to structure the SREs within your organization.

  • Atlassian Survey Shows Lack of Collaboration in Software Development

    Atlassian Survey Shows Lack of Collaboration in Software Development

    A survey of 750 IT and engineering leaders found that while a full 83% of respondents agree that interdepartmental communication is critical for successful software development, only 41% of respondents view cross-collaboration to be a priority.

    The survey gathered responses from within organizations that generated more than $50 million in annual revenue and was conducted by Forrester Consulting on behalf of Atlassian.

    Overall, more than two-thirds of respondents (68%) said more departments than ever are involved with software development. Overall, 80% said they need to improve collaboration with other departments, but 82% don’t feel particularly confident about their ability to accomplish that goal. Three quarters (75%) also noted the need to improve collaboration within their own department.

    Well under half of the respondents (42%) said their culture, processes and tools enable the real-time teamwork and free-flowing information that help leaders make sound decisions.

    Among the subset of respondents that are highly collaborative, however, more than (53%) said they see greater customer satisfaction, with 59% reporting increased revenue because of that increased customer satisfaction.

    Suzie Prince, head of product for DevOps at Atlassian, said the challenge organizations face is finding a way to successfully employ a mix of synchronous versus asynchronous communication mediums to better enable collaboration. In fact, collaboration as part of any DevOps culture needs to be a C-level discussion to consistently accelerate the rate at which software can be developed and deployed, she added. The core issue is that not every management team is equally comfortable with open communications between the various departments that make up an organization, said Prince.

    It’s critical for C-level executives to set that tone and lead by example; DevOps best practices can’t only be adhered to by development teams. The entire business needs to embrace DevOps best practices as an operating model for the business to really attain the maximum benefit.

    Unfortunately, DevOps practices within development teams have been unevenly adopted, so it’s often very difficult to get the rest of the business to embrace DevOps principles. Organizations that do embrace DevOps, however, tend to be much further down the path toward becoming a digital business and can more adroitly take advantage of business opportunities as they arise or shift strategy as changing business conditions may warrant. In fact, organizations that embraced DevOps best practices arguably fared better during the economic decline that followed the onset of the global COVID-19 pandemic that took every business and IT leader by surprise.

    It’s not clear from the Atlassian survey results if a DevOps divide is starting to emerge that makes some organizations more competitive than their rivals. In theory, at least, if every organization is now a software company that happens to build something or deliver a service, then the faster they can reliably build and deliver applications the greater the competitive advantage they can attain and sustain.

    There are, of course, a whole host of factors that determine how competitive any organization might be. However, in the age of digital business transformation, it’s clear that being able to build and deploy software at a fairly regular cadence is the new table stakes required for simple survival.

  • Best Practices for Building Next-Gen Software Teams

    Best Practices for Building Next-Gen Software Teams

    Technology is moving faster than ever before. The pandemic has accelerated digital transformation across every organization in every industry. Creating new remote working environments or developing new digital revenue channels has taken place at lightning speed over the past year.

    At this time of rapid change, software development teams are continuing to help organizations adapt to our new environment and stay ahead of changing consumer habits. They also have to deal with their own personal situations and shifting organizational priorities – from new COVID-19 protocols to virtual offices.

    As 2021 progresses, what are the elements that can help business leaders stay ahead of the latest innovations and build resilient, forward-looking teams? Here are three tips I believe will help build next-generation software development teams.

    Put Your People First

    Finding the right people is never an easy task. Hiring for the skills you need to fill a role is important, but it’s also necessary to look beyond those requirements. Having the right technology skills is not always the be-all-end-all when it comes to hiring. Is this candidate a good fit from a cultural standpoint? Will they be engaged and motivated? Are they interested in learning and growing their own skill sets? Do they have different experiences and perspectives than your existing team members? Each team member brings something special to the table, and treating each person as an individual and as an essential part of your larger operation is crucial. Everyone should feel valued and connected to the company. This is especially true during the pandemic. Managing with a sense of empathy, understanding, and caring helps team members feel a sense of belonging and as though their job matters – because it does. When employees feel that their role is undervalued or unnecessary, they can easily become detached and uninterested. Hiring for the person and caring for your people during and after work hours builds better teams and keeps them connected to the business and mission.

    Allow Software Teams to Experiment

    Staying ahead of the latest trends and technologies is critical for delivering advanced software. Experimenting and allowing your teams to try new things can go a long way to drive innovation. It gives them opportunities to learn new applications and ways to deploy new technologies. Continuous training is a key part of the experimentation process. Our teams, for example, actively participate in more than 20 events per month. We also invite clients to join and participate, adding another layer of value for our clients. These events come in different forms and settings. They are virtual, gamified and often employee-led. The idea is to create space for open-minded discussions, team-building exercises and knowledge sharing. For example, our head of digital security leads regular sessions to educate our people on evolving security threats and effective security strategies to defend against them. Adapting to new ideas and building expertise in new technologies require a “no fear” mindset. Having the freedom to experiment and learn from each other provides the framework to keep teams ahead of the latest digital trends.

    Empower Your People

    Giving team members a certain degree of autonomy and responsibility for decision-making in their daily work improves innovation and productivity. When you empower employees to make their own decisions and decide their own path to success, you create a better workplace culture. At Cinq, for example, all one thousand of our developers know what projects we are engaged with across the company. Not only are they aware of what’s happening, they know the project goals, objectives and KPIs of every client engagement. Team members can request to be added to certain projects and teams at any given time. They may be interested in the client’s business, want experience with a new application or have an idea or suggestion to help us achieve our goals. We also give them the tools and resources needed to successfully manage or lead their own projects, which can lead to massive innovations inside our own business. Empowered employees are loyal, committed and more productive. When employees are given opportunities to work toward their goals and drive their own careers, the benefits are enormous.

    Staying ahead of new technologies and applications can be a challenge for even the most tech-savvy organizations. Recruiting motivated learners, empowering your employees to move across teams, and creating opportunities for experimentation are critical elements to building next-generation software development teams.

  • Transforming Your Digital Transformation

    Transforming Your Digital Transformation

    Change is difficult, so most companies pull the bandage slowly, hoping to avoid the hard, painful part – especially with regard to digital transformation. But in the corporate world, cautious change inevitably leads to entropy. Progress gets made in one department at a time, at the expense of others. Leaders turn over, priorities shift, plans fall apart and the desired results never happen. The bandage never really comes off.

    I see this happen a lot in companies that say they want to transform their digital customer experience – across all industries we work with. The board or the shareholders might not embrace a truly digital-first strategy or understand what the end-state transformation goal actually is. Instead, they end up digitizing the past, refreshening a longstanding brand with a new storefront in a doomed effort to maintain business-as-usual.

    These companies start with a well-meaning internal innovation team. They preach agility, then spend months researching the problem and consulting internal stakeholders, all the while trying to stay close to their comfort zone of existing practices. They write reports, they create roadmaps, run design-thinking workshops and come up with a solution … and then throw it over the wall to IT and wash their hands of it. Nine months later, the team is busy executing the new plan, but, with no internal champion, it can’t innovate. Due to the chaos and complexity of the project, the company burns through its budget and the project becomes a sad statistic. In fact, 95% of corporate digital transformations fail outright or produce diluted results, according to one recent study.

    Shrink the Transformation Timeline

    If a company really does want to digitize for the future instead of the past, it needs to turn that approach on its head. It needs to drastically shrink the timeline for digital transformation, which means starting with a digital operating model and putting the end clearly in sight.

    The result is a digital operating model that continuously delivers and measures value. It’s not a project. It’s not a product. It’s an operating model that puts teams close to the market and customer, and makes them responsible for finding and delivering innovative, high-value customer experiences.

    Get the Team Right

    This approach requires a team that knows how to embrace ambiguity and takes responsibility for creating the solution, not just designing it or building it – both of which fail when done in a silo.

    It needs to be a small, multifunctional team with the authority to implement things quickly, with the fewest possible hoops to jump through along the way. And when the inevitable organizational friction arises, the team and its goal needs to be seen as a transformation opportunity, not a barrier to be avoided.

    Companies assume they can do this work in-house, but it’s next to impossible to form the right internal teams because, typically, they don’t have enough experience with this model.

    These internal teams don’t quite have the right training for the job, but more importantly, they don’t have the right permission and agency. Some are risk-averse or can’t break through organizational silos. Some simply aren’t given the agency to do the work. Organizations with a lot of experience evolving, that give their employees this agency as a matter of course, might have success in-house. But it’s a tough ask for older, established firms.

    Don’t Design – Execute

    To their credit, many companies see this and outsource their transformation leadership to a consultancy firm. But you can’t outsource a transformation. You can’t teach a transformation. You need to implement transformation.

    Both internally and externally led transformations fail to grasp what really makes transformation work: agency and delivery.

    Instead of researching, planning and announcing, the team needs to roll up their sleeves and start executing. They have to drive the evolution of the company’s software toward the identified goals, break down any organizational silos or challenges that get in the way and deliver – quickly. They have to work together with the IT team, and anyone else who needs to understand what they’re doing, but execution needs to be top of mind at all times.

    The solution isn’t insourcing or outsourcing. It’s a joint venture, where the transformation is within the company’s core business unit, working together with a consultancy or partner to complete the digital transformation faster by focusing on a smaller slice of the business. With this foundation, an organization can be pulled through a rapid transformation in nine to 18 months, rather than three to five years.

    In fact, in under six months, a tiny transformation team of five to seven people can scale up to a transformed organization of 120 people, quickly delivering the desired digital-first customer experience. All it takes is a team that has done it successfully, multiple times. And the best thing about this is that transformation is measurable by the ability of teams to continuously deliver and evolve experiences across multiple digital channels.

    Digital moves too fast, and is too complicated, to be managed by traditional processes and approvals. If it does, whatever change your company sets out to enact is at risk of being dead on arrival. Agency plus delivery is the formula for a successful digital transformation.

  • DOES19 London: Effective Team Management Foments New Ways of Working

    DOES19 London: Effective Team Management Foments New Ways of Working

    In the IT space it feels like we collectively begin asking and ruminating on the same question all at once, chewing on all the possible answers and outcomes. Recently, one such question has begun popping up with more and more frequency: “How can we help teams, and especially product teams, be more effective?”

    While more research is coming out to help guide teams through new and iterative ways of working, moving on from static and traditional models is far from the norm. So, it may come as no surprise that insights into different ways of structuring product teams would be a prominent topic at Gene Kim’s DevOps Enterprise Summit 2019 (DOES19) in London.

    There, I sat down with Manuel Pais, an independent IT consultant focused on team interactions and the co-author of a new IT Revolution Publishing book, “Team Topologies.”

     

    In our interview, Manuel shares some takeaways from his book as well as guidance on the importance of managing team “cognitive load” and how organizations can begin rethinking their approach to software delivery when it comes to cross-functional teams.

    In addition to our interview on the Digital Anarchist set, I am also happy to include a video of Manuel’s presentation alongside co-author, Matthew Skelton.

    If you’re interested in hearing from other IT and DevOps leaders, be sure to watch the other 30 video interviews from the conference up on the Digital Anarchist site.

    Don’t forget! DevOps Enterprise Summit Las Vegas 2019 is happening Oct. 28-30 at The Cosmopolitan Hotel. Be sure to register here if you haven’t already and join us for the premier event for technology leaders of large, complex organizations implementing DevOps principles and practices. You can also preview the event by watching presentations from past DevOps Enterprise Summits.

    — Alan Shimel

  • Executive Buy-in May Be Preventing DevOps Success

    Executive Buy-in May Be Preventing DevOps Success

    A common misconception in the executive suite is that an organization can reap all the benefits of DevOps without fully embracing the change necessary. That’s understandable, given that a DevOps transformation is a lengthy, costly disruption for companies of all sizes. DevOps requires businesses to transform, from team organization to culture to development processes. This shift often sparks reallocation of power and influence, which sometimes can make high-level executives uncomfortable. While most companies have begun to recognize the overall business benefits of DevOps, few are fully committing to such a large undertaking. As a result, a hesitant executive suite holds an organization back from realizing the full potential of DevOps.

    The key to successful DevOps implementation has four facets. Executing against one element just isn’t enough—an organization must combine all of them to realize the benefits of DevOps:

    • Training: Once executives have educated themselves on the importance of DevOps and accepted the disruption that comes with it, they must make sure the individuals are properly trained. Everyone involved needs to be open to learning new skills and new ways of functioning. A big part of DevOps is breaking down team silos, so individual contributors need to be willing to work across functional boundaries.
    • Tools: DevOps adoption can be viewed as a cultural and methodological change, and therefore teams also need the right tools that help them work together to create scalable and secure applications efficiently. The executive suite should invest in platforms that support automation and collaboration.
    • Support and determination: To provide support and guidance throughout the shift to DevOps, organizations should consider hiring a DevOps expert or bringing in a third-party consultant. This person can help the executive suite map out a DevOps strategy and help train the teams involved. Lack of expertise is one of the reasons businesses often fail to complete a DevOps transition.
    • Cross functional and accountable teams: A DevOps team should have all of the resources necessary to implement and deliver functionality to the customer. They should also be accountable for operating and supporting it.

    Another important consideration is security. The CISO and security team are responsible for security in most organizations, but developers need to understand the critical role they play in creating secure applications. They need to be accountable for the security of the software they write. It’s up to the security team to train and nurture them until they are self-sufficient. The development organization as a whole must be willing to make security part of the annual goal-setting process and to report on progress transparently at each level of management.

    Every DevOps implementation begins with education, from management to the individual contributor. Educational resources increasingly are available as DevOps continues to gain momentum in the industry. If faced with a hesitant executive suite, teams should leverage proof points from other companies that have successfully implemented DevOps. Leadership should be ready to measure, experiment, iterate and, most importantly, stick with it. When armed with the right training, tools and support, organizations can gain a competitive edge with DevOps.

    — Pete Chestna

  • Understanding the Data Scientist’s Role in Cross-Functional Teams

    Understanding the Data Scientist’s Role in Cross-Functional Teams

    In our previous article, “Data Science Industry Perspectives in the Cloud,” we discussed that evolution is key if you plan to grow your business. You can start with ready-made solutions, then, in time, you can switch to in-house solutions created with the help of a group of data scientists.

    This time around, we’ll talk about the cases and solutions when machine learning as a service (MLasaS) doesn’t work. Your company’s first move is not to go out and hire a data scientist to solve your urgent business needs; rather, your company should invest in a custom-made solution. Only when you have a workable solution can you dive in deeper and create a proper team that can create an in-house data science solution.

    Custom Made End-to-End Solution as a Start

    Data Science in the classical sense involves data, goals and models to solve a pressing issue. The best way to jump start a data science process is by putting together various ready-made services into a single workable product to show your customers clear-cut results in short order. This can be done without any complex or global research, and you can comfortably formulate specifications taking into account all the feedback from your customers to create a more high-level data science offering later.

    One of the biggest issues for any data science project is in formulating the specifications for it. The usual request is, “Create something for my company using data science. Analyze data for me.” This type of a job has lots of trial and error. Having a custom end-to-end solution from the start lets you add on extra services as needed, and enables greater insights and predictions into a specific business workflow.

    I believe companies should build their data science system not from the data science research but from the point of view of an end-to-end solution. And then, bit by bit, they can adjust their services in the process.

    Moving on to Data Science Research

    Companies start data science research by employing a data scientist who matches their needs. You can use the standard data scientist qualifications: This employee should have a firm footing in the application environment, mathematics and programming. Also, their business skills are key to success—I believe an understanding of the topical area is key here because we are solving business issues. And a data scientist who solves problems that are more academic will be more focused on winning a Kaggle competition than on addressing the business needs.

    It is important this person understands the product development cycle. This way, the models and analyses data will be built in such a way that they could be deployed in production. For example, if a person is using R (the language and framework for data scientists), we should note that it is more research-oriented and it is not production-ready. Correspondingly, the results of such research cannot be deployed in any end-to-end solution. Therefore, the data scientist should work with a programmer.

    Although in the above diagram we see that the Data Scientist should have hacking skills, in reality, this is not the case. In their vast majority, Data Scientists are not able to write quality code. And if you need not just the research but a solution then process-wise you need to have a Data Scientist working together with a Data Engineer.

    CAP Theory as an Analogy

    It is impossible to have three database properties at once: consistency, availability and partition-tolerance. This basic rule is known to all developers. This is true for a data scientist as well: People envision a data scientist overlapping all three  properties, but in real life, you cannot find such an ideal candidate. Usually, people tend to lean one or another way in their work and keeping a balance is not always a priority.

    In principle, data scientist should have business insight, understand the math behind it and work in tandem with a skilled developer. Of course, a proper data scientist should be able to write any semblance of code, but a data scientist should work with a data engineer to write and realize code, being responsible for both the quality and sustainability of this solution.

    The classic tragedy of a company that decides to initiate any data science research is when a data scientist says, “I’ve got 40,000 lines of Python code on my PC. Can you make it work in production?” And, of course, this is virtually impossible to do. When that happens, all the research is simply wasted.

    Cross-Functional Teams

    Any team that deals with data science has to be cross-functional. In other words, it has to cover a whole stack of the solutions it writes. In a normal infrastructure there should be a DevOps engineer, data scientist, data engineer and a product developer writing the web app and/or mobile app. This single team is responsible for the result. The team members should work together and solve tasks that are interconnected in their interactions.

    All of this means that the whole team is responsible for the business result. This is also true for the transitionary research done by a data scientist that is impossible to use in production on its own.

    Old-School vs. Vertical Teams

    To dig in deeper, let’s take a classic old-school layered company organization structure when you have a department of data scientists, operations, UI developers, a big data department, QA engineers and so on. In this case, we have every project penetrating most of these teams. And the classic problem is that tickets and tasks are being thrown around by one team to another, and the real business goals are being watered down along the way and not solved in the end. So, instead of this horizontal division, we have divided the teams vertically. This allows us to create teams that see a clear-cut goal they need to achieve. And at the same time, they can improve their cross-skills and boost their responsibility levels.

    As a result, such teams began to deliver, Scrum and Agile properly. It is not directly related to data science, but companies need to realize the difference between a data scientist, which works in academia, and a production data scientist. Companies should aim at employing the latter one within their teams, and not let a data scientist work alone remotely.

    About the Author / Stepan Pushkarev

    Stepan Pushkarev is the Head of DevOps Practice at Squadex.com and CTO of Hydrosphere.io. Co-founded and managed engineering teams for eCommerce, IoT and Ad-tech companies. He has been responsible for the full products stack: math models, infrastructure & operations, enterprise applications as well as hiring, establishing engineering culture and delivery process. Stepan combines strong technology, management skills and entrepreneurial spirit. Connect with him on LinkedIn.