Tag: developers

  • Logging: Turning Good DevOps Teams into Great Ones

    Logging: Turning Good DevOps Teams into Great Ones

    The competition is on for DevOps teams that are expected to rev up production on a multitude of projects, with teams and companies aiming to stay atop the global software product marketplace. The complexity of code creation and the pace of deployment has picked up in recent years, and DevOps managers are starting to face serious challenges of scale and efficiency.

    Constant code generation is the first obstacle. Imagine a team producing millions of lines of code, working for weeks and hoping for a smooth release. But testing time arrives, and there is a total data meltdown.

    Wasting valuable programming hours is not an option, but what if managers had the tool to turn back the time to a version before the mistake happened? Managers or developers may view logging as an incredibly boring activity, but it is vital. It serves like a black box of what went wrong—but also a way to roll back the bad code, and return to a more manageable situation.

    Logging and tracking habits ensure that a DevOps team will not see their code fail in production, and cause expensive breakdowns in service. Logging becomes more than a wishful addition, but a pivotal tool that may save a project.

    Data is proliferating, code is constantly being written to offer innovative products or support existing ones. There is a massive amount of data projected to be generated in 2025, only a few short years away, if the global IT production sector keeps increasing its activity. This amount of data to be kept and logged would be roughly 10 times higher compared to the data generated in 2016. This issue has plagued Silicon Valley, but also worldwide tech centers: how to keep track of what is going on. How do companies such as Instacart, Lime and OpenAI track their changes and ensure viability and awareness of errors?

    There is the new role for DevOps managers: ensure fast, modern and stringent log management. Dedication to logging means generating tight order out of the chaos of code creation. The right tool will be economical, intuitive and fast, ensuring the possibility for control in what could become a runaway data meltdown.

    Logging is one of the pillars in the modern DevOps department, and while now it is important, it will become indispensable, along with metrics and analytics.

    DevOps are already aware of the possibilities of logging, to keep track of multiple avenues in product creation, from databases to server side events, and even the effects and behavior of users.

    Stringent log management means making use of a valuable resource of knowledge and decision making, but also operating with a complex system of data. DevOps managers will need the tools to span the demands of stringent log management, for a multi-step process involving log generation, transmission, analytics, storage, archives and disposal.

    The right logging system can streamline those steps. Inexpensive logging tools and systems can unify log management across teams and simplify the process of logging. More services are appearing each day, either cloud-based or self-hosted, and with varying levels of security. With increasing complexity and the drive to create ahead of the competition, it would be the task of DevOps managers to make the team aware of the tools, and to provide tools intuitive and fast enough for the good habit of logging and log management.

    — Chris Nguyen

  • Should Your Company Hire Remote Full-Stack Developers?

    Should Your Company Hire Remote Full-Stack Developers?

     Is your company seeking to hire a qualified full-stack developer? It seems like an easy enough task—create a job listing, read resumes, conduct technical interviews and make the hire. Finding engineers who understand every layer of the stack is difficult, however. Companies compete for the best and brightest full-stack developers and, unfortunately, they are in shockingly low supply.

    What’s the solution? One way to circumvent the full-stack developer talent shortage is to leverage telework. By seeking to hire remote full-stack developers, you can gain to a global pool of talent.

    There are many other benefits to hiring remote full-stack developers. However, hiring remote developers isn’t always the right fit. Read on to see if your company is ready to hire remote full-stack developers.

    Is Your Company Looking to Save Money on Office Space?

    Hiring remote full-stack developers is a major decision. When companies think about utilizing remote talent, they usually think of the headache of adopting a new form of work and of managing employees who work out of the office. However, it is also important to note one of the major benefits of leveraging remote developers: the cost savings.

    Other benefits to consider: boosted productivity, higher workplace satisfaction and lower incidences of interpersonal issues between coworkers. If you believe your company might be able to benefit from hiring and managing full-stack developers remotely, then you may want to consider beginning your hiring journey seriously.

    Does Your Company Prioritize Quality or Community?

    Finding the top full-stack developers can be extremely difficult. By opening up your search and tapping into global talent by seeking developers looking for remote work opportunities, you can put a greater emphasis on quality and technical excellence.

    If your company is more focused on creating a cohesive community or company culture, you may want to focus on hiring full-stack developers that can come into the office, interact with coworkers and build a rapport with managers, clients and fellow developers.

    However, this isn’t to say that it’s impossible to foster caring and authentic professional relationships with remote developers. It may simply prove more challenging.

    Are You a Brick-and-Mortar or a Click-and-Mortar Company?

    If you run a traditional brick-and-mortar organization that’s been around for a while, it may be more difficult to adjust to change. If you run an online business or if you’re a burgeoning small business or startup that is heavily reliant on tech, then it may be significantly easier for you to adopt telework.

    If you do run a traditional business with a traditional office and traditional approach to project management, it may take a bit of adjusting to accommodate for remote workers.

    You may want to try out remote work in a trial capacity by bringing on remote freelancers or allowing current full-time developers to work from home for a few days out of the week. If you find your company easily adjust to the changes, then you may want to consider hiring fully remote full-stack developers as well.

    Do You Want to Access Global or Local Full-Stack Development Talent?

    As mentioned earlier, full-stack developers are in incredibly high demand. Unless your business is headquartered in a major city, it’s going to be next to impossible to find the top full-stack developers. If your company is just starting or simply short on cash, then it can be difficult to bring on such an expensive full-time employee as well.

    If you want access to global talent, you’ll need to open up to remote work. Through telework, even companies in small cities and towns can gain access to top talent. If you’re more interested in fostering local talent, however, you might not be interested in leveraging telework.

    Conclusion

    Hiring remote full-stack developers isn’t always a simple process. It requires your company to remain open, adaptive and innovative. Without these traits, managing remote full-stack developers will prove challenging to say the least.

    How can you ascertain if your company is ready to bring on remote talent? First, it’s important to assess how much cost savings, boosted productivity and higher workplace satisfaction rank for you. In addition, it’s important to gauge whether  community outweighs the desire for the production of quality code.

    Finally, you’ll want to access your technological readiness and ability to adapt. Are you willing to search for global talent and leverage technologies, platforms and software programs that will assist you in this endeavor? If so, you may want to consider hiring remote full-stack developers.

    — Tania Brooks

  • Fintech: A Profitable and Growing Developer Career Path

    Fintech: A Profitable and Growing Developer Career Path

    What do you think when you hear the word “fintech”? It’s been around for almost five decades, yet it has only just been officially recognized as part of the English language. In the Merriam Webster dictionary, the word is defined as: “Products and companies that employ newly developed digital and online technologies in the banking and financial services industries.”

    A number of factors in recent years have paved the way for the boost in profile for fintech, bringing about phenomenal growth in the sector. In some parts of the world, governments and regulatory bodies have fueled this growth by issuing regulations that open up the banking sector to increased competition, creating a great many opportunities for finance and technology professionals alike. And from the other side, not only are banks being at most compelled, and at least pressured, into opening up to competition, many are now actively seeking to collaborate with more agile third parties—namely fintechs and developers —to innovate the services demanded by their customers.

    So, what exactly are the opportunities available for developers in this newly defined sector?

    From the Classroom to the Boardroom and Beyond

    I am confident that there has never been a better time than the present for developers to get involved in the financial services industry. It’s a bold statement, but one that bears out when the remarkable amount of support being directed at fintech is considered.

    Today, the industry is receiving unprecedented political backing as multiple jurisdictions vie for fintech hub status. From the United States to Hong Kong, from Germany to the UK, governments are offering financial support via tax incentives and grants in a bid to woo fintechs and developers to their financial centers. And it’s no wonder, given how lucrative a business sector it has proved to be. In the first half of 2018, VC-backed fintech companies reportedly raised more than $25 billion in funds. CB Insights’ data shows there are currently some 29 fintech unicorns (venture capital-backed, private businesses worth more than $1 billion) worldwide, with an aggregate value of $84.4 billion—and this is expected to keep on rising.

    Even educational support is being offered to future-proof against skills shortages, with academia and financial firms collaborating to create fintech-related graduate training schemes and degree courses.

    Open Banking

    Around the world, particularly in Europe and parts of Asia, governments and industry bodies are lighting the way for a new approach to collaboration with fintechs and developers. In Europe, the second payments Services Directive (PSD2) and the UK’s Open Banking initiative are opening up the financial services market to third parties eager to enter the space. These initiatives, and others like them, compel banks to share their precious data with third parties at the request of their customers—meaning they can no longer take their hitherto market hegemony for granted.

    Here in the United States, the Consumer Financial Protection Bureau (CFPB), the body that regulates the offering and provision of consumer financial products and services, has released a set of data sharing principles that provide an advisory framework for financial institutions. Though the CFPB’s guidelines are non-binding, they do show that there is growing pressure from regulators for the financial sector to be more transparent with customer data.

    Collaboration is Key

    When it comes to collaboration between fintechs and banks, one might expect there to be a certain amount of resistance from both sides. Fintechs are traditionally disruptors, while financial institutions don’t take kindly to being disrupted. A pPlatform-as-a-service (PaaS) approach can help enable collaboration that is beneficial for both sides.

    The ability to collaborate through a common platform, hosted in the cloud, offers the potential for fintechs to significantly reduce application development time, often from months to weeks. Users can bring innovation to market at pace using a low-code, cloud environment and sell applications to the banks without needing to build a global sales team or worry about a scalable infrastructure. Pre-integrated developer tools, a sandbox environment and an API catalogue also speeds application creation and levels the innovation landscape. With platforms, the fintechs with the deepest pockets no longer dominate the market simply because they have the resources to snap up opportunities. In the platform sandbox, smaller firms, startups and independent developers can bring their own offerings to the table.

    As the model for bank innovation changes, a platform approach also makes it easier for financial institutions to transform their businesses, extend product capabilities and access innovation. Banks can leverage their own development capabilities to create and deploy applications on top of their core systems and can collaborate with fintechs to build and access new capabilities they need quickly through open APIs.

    A key part of the PaaS approach also gives developers the ability to make apps available to banks globally through a digital shop front, much like Apple’s iOS App Store. So, not only are developers and fintechs able to distribute and monetize their apps, banks can readily test and deploy them.

    Get Onboard

    Financial institutions are starting to recognize that an open platform approach is the fastest way to onboard fintechs to enhance their current business offerings. As this understanding deepens, so too will the demand for skilled professionals and new applications. Banks in the not-too-distant future will not only concern themselves with the services they want to deliver, much consideration will be given to the fintechs most able to provide them in a timely manner.

    It’s not just in retail banking that the opportunities lie, but across all areas of financial services—from investment management to corporate banking, payments, trade finance and capital markets. The coalescence of these two considerations is leading to a change in the industry zeitgeist, meaning the scope for innovation across all areas has widened significantly. The great earning potential for those with the ability to convert good ideas into innovative apps and services has never been more apparent.

    So, developers, as banks wait with open arms (and APIs), now is the time to enter the burgeoning world of fintech.

    — Mitesh Soni

  • Understanding Pillar 1 of AWS’ Well-Architected Framework

    Understanding Pillar 1 of AWS’ Well-Architected Framework

    AWS’ Well-Architected Framework provides a wealth of incredibly valuable best practices and guidelines structured around five distinct pillars that more cloud-native organizations would do well to heed. The first of these pillars, Operational Excellence, recommends a number of particularly valuable techniques. However, successfully adopting the design principles and practices within Pillar No. 1 requires a strong foundation: a company culture that is willing to continuously work to improve.

    In practice, this can mean holding high-profile “Game Days” focused on advancing internal procedures, nurturing an environment where engineers and product leaders really collaborate to promote business value while pursuing rapid iterative release cycles, championing operational automation and related approaches. Fortunately, adopting the tenants of AWS’ Operational Excellence Pillar can naturally assist in implementing cultural changes where they are needed.

    Design Principles for Operational Excellence

    To achieve operational excellence within AWS, incorporate these six design principles:

    Perform Operations as Code

    The cloud makes it possible to apply the same rigorous practices used to reduce risks in software development (such as version control and automated testing) across your full application environment. Cloud infrastructure allows all aspects of operations to be defined, managed and implemented in software code, spurring a revolution in their automation and consistency.

    Annotated Documentation

    Without effective automation, maintaining documentation is tedious to keep in sync with reality—ultimately making it challenging to troubleshoot issues when they inevitably arise.

    Fortunately, the cloud makes it possible to create documentation automatically as an artifact of the build process. This documentation is detailed and annotated, always up-to-date and can be read by us humans (as well as systems). With automated documentation, engineers can react to issues more rapidly and accurately, and enlist automated systems to do the same.

    Make Frequent, Small and Reversible Changes

    While agile, iterative development methods have become commonplace in many areas, product-driven organizations tend to stick to large releases. However, operational excellence calls for a systems design that incorporates tightly focused, failure-resistant components in an environment designed for constant change and improvement. Implementing this flexibility allows for more rapid deployment of new iterations—but easy rollback if issues pop up.

    Refine Operations Procedures Frequently

    Operational excellence requires holding regular meetings to reflect, suggest and implement improvements to operations procedures, the same as those held when practicing any agile methodology. Game Days, where procedures are tested and iterated on, will help build and strengthen excellence as well.

    Anticipate Failure

    Rather than waiting for failures to strike and then studying the causes in a post-mortem, practicing routine “pre-mortem” exercises that recognize failure points before any harm is done. Game Days can then be used to investigate these scenarios and ultimately bolster resilience. For example, premier cloud-native organization Netflix demonstrates this design principle with its Chaos Monkey project, which is purpose-built to strengthen application resiliency against random instance failures.

    Learn From All Operational Failures

    Operationally excellent cultures have the openness to share post-mortem findings across the organization without pointing fingers. This broadly shared knowledge can then inform improvements in all areas of the business, not just in engineering and product departments but marketing, finance, and others as well.

    Critical Components of Operational Excellence

    Embracing the six design principles above paves the way for you to practice operational excellence in three key areas defined by the Well-Architected Framework: Preparation, Operation and Evolution.

    Preparation

    Operational excellence requires operations teams to possess a clear and complete understanding of the goals for each workload within the complete system, and how those goals will be achieved.

    Operational Priorities

    This clarity must include knowledge of the workloads the operations team is responsible for, the shared business goals and their role in achieving them and any relevant regulatory or compliance requirements. This information allows teams to correctly prioritize those workloads with greater importance or complexity.

    AWS offers these helpful resources for setting operational priorities:

    Businesses can also enlist AWS Certified Partners—experts selected by AWS to provide consulting, and professional and managed services as an extension of a company’s own operations team—to make sure priorities are set wisely.

    Design for Operations

    A well-architected workload is designed for operations from the beginning, with consideration for deployment, updates and similar needs. AWS provides the ability to model entire workloads as code, from applications to infrastructure, policy, governance and operations. It also offers a robust set of tools and services for operation design, such as CloudFormation and the AWS Developer Tools. Businesses can also create systems featuring the observability and insights needed to operate workloads by using AWS Cloudtrail, AWS VPC Flow Logs, AWS CloudWatch and more.

    Operational Readiness

    Beyond the technology used, operational excellence is also a function of elegant, consistent processes and procedures for deploying and operating workloads. This means maintaining accurate documentation, building a well-trained and resourced team of experts and having governance that makes sure everything is done the right way.

    It’s also smart to utilize automation when evaluating environments. CloudWatch events can be automatically addressed by scripting procedures created with AWS Systems Manager’s Run Command and Lambda. Configurations can be automated and benchmarked against best practices using AWS Config rules.

    Operation

    The first step to operational success is clearly defining it with key metrics based on shared business goals, so that events can be addressed with those goals in mind and success can be recognized when (or when not) achieved.

    Understanding Operational Health

    Because no two businesses are the same, the definition of operational health and success will vary. For example, one company might have workloads that demand low-latency and high-throughput performance to be prioritized over cost. Another might require a focus on cost-effectiveness and high availability. Operations teams must first understand those priorities and how they serve business goals to deliver what is actually needed.

    AWS offers tools and services for analyzing workloads to serve specific goals. CloudWatch Logs and Dashboards can provide system and business level views of key metrics. Amazon ElasticSearch and Kibana can offer further visualization of operational health metrics, as can outside tools and services like Logstash and Grafana.

    Responding to Events

    When events occur, operational excellence means having prepared and even anticipated failures and putting metrics, processes and procedures into action for a quick and effective response. At the same time, teams must have metrics to understand the business impact of workload components to resolve the highest priority issues first. AWS enables teams to script automatic event responses, such as automated rollbacks triggered by failures. Amazon CloudWatch offers a powerful service for these automated responses.

    Evolution

    Curiosity and a desire to learn and improve are essential traits for operations teams pursuing operational excellence. Every experience should be viewed as an opportunity to learn and to share that new knowledge base. The best teams enjoy learning from failure and by asking for the perspective of other business units.

    AWS offers an incredible platform for the analysis and experimentation needed to evolve practices. Amazon CloudWatch and CloudTrail can be combined with Amazon ElasticSearch with Kibana. Data exported to Amazon S3 can enable analysis with Amazon Athena and Amazon QuickSight. AWS also helps share best practices with CloudFormation templates, Chef Cookbooks, Ansible Playbooks, Lambda functions and more. 

    — Jonathan LaCour

  • CollabNet VersionOne State of Agile Survey: Join the Fray

    CollabNet VersionOne State of Agile Survey: Join the Fray

    If you follow the news on Agile software development, topics such as adoption, scaling, Kanban, Scrum or certifications, it’s likely you saw statistics in your reading that were taken from the “State of Agile Report.”

    For the last 12 years, CollabNet VersionOne has been surveying the Agile software development community and providing this valuable body of research to the market. The data is widely cited in publications worldwide and used by organizations to gauge trends in the industry and help business leaders in their decision-making. Many vendor organizations also reference the report to gain insights on how to better serve customers.

    Last year, the 12th “State of Agile Report” was compiled with insights from nearly 1,500 respondents. That’s a lot! The more IT professionals who take this survey, the more useful the results will be. By gathering more survey responses, CollabNet VersionOne can provide a more accurate analysis to help guide the industry.

    The best way for us to get a big-picture view of how organizations are adopting and scaling the practice is to gain an understanding of individuals’ experiences.

    Results from Last Year

    Last year’s report told us that organizations are realizing the benefits they set out to achieve by adopting Agile. Here were some other the key insights from the report: (You can download a full copy of the report here: https://explore.versionone.com/state-of-agile).

    • Agile is expanding within the enterprise – Adoption is growing within organizations, both more broadly and deeply. A higher percentage of respondents this year report that “all or almost all” of their teams are Agile, and that the principles and practices are being adopted at higher levels in the organization.
    • There is room for improvement in maturity – Only a small percentage of respondents indicated that their organizations have a high level of competency with Agile practices across the organization. The majority acknowledge that their organizations are still maturing.
    • Organizations are scaling Agile – The Scaled Agile Framework (SAFe) is reported as the most widely used approach for scaling. Internal coaches, consistent practices and processes across teams and the implementation of a common tool across teams are the top three factors reported to have been most helpful in scaling Agile.
    • DevOps initiatives are on the rise – The survey reveals the importance of accelerating the speed of delivery of high-quality software and the emphasis on customer satisfaction. It’s no surprise, then, that a large number of the survey respondents report that a DevOps initiative is underway or planned for the next 12 months.

    Who Should Take the Survey?

    The “State of Agile” survey is for anyone with stakes in the software development life cycle. It is not just for enthusiasts or members of teams using the practice; even if your organization is not using Agile, we still want to hear from you.

    Here are some examples of job titles that fit within the scope of this survey:

    • Business analysts
    • C-level executives
    • Development leaders
    • Software developers
    • DevOps managers
    • Consultants/trainers
    • Product managers and owners
    • Project managers
    • Scrum Masters
    • Agile coaches

    If any of these roles describe you, we welcome you to take the 13th annual “State of Agile” survey today. It will take you around five to eight minutes to complete. You can begin by clicking here. Responses will be accepted until midnight, Dec. 31.

    Thank you for your valuable input.

    — Thomas Hooker

  • Developers: Key to the User Experience in a Multicloud World

    Developers: Key to the User Experience in a Multicloud World

    It’s amazing how many applications behave as if the network is always available, bandwidth is infinite and latency is zero. While optimistic, this approach to applications is ultimately rooted in fantasy. As it stands, there is a major proficiency gap between application developers and the networks in which they operate—but this can’t continue as our world becomes increasingly multicloud. It’s time for developers to understand the inner workings of the network to close the chasm between applications and the network.

    Ships in the Night to Star-crossed Lovers

    For the most part, applications and the infrastructure on which they operate have little to no interaction. If you’re not sure about this assertion, try asking any network operator what applications are running on the network. It’s possible they can tell you about key mission-critical applications, largely because at some point the person has had to troubleshoot an application issue. But beyond the top few applications, it’s likely that they are considered largely nameless bits and bytes that fill up wires.

    Similarly, if you ask an application developer to explain anything about the network, you are likely to get blank stares. Yes, developers are aware that the network provides connectivity to critical applications resources, but it is atypical for them to know much beyond that. The acronyms that make up a network engineer’s livelihood are largely just jumbled letters—a bit of a nod to the dark arts required to actually make networks work.

    This missed opportunity has the potential to be the perfect match. There have been various efforts over the years to bridge the gap between these worlds with movements such as application-aware networking and the somewhat lesser-known network-aware applications. The central thesis of these approaches is: The more these two co-dependent but largely separate organisms know about each other, the more they can adapt and adjust to deliver a superior end user experience. To achieve peak usability, each side must know something about how the other operates.

    App Developers, Meet the Network

    While technology is making networking more interesting than it has been in years, excitement about the network is largely contained to those directly involved in designing, deploying and maintaining it. This is no fault of anyone, though, because the main appeal of the network is that it remains essentially invisible to the user. The network is a success when you don’t know it’s there.

    Consider the network as a power grid. When you turn on a light switch, you’re not concerned with how power is generated and distributed—you only care that it can light up the room, and you are concerned when it won’t. The physical infrastructure that makes this all possible is largely hidden to users and visible only when something goes wrong. Just as physical infrastructure is invisible, the network should be, too.

    That said, it is important for application developers to “see” the network and not let it fade into the background that same way that users do. Awareness of the network and its functions is critical when developing applications that will run on this system, as simply invoking an application should also include the requisite infrastructure.

    Broader Network Shifts are Required

    Of course, no change happens in a vacuum. Application development must be considered alongside the broader industry move toward distributed workloads and the proliferation of multicloud, as it is increasingly likely that applications will be hosted in different environments.

    Application developers need to not just understand the network, but also the different contexts in which that network exists. For the infrastructure to be invisible, this means those different environments—on-premises, in public cloud A or in public cloud B—must deliver the same experience, inclusive of both policy and security. This creates a bit of a challenge if the future is to be a tight mesh of interwoven applications and infrastructure. For a group that today assumes 100 percent availability and infinite bandwidth, this would be a pretty massive shift in thinking, and a correspondingly massive additional burden to bear.

    Abstraction is the Answer

    In the same way that containers serve as an abstraction above the compute environment, there must be a level of abstraction that sits above the cloud environment. This is really what underpins the multicloud movement.

    Application developers must ensure that application requirements are agnostic to the underlying infrastructure to ensure the system remains invisible to the user. Likewise, multicloud management must create an abstracted platform capable of translating application intent into device behavior regardless of where that device resides, whether it is physical or virtual, or even whether it is from vendor X or vendor Y.

    Taking on the Future of Multicloud

    Much too often trends in IT favor silos. As we make the shift toward multicloud, networking will be brought into focus like never before, requiring a different way of thinking about cross-domain management. What is required from application developers and network operators alike is communication and collaboration.

    Having a baseline understanding of how the distributed network functions will allow application developers to create products that will ultimately run better. Developers should also be transparent about what they are expecting from the network and what they need. This is the only way to get us closer to where we want to be: operating on a network that is omnipresent and wholly capable of anything we can throw at it.

    — Mike Bushong

  • DevOps and Collaboration: Fraternizing with the Enemy?

    DevOps and Collaboration: Fraternizing with the Enemy?

    DevOps is all about collaboration, we hear.
    “You must collaborate,” say managers initiating DevOps adoption programs.
    HR make new posters declaring the corporate values “We value Collaboration & Teamwork.”
    Can’t fail!
    Then when it appears not to be as effective as we had hoped we tell teams they are “Empowered” to collaborate.
    Blank stares!
    When things still don’t appear to be working we tell teams they are “self-steering,” “self-organizing, collaborative teams”….
    Still, we don’t get what we were hoping for.
    Luckily the tool vendors come to the rescue and tell us why it is failing. We need “collaboration tools and software” – Throw some “Slack” at it.
    Can’t fail!

    This is when we usually get frustrated managers knocking on our door asking about the Phoenix Project DevOps simulation. They are moaning about communication and collaboration! When we talk to the teams they are frustrated and moaning about “empowerment” and managers not “walking the talk.”

    In my mind these lack of collaboration skills is not really surprising. Why?

    Impression: Well, it doesn’t help when the word has also historically been connected to global conflict situations. “He faces charges of collaboration,” “fraternizing with the enemy.” Dev & Ops. Traditional adversaries?

    Interpretation: The interpretation of the word itself is also part of the problem. I googled an English Language Help Desk:

    Collaboration: “To work together with somebody in order to achieve a single shared goal.”
    Cooperation: “Work with other people by achieving one”s own goals as part of a common goal.”

    In an HBR article titled, “There’s a difference between cooperation and collaboration,” it was argued  “… Managers mistake cooperativeness for being collaborative,” adding, “… most managers are cooperative, friendly, willing to share information – but lack the ability and flexibility to align their goals and resources with others in real time.” If managers don’t get it then what chance do the teams have when told – “You must collaborate”!

    Induction: Part of our failure to be able to effectively collaborate could come from our schooling. We are chastised at school for asking others for help, for looking to see how somebody else is doing it—”That is cheating”—things are changing with more “collaborative assignments.” Yet, my children (now in their early 20s) were not taught what effective collaboration is, other than discovering for themselves. As my daughter explained about her school assignment, “You have to do it yourself if you want it done because others will slink off and not do it, then my individual grade may suffer.” They are told they need to collaborate but individual performance grades are scored. My daughter once came home stressed and almost in tears, declaring “Oh no! I have been given another collaborative assignment for my most important subject!” Children leave schools distrustful of collaborating.

    Indoctrination and Imperative: There is also the historical perspective. When we, the current workforce entered our chosen profession we had decades of functional silos that created a “them” and “us” attitude. This stimulates a sort of “territorial imperative” or “tribalism,” not helped by our own “technology” languages and tribal cultures. We have learned over time that other teams “dump things on us” or other teams “stop us from getting stuff done.” In other words, Other teams frustrate work.  (Territorial imperative—”the drive in animals and man to take, hold and defend a particular area, zone, turf,” —e.g. defending their best practice framework! And attacking others that come near!).

    Inconsistency: We then built walls between these teams and if they wanted something from each other they needed to escalate it to their manager. Managers would collaborate or not, especially when they were given conflicting, inconsistent goals and KPIs. Development teams are given goals to release as fast and as often as possible. Operations are given goals to make sure everything is as reliable as possible and available at all times.

    I had an example of this “them and us attitude,” once with a board of directors. I split them into two teams and gave each team a pack of ABC cards (Attitude, Behavior and Culture) and within 10 seconds heard, “Oh good, our team will win this!” automatically creating an atmosphere of competition and winning and losing, I hadn’t even explained the task yet! Teams compete.

    ‘It’ factor: A study by Google (2012) called Project Aristotle, named in tribute to Aristotle’s quote, “The whole is greater than the sum of its parts,” looked into why some teams work and others don’t. One of the key conclusions was that “psychological safety” was the “it” factor, “… a sense of confidence that the team will not embarrass, reject or punish someone for speaking up.” Because we have not learned to collaborate and because we have a “them and us” history often accompanied by blame, there is fear, uncertainty and doubt (FUD) about speaking up, sharing ideas or giving feedback.

    In the dark: However, there is another significant reason for poor collaboration, as I have discovered from running hundreds of Phoenix Project simulations with teams globally. It is the fact that nobody has defined the “behaviors” that demonstrate effective collaboration. We assume everybody knows what we mean when we say, “You must collaborate.” Fact: We don’t!

    Below are the results of a series of recent Phoenix Project simulations. One of the goals was to learn DevOps practices such as CALMS and the Three Ways, while another goal was to learn effective team-working and collaboration.

    I asked the teams at the start of the sessions, “What is effective collaboration? What behaviors will I see today that demonstrate this”?  The teams came up with a list of behaviors (each team came up with a top five or six behaviors; this is the consolidated list from seven teams).

    • Transparent communication – “open and honest” and “frequent.”
    • Calm and constructive dialogue (no blame, accusations and shouting).
    • Feedback “asking for and giving feedback.” Constructive feedback.
    • Shared team goals (and cross team). What are we aiming for? Impact on others and their goals. Maybe conflicts that need managing.
    • Listen – is it clear and understood? Show that you listen.
    • Being open for dialogue, being approachable –not closing the door or hiding behind.
    • Giving and taking responsibility.
    • Sticking to agreements.
    • Central moment to get together (e-2-e).
    • Knowing the roles and responsibilities, sticking to responsibilities.
    • Mistakes are OK.
    • Be proactive to get information needed.
    • Equal treatment: “Respect” ideas from ALL
    • Don’t HOPE, confirm.
    • “Willingness.” Confirm and agree.
    • Ownership of responsibilities.
    • Clear agenda and purpose of meetings.
    • Pleasure – celebrating success.

    “Are these behaviors you want to see more of in your daily work?” I asked.

    Yes! Was the answer.

    “Do we now all commit to sticking to these behaviors today, in this simulation?

    Yes! Was the answer. Easy! See there’s nothing to this collaboration stuff.

    However …

    During the simulation it soon became evident that roles and responsibilities were not clear or not being carried out; it was not agreed which information was needed by the business nor when; goals were unclear, meaning that priorities were arbitrary; people did not understand procedures and did not ask for help; assumptions were made (“I told you …,” “I thought you meant …”); people taking responsibility were not respected (“Let’s stop and try and sort this out …”)— everybody just carried on running around! “Headless chickens,” said one delegate; people were not listening to each other; there was a lot of “assumicide”; mistakes were made and people blamed—there was no exploration as to why and what can be done to stop these  mistakes. Mistakes were often corrected by somebody else but without giving any feedback to correct future mistakes; agreements were unclear across the team; there was no end-to-end flow—work and information was ping-ponging back and forward.

    “All in all, just like reality!” declared the teams.

    See how easy collaboration is. Within one hour of making a list of desirable behaviors, they were totally, utterly and universally ignored. Nobody appeared to own these behaviors or to confront each other on them.

    After the Experiences in Simulation

    At the end of the simulation round we looked at the team scores (number of releases, number of successful releases, business value realized). It wasn’t a pretty sight. The CEO wasn’t happy. The team felt stressed, unhappy, frustrated and powerless.

    We explored what this had to do with “effective collaboration.” Had anybody used the list of behaviors? No.
    Had anybody consciously practiced these behaviors?
    No.
    Did anybody give feedback on these behaviors when they were clearly not done?
    No.
    “Giving feedback feels strange, it feels like we are criticizing and blaming,” said one delegate. Another assumption!

    “It’s hard,” said one delegate. “The manager needs to manage this,” said another.
    “It was YOUR list and YOU all agreed this, why is this a manager’s job to get you to do what you said you wanted to do?”
    There was silence. “It’s not easy, we fall back into old ways of doing things, we need help!”

    Is it a Bird, Is it a Plane, Is it a Bus? No, it’s a coach!

    As the simulation progressed, the team refined their list of desirable behaviors based upon their experiences. They were coached for two rounds and consciously practiced and experimented with the behaviors. The coach role became a critical catalyst.

    In the final round the team owned, and were executing on these behaviors without coaching.
    Why were they suddenly executing?
    They felt less stressed; they were trusting each other; they felt the positive impact on their work; they saw, felt and shared the value gained by these behaviors. They felt immediate positive consequences from these behaviors.
    Why no more coaching?
    The team were taking it in turns to lead their own stand-ups; they were giving feedback and correcting their own behaviors. These were the additions the teams made to the behaviors as they practiced them.

    • Transparent communication – “open and honest” and “frequent” (Relevant, and timely as agreed with the various stakeholders).
    • Feedback “asking for and giving feedback.” Constructive feedback. (Related to agreed behaviors and responsibilities, calling stop, then swarming to solve an impediment, constraint, or remove waste; immediate fact based feedback).
    • Shared team goals (and cross team). What are we aiming for? Impact on others and their goals. Maybe conflicts that need managing. Understanding strategic portfolio and how we contribute to strategy, show you understand and agree common goals and prioritize accordingly. Goals for both value creation (features and benefits) and value leakage (risks, debt).
    • Leadership within team – team members took turns in facilitating the stand-up and retrospective. This role was respected.
    • Listen – is it clear and understood? Show that you listen, sender: Ask for clarification, ask to confirm understanding, listener: summarize what was understood and agreed. Avoid assumptions.
    • Being open for dialogue, being approachable—not closing the door or hiding behind. All were engaged and equal in the stand-up and retrospective. All were asked for feedback, ideas, suggestions.
    • Giving and taking responsibility. Respecting team behaviors; e.g if somebody calls “stop,” or “respect somebody taking responsibility for an improvement stand-up.”
    • Sticking to agreements. Confront people to understand why agreement not met and what can be done about it in the future (e.g conflicting priorities, lack of understanding).
    • Central moment to get together (e-2-e). Scheduled stand-ups, retrospectives, and improvement prioritization.
    • Knowing the roles and responsibilities, sticking to responsibilities, confirming your responsibilities, confronting people on responsibilities (feedback). Showing that you take ownership.
    • Mistakes are OK – no blame, so long as it is used as an opportunity to learn and improve.
    • Be proactive to get information needed (or ensure information is available, e.g. visible management board, avoid unnecessary “disruptions).
    • Equal treatment: “Respect” ideas from ALL. Ask people for input, engage ALL in ideas generation and decision-making.
    • Don’t HOPE, confirm.
    • “Willingness.” Confirm and agree – “Willingness” and “ability,” empower to be able. Share knowledge, cross train, coach, help.
    • Ownership of responsibilities. Confronting people on responsibilities.
    • Clear agenda and purpose of meetings. My role, what is expected from me? What is in it for me? avoid “wasting” time in meetings.
    • Team coaching to give feedback on behaviors.
    • Create trust by demonstrating team behaviors.
    • Check understanding and commitment to “agreements” and confront people (feedback) on agreements.
    • Improving own work as a team.
    • Asking for and offering help.
    • Having insight, overview, visibility.
    • Limit WIP (reserve WIP for learning, improving, technical debt).
    • Pleasure – celebrating success. Product owner shared results including customer feedback. Team complemented each other.

    The teams were then asked to make a list of behaviors and actions they wanted to take away and apply. Their line managers and assigned team coaches were there when the list was made and gave commitment to help guide, coach and empower teams to experiment and practice these new behaviors in their daily work.

    The team would prioritize one or two behaviors to start with and consistently practice in the coming weeks. When the new behavior becomes the accepted norm and a “habit” they would focus on more behaviors.  This fits in with the Third Way of DevOps: Continuous experimentation and learning. “Understanding that repetition and practice is the prerequisite to mastery.” And to tie this back to Aristotle the philosopher (not the Google project): “Excellence is an art won by training and habituation … We are what we repeatedly do. Excellence, then, is not an act but a habit.”

    Meetings, Including Training Sessions!

    And finally, one of our industry’s most popular, and wasteful, examples of collaboration are poorly run meetings, which are the biggest time-sink in organizations and a source of frustration. I include some training classes as forms of meetings.

    In more than 100 training sessions in the last year, less than 10 percent have started on time.
    Nine minutes late is the average. Delegates say this is normal in their organizations. When I ask people what their own personal learning goals are for sitting eight hours in this “training meeting,” more than 25 percent had not defined this and more than 70 percent had not agreed this with the “sponsor” of the training.

    We need to manage our overload of wasted meeting effort. How? One way is the ask for goal and agenda in advance, asking what value you need to bring in and what value you need to get from the meeting; also to limit your time to the agenda items relevant for you or your team.

    Efficient collaborators decide when they do, or don’t, have unique value to add—“maintaining focus on what matters.”

    The same applies to training, asking, “Why am I going on the training?” “What am I expected to learn?” “What problem is this training expected to help solve?”  “What will I be expected to do differently after the training?” “How will the learning be transferred back to the workplace?” “Will I be mentored and coached?” “Will I be given opportunities to experiment?” “Will I get constructive feedback?” “How can we demonstrate that the new knowledge delivers expected benefits?” This is better than the most common answer, “I don’t know I was told to come to this training!”

    A training is also a form of collaboration between you, the trainer and the person sponsoring the training.

    There is an “I” in Team, after all. “I” am responsible for my effective contribution to team performance.

    — Paul Wilkinson

  • Software Delivery: Shaking Loose of a Local Optimum

    Software Delivery: Shaking Loose of a Local Optimum

    This is an article about software delivery, that magical or onerous field between pushing code to version control and using it in production. Why does software delivery matter? Because software matters, and because software developers matter. I don’t want my software delivery to get out of the way; I want it to make me a stronger developer.

    The path from code to production is crucial because it determines the length of our feedback loop and, therefore, our rate of improvement and that all-important lead time.

    Our current tools think of delivery as generic—they take control, and we fill in the blanks. They let us build, test, deploy. But these words mean something different for every application, on every team and within every business. And there’s a lot more that goes on, or can go on, between the writing and the running of code. There’s often linting, approving and coordinating. There’s data to think about and security policies, business team structure and auditing. There are other steps we don’t bother with, because there’s no slot for them: updating libraries, fixing the code to meet standards, taking a different set of steps for a documentation-only change.

    What else have we not thought of, because it doesn’t fit inside the current set of tools? I want to think about delivery from the top. In doing so, I can gain a greater understanding of my work.

    We’ve seen major shifts in Operations technology: containers, log aggregation, observability, programmable infrastructure and Kubernetes. In my opinion, the killer feature of Kubernetes is its API—it is controllable by automation. It accepts instructions and sends out events.

    The potential here is even bigger than we have realized, for operations and for delivery.

    Modern options for operations have achieved new levels of abstraction and potential. Yet progress in delivery has only been incremental.

    Our tools are very configurable, with plugins and scriptable bits, and we can always fall back to running shell scripts of unlimited length. But are we stuck in a local optimum? Are we thinking big enough?

    What We Have

    Today’s popular software delivery tools are fill-in-the-blank. This is great for getting started, when you don’t know what you’re doing. What about when your business grows, when your delivery is no longer cookie-cutter? Let’s look at two examples.

    Travis CI is a favorite of mine because it’s so easy to get going. It will happily build any public GitHub repository, on its infrastructure, for free. If my repository fits a few simple profiles, such as a Rails app that deploys to Heroku, poof! Some very simple YAML in your repository:

    I like that YAML. It is declarative. It contains data. It fills in the blanks that Travis provides. But the real measure of a tool comes when you get off the happy path. You can do just about anything in Travis; just call out to a shell script. Here’s a small piece of one of my .travis.yml files:

    It has shell commands. Shell syntax for an if-condition. And that infinity of flexibility, a call to a shell script. That travis-build.bash script happens to contain four functions, eight conditionals and (if I’m reading it correctly) up to seven program calls.

    This is programming, OK? As soon as your YAML calls out to a shell, it is not declarative. It is imperative. Or a mix. 😕

    This is programming, but my YAML and Bash do not qualify as software. Bash is a scripting language. It is designed to glue programs together, not to construct programs; it lacks composable modules with encapsulation. I can live without types, but in Bash you don’t even declare function parameters! And it’s full of pitfalls. Use Bash for what it’s good at: calling other programs. As a fallback for flexibility that we lack at a higher level, Bash does not shine.

    YAML is not a programming language, although it does support variables and has its share of pitfalls. Until the YAML contains commands that get executed; then it becomes a program, with even fewer options for error-handling or clear naming than in Bash. Please use YAML for data, not for code.

    There is a gulf between programming and software. We have standards for software: it has design—domain-driven design that we’ve worked through with experts. It has automated tests. It has modularity. It uses libraries and abstractions. I want to use a language designed for all this.

    Skip ahead to Jenkins. This is more widely used than Travis for more extensive delivery flows. It can run on-prem. There are hundreds of plugins available. So many blanks to fill! So many options! And now, there’s pipeline as code. For one thing, you can write a piece of Groovy to define which repositories and branches get a pipeline. That makes Jenkins more responsive to new repositories and branches. It’s a step in the direction of responding to new projects.

    The people building Jenkins recognize that delivery needs to be controlled by the people who control the code. Instead of constructing each repository’s build in a GUI (hello, TeamCity), we put a Jenkinsfile in each repository. That file is in Groovy. Groovy is a programming language with all the modularity and libraries of the JVM behind it. Except, you don’t get all that in your Jenkinsfile. You’re limited to what is available when Groovy runs within Jenkins. If you want consistency from one repository to another, you can install custom-shared libraries in Jenkins and call them in each Jenkinsfile. Rolling out change is challenging; any of those disparate Jenkinsfiles can break after the shared library changes. It’s pretty fragile and still limited. It’s progress.

    This is a step up as programming, but I still wouldn’t call it software. When I write an application, I work in more than a programming language; I work in a language system. A language system includes dependency management, frameworks, linters, build systems, editors, community and the whole ecosystem of libraries available in the language. Great language systems exist around the JVM, Node and .NET, for instance. Groovy that you run can participate in the JVM language system, but Groovy running within Jenkins does not have this access. The Jenkinsfile DSL is still fill-in-the-blank.

    What We Need

    Conceiving delivery as software invites us, development team members, to sit down with the local experts on our deployment environment and security and other concerns. It invites us to have a conversation: What does delivery mean for this application, on this team, in this organization? Where do we want consistency and variability? Then we can create and encode a model—a model of the domain of software delivery, within the context of our business and our environment. We can brainstorm events such as a push or a new repository and what we want to always happen. We can divide the parts into team-specific, organization-specific and community-wide (open source).

    Instead of, “How do run the build? How do we run tests?” we can ask, “What needs to happen after a code push?” And we can think carefully about how we work, how our software works and how our business works with it. We are the domain experts in our own delivery.

    This is the top reason I want delivery as software from the top down. By thinking about delivery as software, we think rigorously about our work as software developers. And we get better at it.

    I want more than delivery-with-some-code. I want delivery as software. Controlled top-down by my team, with a rich ecosystem of libraries, with frameworks and services for the parts that are common in the community. I want delivery that’s event-driven, that’s testable and that communicates with me when it’s time for a decision.

    We’ve come a long way since the days before automated deployment. Our existing continuous integration/continuous deployment solutions have served us as far as they can. Incrementally better (fill-in-the-blank) won’t take us to the next level of software delivery, and it won’t help us reach the next level as software developers. It’s time to break out of this local optimum.

    Let’s Go

    To this end, we composed the Software Defined Delivery Manifesto. Say what you want about manifestos: They have a certain flair, a certain impact, a certain dramatic longing for a better world. That is what we are expressing here. Yes, it’s a better world only for development teams, and only for development teams in complex organizations. But we matter! Software has a huge impact on the world, and to make that software better, we need to get better at delivering it. Let’s push past what we’re used to. Accept that our job is not writing code, but running useful software in production. Let’s get better at delivery by modeling it in software.

    If you’re in, join the movement. Sign the manifesto.

    — Jessica Kerr

  • Low-Code Application Development: A Catalyst to Digital Transformation

    Low-Code Application Development: A Catalyst to Digital Transformation

    Low-code and no-code software development platforms have emerged as a focal point for CIOs, and it’s easy to see why. With an increasing emphasis on market disruption and continuous innovation across industries, tools such as these fit the bill in simplifying application development. Capabilities such as drag-and-drop integrations, out-of-the-box user interface (UI) templates and easy process flows allow anyone—including those with no coding background—to build and change applications. As a result, organizations can quickly deliver new applications and functionalities to keep up with, or disrupt, their markets. Traditional software development cannot match the enhanced collaboration, speed and productivity gains that come with low-code development.

    Laggards need to pay attention: Low-code is establishing itself as a key enabler for digital transformation and differentiator for organizations. Below are five reasons why every organization should embrace low-code adoption.

    Enterprises will struggle to find developers to keep up with global app development demands: Enterprise IT departments cannot keep up with market demand for application development. In fact, Gartner predicts that market demand for app development will grow at least five times faster than IT’s capacity to deliver through 2021. Low-code application development platforms help bridge an organization’s skilled developer shortage gap. Instead of relying only on short-staffed professional programmers, these platforms enable non-programmers to participate in and contribute to application development. Non-programmer citizen or business developers, such as business analysts, process specialists and departmental specialists, can use their experience with visual drag-and-drop tools to work collaboratively with IT to design and modify business applications.

    Low-code increases productivity: Organizational leaders today preach agile processes and methodology to build apps iteratively with constant feedback and collaboration. Bringing the business or citizen developers into application development enables organizations to deliver on the promise of agile. Low-code platforms make it much easier for business users to enter their requirements directly into the application development environment than the old way of creating huge requirements documents. This ensures business requirements are not lost in translation by IT. And just as importantly, the company’s skilled developers have more time to focus on higher-value work, rather than constantly chasing down business users for business requirements.

    Low-code development allows disparate teams to build apps collaboratively: How do IT, operations, business, marketing and sales collaborate to build apps that improve customer engagement and drive operational efficiencies? Instead of code, they need a common language that everyone understands, in the form of visual models. Adopting a low-code platform can empower every part of the organization to contribute to application development. CIOs and IT leaders can tap into the knowledge of their front-line workers to create apps that provide next-level customer experiences. And on the back end, organizations can leverage their internal subject matter experts from various departments to create better internal apps that drive operational excellence.

    Low-code empowers continuous innovation and scaling: Low-code platforms help organizations start small, evolve and then scale to drive digital transformation. Organizations can quickly build out minimum viable products (MVPs), deploy these apps on the fly, learn and optimize and scale these innovations to foster continuous innovation. Without writing complex code, businesses can easily extend their apps to more channels—mobile chatbots, self-service portals, etc. And cloud-based low-code platforms allow organizations to move between cloud platforms or use a hybrid cloud, enabling even more flexibility.

    Competitors are already using a low-code approach: Organizations are looking for ways to innovate, break down silos and reduce costs—and low-code helps them do all of those. A recent Pega-commissioned survey from Frost & Sullivan found that it’s not just hype; 81 percent of respondents already use a visual or low-code approach.

    The question isn’t whether to embrace low-code; it’s what to look for when implementing the technology. Some low-code platforms enable organizations to create only simple lightweight applications that fail to scale and offer limited reusability. Other low-code platforms provide disconnected app authoring environments that perpetuate a siloed approach to development for business and IT. With these types of platforms, organizations end up spending too much time stitching together pieces of apps created in these disparate environments.

    On the other end of the spectrum, next-generation, role-based authoring environments provide the right tools to anyone who touches enterprise applications. These include visual and intuitive authoring experiences for citizen developers, professional developers, system administrators and data scientists—where users can seamlessly toggle between different studios to quickly access what they need, where and when they need it, all in one unified platform. Users can quickly build and deliver MVP—or better yet, minimum lovable product (MLP)— while evolving and scaling the app to deliver enterprise-grade performance.

    The time has come to abandon slow and expensive code-heavy app development. True organizational collaboration and digital transformation depend on it.

    — Sid Misra

  • DevOps: Making Developer Experience Reality

    DevOps: Making Developer Experience Reality

    As DevOps professionals, among other things we are responsible for making developers’ lives a little bit easier. How many times have you heard, “Do you have a working vagrant box?” Or maybe you need to set aside a few days to onboard a new developer.

    Many times a development environment is created when a project is started, usually on a developer workstation. This typically means it will work on that laptop. But, as the project evolves and more engineers get involved, the need to share this environment becomes apparent. When the project grows in complexity, requiring more components and third-party services, this organically grown developer environment quickly becomes unmanageable.

    Get Ready for Production

    Another interesting aspect of the developer environment is the level of similarity with the actual production environment. Even evaluating this aspect requires that you have insight in what production looks like. If projects start on a developer laptop, the desired production setup is seldom considered. More times than I care to remember, I was called into a project when the “thing” that worked well on the development teams’ laptop had no sensible production deployment format.

    This teaches us that even if we start out small, we must think about how the project will function in a production environment and how we are going to get it there. In practice this means that any new project requires at least the following:

    • Some unit tests, they may even start out as stubs.
    • A minimal deployment target environment.
    • Continuous integration (CI), running the (stubbed) unit tests.
    • Automated delivery, preferably continuous deployment (CD).
    • At least one simple integration test, possibly just a curl to the deployment app to verify it is alive.

    This does add some complexity to your project startup, possibly beyond an individual developers’ abilities. It does, however, lay the foundation to, at the very least, be able to deploy the application somewhere. For a seasoned DevOps professional, adding a basic CI/CD only takes an hour or so and there are plenty of free cloud resources for small projects.

    When the project grows and the development environment gets more complex, things get a bit more tricky. The above steps will not really help you mitigate the possible frustration of keeping all the moving part inline. A pattern I see often is that developers try to spin up the entire production stack on their laptop. For a containerized environment, this might mean a Docker Compose-based setup or even minikube to run a local kubernetes cluster. These tools often are using shared volumes for quick iteration, which work well until a large number of files need to be kept in sync. Vagrant is a great way to quickly spin up an environment. This does offer some level of production parity, but certain parts are missing: The CI/CD system watching commits to your source code repository is not deploying to your local setup and traffic to and from it are not passing any additional infrastructure such as load balancers or firewalls, to name a few.

    Local-Remote

    When things break in production, “It it works on my laptop!” is a phrase we have all heard at some time or other. What if we could bring the laptop environment a bit closer to production? It’s not the developers’ fault if things break in production but work well on the local workstation. Providing an option for developers to develop iteratively using a remote infrastructure means we can get closer to production parity. Also, it means that onboarding new developers in the team could be simplified significantly because all the complexity is in the remote environment. This remote environment is provisioned in the same automated way as production, so we have a great degree of confidence about its state. For Kubernetes-based environments, many great tools exist including Skaffold and Draft. They both provide an easy way to build, push and deploy applications to a Kubernetes cluster. This makes it easy for the developer to take on the responsibility to ensure the application under development will run in the target environment.

    If, for some vague reason, you are not containerizing you applications but run code in a virtual machine instead, there are also options. Most modern editors provide automatic syncing with a remote server, either on-demand or when code is changed. Ksync is a tool for syncing your local code into a pod in a Kubernetes cluster on file change. Syncing code live removes the delay of rebuilding a container image and uploading it every time you want to see it in the deployment environment. As such, code sync tools are better suited for iterative development.

    Recently, I worked with a client setting up Telepresence  for a development team. This is also a tool for Kubernetes environments but it offers a twist—developers get to run their code locally with all traffic in and out proxied through a remote Kubernetes cluster. Like with sync tools, this means there is no more long delay before viewing changes, IDE debugging works as normal and all the log output is local as well. When code is committed and pushed, the CI/CD system should update the cluster with the latest version, which in turn can be used to iteratively develop on with Telepresence. It’s a relatively young project but offers great potential. An added advantage is that only a small portion of the entire application stack or landscape needs to run on the developers’ laptop, which means it can do with less memory and a cheaper CPU.

    Conclusion

    Containers and microservices are more widespread than ever, making it increasingly difficult for developers to have a production-like environment to develop in. The key to solving this problem is to not try to load everything into an evermore powerful laptop, but splitting the load and complexity between developer workstation and a remote environment.

    — Alex Knol