Category: Enterprise DevOps

  • Era Software Adds SaaS Edition of Log Management Platform

    Era Software Adds SaaS Edition of Log Management Platform

    Era Software has unfurled a software-as-a-service (SaaS) edition of its platform for collecting and managing log data that is based on a time-series database it built.

    At the same time, the company has revamped the user interface of the underlying EraSearch log management platform that IT teams can deploy themselves to make it simpler to troubleshoot IT issues.

    Stela Udovicic, senior vice president of marketing for Era Software, said the EraCloud edition of the log management platform provides a more accessible way to access a log management platform based on an object storage system that can handle any type of log data.

    There’s also an indexing engine based on machine learning algorithms that enables the platform to scale better than rival platforms, she noted.

    Available via the Amazon Web Services (AWS) marketplace, EraCloud collects, indexes and analyzes log data from cloud applications, Kubernetes and IT infrastructure platforms to provide a centralized means of collecting data at a lower cost. In addition to being able to continuously monitor billing costs, an EraSearch cluster can now be provisioned in minutes.

    Era Software, previously known as EraDB, earlier this year raised an additional $15.25 million in financing to drive development of a log management platform that provides an alternative to open source platforms, such as Elasticsearch, that don’t scale as well. The result is a higher total cost as the volume of log data that is less efficiently managed via Elasticsearch steadily compounds. As part of that effort, the company also supports a subset of the Elasticsearch application programming interface (API) to facilitate transitioning from Elasticsearch.

    Many organizations today already have terabytes of log data they are trying to manage and store, noted Udovicic.

    It’s not quite clear how many IT organizations are considering alternatives to log management platforms. However, as the amount of log data that IT teams want to be able to store over a longer amount of time increases—in part to facilitate greater observability—the pressure to contain the cost of retaining all the data is also rising.

    Observability, in general, requires combing logs, metrics and traces—a specialized form of logging—in a way that makes it easier to identify the root cause of an issue by querying data. It differs from monitoring in that observability is achieved using a predefined set of metrics that track whether a platform or application is performing within expectations.

    It may be a while before most IT teams generally achieve observability, but the journey typically starts with finding a more efficient way to collect log data in both proprietary and open source formats. Rather than having to normalize all that data, Era Software is making a case for analyzing that data in whatever format it happens to be available.

    One way or another, a much larger percentage of the applications deployed by IT teams in production environments will have to be instrumented to achieve observability. The issue now is determining how best to make all the data collected from those applications available to DevOps teams at a reasonable cost.

  • Successful Software Development in a Hybrid Environment

    Successful Software Development in a Hybrid Environment

    The desire to work from home or from anywhere has only increased as the COVID-19 pandemic lingers. Employees cite a number of reasons why they prefer to skip the office, including cutting commuting times, being more productive, caring for family members and exploring new locations. People are increasingly comfortable with remote interactions and the new technology tools that have sprouted to support the work-from-anywhere trend. On average, employees said they want to work from home 2.5 days a week, according to a monthly survey of 5,000 Americans, while 73% of employees seek hybrid work environments where they have some flexibility in where and when they work. 

    The rapid spread of the COVID-19 Delta variant also prevented employers from enforcing a full-time return to the office. As a result, the new normal for today’s workforce will remain hybrid at best—a few days in the office and a few days working at home. We are a workforce in transition. For many companies, this has put an unnecessary strain on their digital transformation initiatives.

    A question remains: How can companies continue to innovate and execute on their software development initiatives with their employees separated from their teams and offices? Here are some of my recommendations for helping organizations to continue their software development projects in today’s hybrid work environment:

    Inclusive Environments

    Hybrid workforces require a highly inclusive environment. As a nearshore IT partner that works with hundreds of clients remotely, we ensure that our teams have the same access to managers, HR and the technology necessary to succeed on projects and to productively engage with clients. For example, we have learned that the best way to be inclusive is to take either an all-virtual or all-in-person approach during client meetings. It’s also critical to train your managers to provide the same advancement opportunities to employees regardless of whether they are in or out of the office. From the executive team on down, everyone needs to support employees, regardless of where they are. The same goes for onboarding new employees and partners—a mostly virtual process over the past 18 months. Increasing online mentoring and onboarding platforms to connect employees and to manage client projects while the workforce is hybrid is another significant approach we have taken to ensure everyone feels included and engaged. 

    Collaboration and Knowledge-Sharing 

    Employees were abruptly forced to transition away from the office where they were used to frequent impromptu meetings and lunchroom chats. For many, this shift had a negative impact on brainstorming, collaboration and knowledge-sharing and thus, innovation. We give our teams the flexibility to use different technology tools to make up for what they miss most in an office environment—regular, casual human interactions. Our teams use a variety of communications tools for frequent check-ins to connect with clients and their co-workers. As a result, our people say they are more engaged with team members and clients than ever before. They are doing more impromptu chats and “instant” meetings with colleagues and clients as if they were in the office. We also migrated to video conferencing as our communication channel of choice. Video conferencing tools increase engagement and allow team members to join virtual meetings anywhere from their preferred devices. The ability for teams to interact, share and collaborate wherever they are is essential to developing cutting-edge teams and software. 

    Trusting Cultures

    According to the Harvard Business Review, 34% of employees felt that their supervisors “expressed a lack of confidence in their work skills,” and an even more significant number of workers reported feeling that they needed to be constantly available to prove themselves. Sentiments like these highlight a lack of trust, impacting development outcomes. Employers need to trust that their employees are working diligently and consistently throughout the workday or whenever they can successfully complete the tasks at hand. For example, working remotely allows individuals to work when they feel most productive. Organizations need to recognize individual productivity cycles and understand that different team members may be productive during different times of the day, including outside of normal business hours. 

    Being transparent, forthcoming and open about expectations are keys to building trust with employees. Successful digital transformations and innovative software development are only possible with the right team and people. From a client’s perspective, these are the types of IT partners they want on their side. Technical aptitude and an environment where all employees feel valued, trusted and motivated to do their best are critical for developing effective digital solutions.

    Employees have made it clear that they don’t want to be back in the office full time. For many organizations, a return to normalcy centers on developing a hybrid workforce that can work effectively with digital development partners. Creating the right environment and strengthening employee relationships are keys to ensuring software development projects can continue successfully in today’s new normal.

  • What Developers Need to Know About Edge Computing

    What Developers Need to Know About Edge Computing

    More and more companies are living “on the edge” as they move from “I’ll think about edge computing” to “I don’t think our business can optimally grow and compete without edge computing.” Developers will play a key role in helping organizations use the edge, but their effectiveness will depend on having edge-aligned processes, products and people in place.

    Edge computing takes place near the physical location of either the user or a data source. It’s one way that organizations can use and distribute a common pool of resources across multiple locations. The edge model provides users with faster, more reliable services while expanding the potential of hybrid cloud computing for organizations. Customers and businesses alike will benefit from the way in which edge computing optimizes the use and application of artificial intelligence and machine learning by running data-intensive applications closer to the data source.

    While it is in many ways a question of architecture, applications are key to effectively exploiting the edge. Here are some of the things developers—and their organizations—need to know.

    Edge computing and cloud computing are not mutually exclusive. The best solutions will leverage both platforms. For example, centralized computing might be used for storage and compute-intensive workloads, while edge computing would be used to exploit data at the source, in near-real-time. Infrastructure and company culture need to be in place to support this, but developers must be thinking in terms of building applications that can cross and bridge computing platforms.

    Consistent tooling is essential. Developing for legacy IoT was very specific and required specialized tools and skills. The modern edge, in contrast, is part of a technology continuum, and developers need tools that they can use across that continuum, including:

    • Provider/enterprise core: The “non-edge” tiers, owned and operated by public cloud providers, telco service providers or large enterprises.
    • Service provider edge: Tiers are located between the core or regional data centers and the last mile access.
    • End user premises edge: Tiers on the end-user side of the last mile access can include the enterprise or the consumer edge.
    • Device edge: Standalone (non-clustered) systems that directly connect sensors/actuators via non-internet protocols. This represents the far edge of the network.

    Containerization enables flexibility, scale: Linux container technology wasn’t built for the edge, but the model fits perfectly with edge requirements for modularity, segregation and immutability. 

    Keep an “open” mind. Open APIs open up new opportunities. Developers using open APIs can access real-time data programmatically, making it possible to build applications that can leverage data as it’s “born.” Likewise, a focus on open source development tools and platforms will help to ensure that organizations can continue to leverage the edge as the model evolves. 

    What’s different is the devices. Developing for the edge is just like developing for any other platform–except that developers must write consistent and secure code that can be seamlessly copied to all device types. 

    Edge computing may not start with developers, but developers will play a huge role in organizations’ ability to leverage the edge. A focus on flexibility, openness and the future will help ensure that developers build applications that fully exploit and expand the edge advantage. 

  • Building and Leading Product Teams in Uncertain Times

    Building and Leading Product Teams in Uncertain Times

    The world seems to be changing faster than ever, and there’s no reason to believe it will slow down anytime soon. The pandemic has compounded this sense of constant flux, with customer priorities shifting overnight. Against this backdrop, product teams need to constantly evolve their development work without losing focus or feeling they need to react to every small change in the market. 

    This requires team leads to throw out their old playbooks and develop a new strategy that allows them to deliver continuous innovation without creating chaos for their teams or losing sight of their north star. Road maps are shrinking from years to months, and are now being created with the understanding that they may become outdated or redundant at any time.

    Building and maintaining an organization that can not only survive but thrive in these conditions is a daunting task. It requires a structured, programmatic approach that also supports agility and adaptability. Those requirements may feel at odds, but they actually are not. Here are five tactics to help your product strategy evolve with the times. 

    Keep That “Day One” Mentality

    It’s important to keep the soul of your product top of mind. It helps to retain a “day one” mentality which means constantly reminding yourself why you started in the first place and what problem you set out to solve. That founding problem statement should serve as your guiding light, especially through times of rapid growth and change.

    Customer requests are, of course, a high priority and should always be thoughtfully considered. But ask yourself if the request is aligned with that day one problem you set out to solve. If it’s not, work with the customer to identify the underlying issue they’re really trying to solve for versus focusing on the symptom they’re experiencing. 

    Cultivate a Culture That Thrives on Change Without Succumbing to Chaos

    It takes a combination of strong organizational capability and the right culture to build a team that can thrive on ambiguity. Strong organizational capability is achieved through talent, architecture and execution skills. 

    Great product teams build an architecture that supports iteration and are able to execute toward their north star in small, rapid steps. Team leaders should build a culture that takes failure in stride and learns from it to improve each subsequent iteration in the product journey. 

    Change is inevitable, but chaos is avoidable. Leaders must create an environment that feels psychologically safe and supportive, so that team members feel confident in changing priorities and making rapid decisions. 

    Build an “Innovation Factory” or a System that Reduces the Potential for Chaos

    Innovation and agility require well-planned foundational resources. Following are the key steps that can help put these building blocks in place.

    • Think of velocity as a part of your product. Teams should be broken into small groups to autonomously drive the product forward, but in such a way that the overall experience feels seamless to end users.
    • Democratize the product concept phase by creating conditions for a large number of ideas. Everyone involved should be inspired to contribute, including employees, partners and customers.
    • Create a system for product development excellence. Fully automate your product development life cycle by approaching developer infrastructure as if it were a service, with clear service-level objectives within a CI/CD framework. Digitize each step in the product concept, development and deployment journey, with a focus on measurement, learning and iteration in every phase. Ask who and how many people are building versus approving at each stage and ruthlessly optimize and automate where possible. 

    Bring a Service Mindset to Everything you Do

    If you’re building a SaaS product, as many startups today are, you’re no longer delivering a product in the traditional sense. You’re delivering a service, and you’re responsible for the entire experience from start to finish. This means you’re in full control over when to release new features and how you measure user engagement. You can see where the friction points are, and where customers are delighted. We have a philosophy at my company of “less input, more output”—simplicity is the name of the game, where the output and value your service creates for customers is the only measure that matters. 

    The outcomes required may change as customer priorities evolve and new requirements flow in, but adapting quickly and effectively should just be a matter of (re)connecting well-built services within your platform to prioritize and address those changes. Leverage the cloud ecosystem in a way that allows you to safely bet on other players and partners who are moving as quickly as you are. Focus on making sure your foundation is strong; one that can provide a service at scale even as the outcomes customers want change over time. This allows you to stay laser-focused on the task at hand: Maximizing value for users. 

    Incorporate Users in the Product Development Cycle and Learn as you Go

    The cloud has transformed the way organizations build products and integrate feedback. The entire life cycle is digitized and measured. Product teams can move at a much higher velocity and in smaller increments, with the ability to make near-real-time adjustments along the way. But the greatest advantage a cloud environment offers is that we can now include users in the entire product life cycle, every step of the way. It gives us the ability to measure outcomes quickly and take action on the information that immediately benefits customers.

    It sounds counterintuitive, but product teams can prepare for the unknown. There’s no way to see what’s over the horizon, but with the right people, processes and tools in place, there’s no reason that uncertainty should derail you. There has never been a more exciting time to be an innovator or a builder. Meet the uncertainty ahead with open arms and a strategic eye.

  • 4 Strategies to Manage Freelancers with In-House Dev Teams

    4 Strategies to Manage Freelancers with In-House Dev Teams

    Remote workers are a growing segment of the global workforce. Freelancers account for a big part of that segment, especially when development teams need to bring in specialized skillsets for upcoming projects. Working with freelancers offers many benefits, but there can also be challenges when integrating freelancers with in-house teams.

    Here are four strategies to help in-house developers effectively manage freelancers in their development teams:

    1.    Keep Remote Workers in the Loop

    Don’t leave freelancers and remote workers in the dark about your overall business direction and goals. If a freelancer is only joining the team temporarily to handle one part of a project, it might not seem necessary to give them the full details about the project’s goals and the reasons behind your decisions.

    However, providing this information enables every member of the team to fully understand what you need from the products and features they are working on and how they will serve the end user. It also helps to create common goals and motivations that the team can work toward together, rather than keeping your remote workers’ tasks separate from other members of the development team.

    2.    Make Consistent Feedback a Priority

    In an OfficeVibe survey, 96% of employees said they wanted to get regular feedback from their managers, but 32% said they receive feedback less than once every three months. Freelancers and remote workers arguably need more feedback than in-office employees, for several reasons.

    Freelancers and remote workers are often less exposed to the wider business context in which they are developing products and features. This can make it harder to assess and optimize their output from the viewpoint of an end user. Just as importantly, it can make it less clear what the other members of the team are working on and how their output affects their colleagues’ work.

    “Without consistent feedback, even the most talented developers in a team can invest a lot of time in a non-feasible solution. An ongoing feedback loop is one of the best ways to support a team and enable them to put their skills to use,” said Irene Avdus, a product manager at TextMagic, which has offices in multiple countries and which regularly brings on freelance talent.

    Giving freelancers and remote workers consistent and regular feedback doesn’t just help them improve their output for a better end result. It also helps them to fit in with the workflow of the rest of the team, enabling the project to go more smoothly for everyone.

    3.    Provide Growth Opportunities

    “Great leaders invest in their employee’s career development and steer them from being a one-trick pony,” said Phidelia Johnson, Pac-J Investment Services executive director of HR.

    When it comes to upgrading a team’s skillset, companies tend to skip over freelancers and remote workers in favor of employees they see as more permanent. However, opportunities to develop skills and advance their career are just as important to remote workers and freelancers as they are to any other employee.

    As a result, the perception of remote workers and freelancers as more temporary team members can be a bit of a self-fulfilling prophecy. When companies don’t invest in developing employees’ skills outside of their in-house teams, it is to be expected that remote workers will move on after the project is done to advance their careers.

    Often, treating remote workers as temporary doesn’t result in any less training, anyway. Whenever you add a new freelancer to your development team, there will be a certain amount of training and onboarding necessary to get them up to speed with how your team works. In fact, this can often be a major time and resources investment

    “In a lot of situations, the biggest challenge of working in data engineering or BI is not learning technology X or Y—it is being able to pick up quickly all the business rules and processes of the department you are building your pipeline or dashboard for,” said Mohamed Khalil Ouachikh, ArmedAngels data engineer and BI specialist.

    Rather than spending time onboarding new remote workers with each development cycle, investing in upgrading and expanding the skills of current team members and freelancers can result in a more capable team that is already familiar and comfortable with your business and the needs of your clients.

    4.    Informal Team Meetings

    Making people feel like part of a team is important to ensure they can collaborate and coordinate effectively with other members. Both of these factors are crucial to developing any successful product.

    Why is this relevant to managing freelancers in your development team?

    Remote workers don’t have the same opportunities to connect with colleagues as in-house employees unless you provide them. As many as 20% of remote workers say they feel isolated from their colleagues. This can take a toll on their emotional well-being, and that lack of social connection to coworkers will also affect their ability to collaborate.

     As a result, they may not be able to communicate as effectively with other team members. Building connections between team members let them understand each other more easily and reduce miscommunications that could lead to development mistakes and delays.

    Informal meetings via webinar platforms or video conferencing solutions can have a significant positive impact in this area. A weekly meeting gives everyone a chance to talk about what they are working on and their challenges. As a result, every team member, including freelancers, has the chance to lend their skills to a colleague’s work and help the team pull together toward achieving a common goal.

    Just as importantly, it gives remote workers a much-needed opportunity to connect with their colleagues socially and chat about things other than work. Besides the importance of team-building, this develops more of a connection between remote workers and your business. Inviting freelance developers to all meetings and social events is a great way to ensure greater inclusivity, and it can pay off if freelancers later take full-time positions at the company.

    Make Freelancers Part of the Team

    Keeping freelancers in the loop and ensuring they have the same opportunities and voice as other employees ensure they have the same level of investment in a project’s success as every other member of the team. The tools and the opportunity to coordinate with colleagues and receive feedback are critical parts of this. Following these strategies will help you integrate and manage your freelancers more effectively.

     

  • Observability Matters. But Are You Observing the Right Things? 

    Observability Matters. But Are You Observing the Right Things? 

    It’s like clockwork. Every few months, a new buzzword descends on the tech industry, and soon it’s on everyone’s lips. Take your pick from the graveyard of buzzwords past: “Big data,” “internet of everything,” “hacktivism,” “gamification.” It’s not that these terms are meaningless—just that they go from relatively obscure to ubiquitous seemingly overnight. We can add a new one to the list in the digital monitoring space: “Observability.”

    Unlike some trendy buzzwords, observability captures something that’s actually quite important about how we monitor digital systems and how current approaches differ from those of the past. So, it’s a good and useful concept. But where I take issue is with the way many analysts, technologists and vendors seem to use it. Whether or not they mean to, they often paint a far too narrow a picture of what observability is or ought to be, and that linguistic oversight can have real-world consequences. If enterprises invest in observability without understanding what, exactly, they’re observing—and what they’re not—they risk leaving themselves vulnerable to some serious blind spots. 

    What is Observability and Why Does it Matter?

    Technically, observability is an approach with the goal of understanding how a system is doing. The distinction lies in what the system is and how it is being observed. The monitoring tools of yesterday typically focused on one or more domains to identify typical and known problems. Usually, that entailed collecting metrics directly from those domains and measuring them against predefined thresholds. For example, storage levels shouldn’t exceed a certain threshold; if they do, send me an alert so I can investigate what’s happening. 

    On the other hand, observability takes a more holistic approach, examining multiple outputs of the system to infer system health. For example, suppose you were trying to gauge a car’s condition. In that case, traditional monitoring might track engine temperature or idle speed RPMs, while observability would examine things like power output, engine leaks, transmission noise, emissions and fuel efficiency to infer the car’s overall health. 

    This output or outcome-focused approach to managing performance maps well to today’s complex and distributed digital systems. Even if you wanted to, you couldn’t pull real-time metrics from every relevant component of a system when those components consist of thousands of microservices running in any number of physical servers across multiple clouds. However, the bigger problem with more antiquated monitoring models is that they only capture the specific metrics they’re designed to track. If you’re encountering a problem you’ve never seen before, your monitoring dashboard might look perfectly healthy while actual system performance is terrible. 

    Observability and APM 

    So then, observability should give you everything you need to track the health of a modern digital business, right? Well, not so fast. Because for today, people seem to use “system observability” interchangeably with “application performance monitoring,” or APM. And they are not at all the same. 

    Yes, you want an observability-based approach to tracking your application environment, and modern APM tools provide a great way to do it. But knowing the health of your application environment is not the same as knowing the health of your overall digital systems as experienced by users. And at the end of the day, if you’re not gauging performance from your users’ perspective—whether that’s customers, employees or other devices relying on your systems—then what’s the point? It’s unfortunately not uncommon to forget that the most important key performance indicator (KPI) of any digital system is the quality of its intended output or outcome. In the case of any application, the most important KPI is the user experience.

    The outputs that APM tools track do contribute to the user experience—and you want to be monitoring them. But so do many, many other things. There’s the user access itself (browsers, virtual or physical user devices loading the application). There are local, regional and global networks connecting those users to your application environment. And then, of course, there are the dozens, sometimes hundreds of third-party digital services running between and among users and applications. Problems in any link of that digital chain can seriously impact the user experience—even if your application environment itself looks healthy. 

    Observing End-to-End, from User to App and Everything in Between 

    To get the full picture of the health and performance of your digital assets, you should be tracking outputs from across the end-to-end digital service chain. That includes telemetry (logs, infra metrics, traces) from your application hosting environment. It should include actively and passively observing at the point of consumption—that is, outputs from customer browsers and worker devices. It must include active and passive monitoring of network connectivity. And it should include continuous active monitoring of all the disparate services—APIs, content delivery networks (CDNs), DNS services, cloud-based security services and every other cloud and web service that contributes to what the user is experiencing. (And by the way, you should look for independent, objective telemetry whenever possible rather than taking what your various service providers report at face value.) 

    When you approach observability from this outcome-based, holistic perspective, you can capture a real and truthful picture of the performance of your full digital business, not just pieces of it. Just as important, when you do identify a problem, you’re in a much better position to do something about it. Because now you have outputs to measure from all the many systems and services that contribute to the user’s application experience instead of just a subset of them, which means it’s much easier to zero in on which piece of the puzzle isn’t behaving the way it should. 

    A Smarter Approach to Observability

    So, should you take all the hype about observability seriously? Yes. As tech buzzwords go, you could do a lot worse. This one actually means something important. But make sure you’re keeping the big picture in mind as you do. 

    Observability that ignores the outcome begins and ends with the application environment and is nowhere near sufficient. We need to expand our sense of observability, begin with the user experience and encompass every link in the chain of the end-to-end digital business. When we do, we create a much more holistic, useful and actionable picture of the health of our digital systems. And we can make sure we’re continually monitoring digital performance in the way that matters most: As experienced by users. 

  • Allstacks Adds Free Dashboard to Track Software Dev Benchmarks

    Allstacks Adds Free Dashboard to Track Software Dev Benchmarks

    Value stream intelligence platform provider Allstacks today added a free live dashboard populated with anonymized data that DevOps teams can use to benchmark their own progress.

    Hersh Tapadia, Allstacks co-founder and CEO, said the company is making this data available to make it easier for DevOps teams to uncover bottlenecks in their software development processes.

    As part of that effort, the dashboard includes pre-configured benchmarks for the following:

    How many days per week do benchmark teams write code?
    What percentage of issues are planned versus unplanned for benchmark teams?
    What are the cycle times per issue type for benchmark teams?
    What types of code do benchmark companies produce?
    What is the average commit size for benchmark customers?

    Allstacks

    Allstacks has committed to making 12 months of data publicly available on the dashboard and said it will update that data on an ongoing basis. Allstacks customers, however, can also access five years of data to compare historical trends.

    Tapadia said Allstacks plans to add additional benchmarks to a dashboard that can collect data via the various application programming interfaces (APIs) that most DevOps tools now routinely provide. The overall goal is to increase transparency into software development processes, he said.

    Allstacks’ value stream intelligence platform uses machine learning algorithms and artificial intelligence (AI) models to identify projects that are at risk of not meeting delivery goals. That data is then used to analyze the impact those missed deadlines are likely to have on various business objectives.

    Value stream management platforms automatically take technical metrics, gathered via a DevOps process, and match them to a set of key performance indicators (KPIs) defined by business leaders. The challenge many organizations still face is making sure they have the tools in place to collect the appropriate technical metrics and then define the KPIs that truly matter to the business. Once those metrics are gathered and KPIs established, organizations are then able to apply advanced analytics to the massive amounts of DevOps data they collect.

    The concept of value stream management traces its lineage back to lean manufacturing methods, which called for each step of a manufacturing process to be continuously measured. As software development has evolved from being a craft to a process that is highly automated using DevOps best practices, an appreciation for the value of monitoring the impact of missed software development deadlines on the business has grown as organizations realize how dependent they are on software.

    As a result, more organizations realize there is a clear disconnect between their business goals and development processes, noted Tapadia. Business leaders are especially anxious to understand how a software development project delay might impact future revenue projections. Rather than being a set of tools employed mainly by IT leaders to track software development projects, value stream management platforms make it possible for other stakeholders to track KPIs spanning multiple software development projects. It then becomes easier to make informed decisions about how to reassign resources across various projects as different bottlenecks are encountered.

    It may be a while before organizations achieve that level of transparency into their software development processes, but as software continues to ‘eat the world,’ it’s only a matter of time before value stream management becomes a required set of DevOps tools for every organization.

  • Automated Refactoring with Moderne

    Automated Refactoring with Moderne

    Jonathan Schneider and Olga Kundzich are co-founders of Moderne, a Seattle-based startup. Moderne received $4.7 million in seed funding in July 2021 to commercialize OpenRewrite, an open source automated refactoring tool for code (initially Java) that Schneider started while at Netflix. Moderne’s technology reduces the tedium and time it takes for software remediation; a big problem in today’s world of cloud-native development. The video is below, followed by a transcript of the conversation.

    Announcer: This is Digital Anarchist.

    Alan Shimel: Hey, everyone. Welcome to another segment of TechStrong TV. We have a new company to detail for you, and we have two cofounders here to tell you all about it. It’s a great open source story as well. Let me introduce you to – and I’m gonna do my best not to mess it up – Olga Kundzich, Kundzich, Kundzich – three times the charm – and Jonathan Schneider. I think I got that one on the first try. Olga, Jonathan, welcome to TechStrong TV. 

    Olga Kundzich: Great to be here. 

    Jonathan Schneider: Thanks. It’s good to be here. 

    Alan Shimel: Thank you. So the only rules is only one of you can talk at a time, but both of you can talk as much as you’d like. But why don’t we start off with maybe a little bit about each of you. Olga, if you don’t mind, why don’t you go first and maybe share with the audience a little bit of your background and how you came to be here today. 

    Olga Kundzich: Great. So my background, I was working at Pivotal on a Spinnaker continuous delivery solution, and that’s where Jonathan and I had met. We worked together there for about a year and a half. Prior to that, I worked at EMC in engineering for, in different functions for a number of years. So when we were working together on Spinnaker, we talked to a lot of people in the industry. There was a lot of interest around that product, and kind of we realized that everyone had sort of the same problem. We would start a conversation about some old feature, and it will come back to, well, all my application is still stuck on this old, aging framework, and let me talk to you in a year when I migrate all of this back.

    So and the problems were highly repeatable and kind of same framework, same third-party vulnerabilities, same libraries, APIs, so after reflecting on this we realized that this is really not a coincidence. It’s a consequence of modern application development practices. Our software right now is 90 percent third-party dependencies, open source, APIs, so the problem’s kind of highly repeatable. And that’s kind of led, what led us to, back to technology that Jonathan developed earlier at his time with Netflix and kind of take it, develop a lot of more functionality into that sort of large-scale distributive factoring that will help us solve these type of problems, the framework migrations, API migrations, CVE patching that cross all of these open-source and third-party dependencies. 

    And I will let Jonathan speak about his background and a little bit on the open-source framework.

    Jonathan Schneider: Yeah, so Netflix had this engineering culture called freedom and responsibility, so product engineers were sort of, had the freedom to do whatever they wanted really. They could use their, a different build tool, they could use their particular Java style, they could use whatever libraries they wanted, and so forth. And so as a central team, if you’re on an engineering productivity team or you’re on a platform team and you’re trying to effect some change inside the organization, we couldn’t put up gates in front of product engineers that would, say, break their build if they didn’t comply with a certain thing. Instead, you tried to build tools that would help bring them along on a journey with you, and I kept hearing the same thing over and over again from engineers which is, hey, I got a lot on my plate. Happy to move forward to this new pattern that you want to see, if you do it for me. If you just do the work for me, I’m happy to commit it. 

    Alan Shimel: Nice of them. 

    Jonathan Schneider: And of course, you know, you think there’s two of us and 700 of them, and that doesn’t scale super well. Right? So that’s where we started working on a way of actually automating the source code transformation. When we developed that technology, one of the consequences of that freedom and responsibility is that there was not a consistent style of the code or consistent patterns, necessarily. So that refactorization we built had to make transformations that were, looked like they were a developer writing in that code base stylistically consistent, hundred percent accurate changes. And so we got some interesting results out of that, so when Olga and I were working together and you would hear, “Oh, I’m still stuck on Spring Boot 1 and I’m trying to get to Spring Boot 2,” or, “I’m stuck on JUnit 4. I’m trying to get to 5,” we just found another application of this at a larger level in a highly repeatable problem space. 

    Alan Shimel: Fair enough. You know what, I’ve spent the last 20 plus years talking to founders. I’ve founded my own companies. This is what happens. You see a real problem. Right? See a real-world problem. There’s got to be a better way we could solve it. Sometimes you find out solving that problem is not as easy as you thought. Sometimes it is, very rarely. But you work through it and that’s what startups are all about. In your case, though, I mean, look, Spinnaker, which is now part of CDF, right, was managed by CDF, but Google, Netflix were instrumental in the development of Spinnaker. But Spinnaker is an open source tool, and you guys went with an open source route in what you’ve developed. Why don’t we talk about that a little bit? 

    Jonathan Schneider: Sure, yeah. There’s really two parts to our product offering. There’s OpenRewrite, which is GitHub slash OpenRewrite. OpenRewrite is that core refactoring technology we’ve been talking about that can make stylistic automatic and consistent changes to a code base. On OpenRewrite we built recipes, so we have a recipe to migrate you from Spring Boot 1 to 2, from JUnit 4 to 5, to do code cleanup things, to do security vulnerability fixes. We want those recipes to always be Apache licensed and open source. Framework authors are going to develop the recipe, so the Spring team will develop the recipe that will move you from Spring Boot 2 to 3, or the Quarkus team at IBM or Red Hat would develop the recipe that’d move you from one version to another.

    What Moderne does commercially is we take those recipes and we run them at massive scale, so I could take that recipe and run them against tens or hundreds of millions of lines of code, and you can see that, the impact of that recipe running against your whole organization’s code base. 

    Alan Shimel: Fair. And so you mention – how did – it’s not Modern. It’s Moderne. Correct? 

    Jonathan Schneider: Yeah, we call it Moderne. Yeah.

    Alan Shimel: Moderne. So that’s the name of the company.

    Jonathan Schneider: It’s the name of the company, yeah. 

    Alan Shimel: And OpenRewrite is the open source aspect of it or –

    Jonathan Schneider: That’s right. 

    Alan Shimel: – owner of the solution. So let’s get into the mind of the entrepreneur here a little bit, Olga. Having worked at Pivotal and EMC for years before this, you saw this issue. You saw this problem. There’s got to be a solution. When did you say, you know what, this is something worthy of starting a company, right? When I was starting companies, one of the big questions is, is this a feature or a product? Right? Is it just a feature of someone else’s product, or is it truly a product in and of itself? And that makes a big difference in, obviously in what you’re building and what your aims and all of that. Olga, I’d like to hear from you. Feature, product, how did – what was kind of your thought process in saying, you know what, this could be its own company?

    Olga Kundzich: I think this, something like this is essential for managing modern cloud-native applications. These are becoming, the applications sort of become a glue code teaching all of these third-party components. They evolve and change at their own pace, and developers really right now are just restitching those things back together, upgrading CVs, upgrading versions, and it takes a really significant amount of time just to – sort of anecdotally we hear from people that 20 to 30 percent of their time, engineering time, is just now dedicated to these tedious migrations. So if we can automate this, we can unlock innovation, productivity of engineering organizations, and help a lot of businesses actually innovate.

    Alan Shimel: Got it. Jonathan, what about from your end? Well, let me ask you both. Is this your first time starting a company? 

    Jonathan Schneider: It’s first time starting a company, yeah. I think it’s real special to me. I think one thing I love about this business is that we’re really after helping organizations fix their code, and our outcome and our future success is tied directly to how much code we’re able to fix, how much help we’re able to provide. We’re not about reporting and saying like, you know, here’s the problems that you, engineering team, need to fix. When we partner with an organization, we’re kind of in the trenches with them getting these issues off their plate.

    We talked to one organization recently – and this is why I think it’s a product and not a feature. We talked to one organization recently and they say we just look at our top ten issues reported by a series of static analysis tools they have in-house, and it says, well, it’s gonna, these tools estimate it’ll take 1,700 developer days to fix these top ten issues. So that’s like 4.8 years’ worth of developer time to fix those top ten issues that they have, and those aren’t false positives. Those are just literally top ten critical problems they have right now. So we developed recipes for a lot of those, and their engineers said, you know, “We estimate it would take us five days with these recipes to apply that change and be done with it.”

    When I think about 1,700 developer days or 4.8 years of time, what that really means to me is infinity, or like it means that the engineering team is never going to get through that backlog of issues. Five days seems like it is possible, and that’s, there’s this line we have, the, basically software grows until it can no longer be maintained, and I think our business is in keeping that software in a maintainable state so that the business can continue to innovate and thrive. 

    Alan Shimel: Absolutely. So companies launch now – we didn’t even mention the website. Shame on –

    Jonathan Schneider: Yeah.

    Alan Shimel: What’s the website? 

    Jonathan Schneider: Yeah, so the website is moderne.io, so it’s modern with an E.

    Alan Shimel: Can you spell that, Jonathan? M-O- –

    Jonathan Schneider: Yeah, it’s M-O-D-E-R-N-E, so modern with an E, dot I-O.

    Alan Shimel: Dot I-O.

    Jonathan Schneider: Yep, that’s right. We do have a restricted beta going on right now that you can just sign up for that on that website, and we’ll do an invite to you and you can see how we can apply these recipes across tens of millions of lines of code. 

    Alan Shimel: Absolutely. Are you working with CDF and the Spinnaker community?

    Jonathan Schneider: We’re really in an adjacent product space now. I think there’s a lot of value to operational automation, what Spinnaker is doing, what different APM vendors are doing. I think we were working in that and we saw just an adjacent pain point that we just decided to go after instead. 

    Alan Shimel: Good stuff. Well, guys, I want to wish you nothing but a lot of success with this. It’s always, you know, I gotta be honest with you. I always enjoy interviewing new companies coming out of stealth and beta, first-time entrepreneurs, ’cause you make the world go around. And good luck. Keep us posted. Come back on and tell us more as you, as the story continues to unfold here. 

    Jonathan Schneider: Yeah, look forward to talking again soon. 

    Alan Shimel: All right. 

    Olga Kundzich: Thank you.

    Alan Shimel: Jonathan and Olga from Moderne, M-O-D-E-R-N-E, dot I-O, here on TechStrong TV. We’re gonna take a break. We’ll be right back.

    [End of Audio] 

  • How Dispersed Can We Be?

    How Dispersed Can We Be?

    It is interesting to consider DevOps from the storage perspective. DevOps in general—and specifically newer development and deployment models—tried to pretend that long-term storage was not a necessary component of software delivery. In short, it appears that because there wasn’t a good answer to long-term state in stateless development models, the storage and databases that used state were hand-waved away. Which, of course, failed. Container deployment models did the same thing and failed just as miserably. Oh, containers didn’t fail, but the attempt to ignore persistent storage did; from the very beginning, it was clear that was going to happen. We were doing early container implementations and working through how to ensure that containers ABC had access to the same shared storage. That need never went away and was never going to. Apps can run without persistence, but business doesn’t run without persistent data.

    Checking out the wave of cool new products like, for example, Ionir (disclosure—I know more about them simply because I’m friends with their CMO. Other vendors likely have good solutions, too) makes me realize the answers that we needed were right in front of us. And not just the answer to persistent storage, but to storage portability, which has been a problem from the beginning of the cloud. Data gravity is the idea that data holds apps in place. Tools like Ionir reduce the gravity of data. A lot. They can’t get rid of data gravity, because some platforms charge for data storage, data ingress and egress, etc. but that becomes a TCO operations problem rather than a near-insurmountable tech problem.

    But just as interesting is this: What if the slow but steady increase in core Internet speeds will allow us to access data from anywhere? We could do that now, but performant applications struggle if you force them to leave cloud A and go to your data center or to cloud B to get a dataset. Yet we use APIs that do just that on a regular basis. What are Google Maps and Google Places if not simply big old honking databases? And yet we call out to them from just about every platform.

    So, it’s worth trying out and seeing if the idea of data gravity is maybe an obsolete idea for you. For some applications that really do need super-fast response times, the concept is still relevant. But for many applications? Where your app sits in relation to data may not be as important as you think. Write or grab an API wrapper for a critical dataset and throw a simple test app up on a different platform. See what you get. If it performs well, consider putting apps where it makes sense and using a wrapper API with secrets to grant access to data from any validated app.

    Some of you are doing this already. Not many, but it’s out there. Excellent job; keep rolling!

    Everyone else is kicking rear and taking names, with the world still online every day because of your collective work. So keep rocking it, but consider if you really need the application close to the data these days and, if you do, how data mobility tools might facilitate moving data to the app—something most of us would not have considered for large datasets before the last year or so.

  • Neo4j Will Use Funds to Expand Managed Cloud Database

    Neo4j Will Use Funds to Expand Managed Cloud Database

    In July 2021, Neo4j announced a large round of fundraising—its largest to date. Philip Rathle talked with Alan Shimel about how the capital will be used to further Neo4j’s go-to-market strategy and what that means for DevOps professionals. The video of the conversation is below, followed by a transcript.

     

    Announcer: This is Digital Anarchist.

    Alan Shimel: Hey, everyone, welcome to another Tech Strong TV segment. My guest right now is Philip Rathle. Philip is with Neo4j, VP Product. Hey, Philip, welcome to Tech Strong TV.

    Philip Rathle: Thanks, Alan. Pleasure to be here.

    Alan Shimel: Absolutely. Philip, we’ve interviewed folks from Neo4j before, but I don’t know if everyone in our audience is familiar with the company. So, why don’t we start there? Why don’t you give us a little company background, maybe a little bit of your own background as well?

    Philip Rathle: Sure. Let me start with the second and tie it in. The thing that really attracted me to this company when I joined a bit over nine years ago was the idea that the most valuable thing in data, in my opinion, has always been the model and what’s built into the model. It’s how things are related, and what this company was doing was leaning into that some more at a time where the data space—and you look at everything that happened in NoSQL was about really simplifying the data model so that you could do this very, very high level volume and scale, but very simple data problems, store and retrieve. And in order to do that, you can lighten up on certain of the more technically complex requirements like, you don’t need acid if you’re not doing complex relationships and store and retrieve of shopping cart data at a massive scale.

    And so, what Neo4j’s been doing for more than a decade now is solving the problem of how to understand context and causality in your data through looking at how things are related. So, if you look at my relationship with—I don’t know, let’s look at an insurance scenario—with my car, other people who drive my car, other vehicles I might own, other things that I might own like a house and then line on that, and a business and lines on that and then bringing that together, suddenly, you’re into these complex kinds of enterprise data scenarios that have always been a challenge, persistently. And I lived through a lot of those as a consultant trying to fix those, kind of on a one-off basis.

    And so, what Neo4j did was something that, in the database space at the time was a bit crazy is build an engine ground up that assumed that you could do random I/O really well. Because if I’m going to understand relationships, a query can go any which way through the data, potentially very, very deep if I’m in a 20-level deep supply chain, for instance, following an item or a big payment network and looking for fraud detection, the fraud patterns or money launderings so that I can catch those. Those are all about patterns in the data, so Neo4j is a company that’s built a database from the ground up, looking to help users understand the relationships and the data so that they could understand context, causality, second order, third order, fourth order effects more deeply.

    Alan Shimel: Got it, got it. What about a little of your own background?

    Philip Rathle: I did consulting for the first half or third of my career, and a lot of that culminated in ending up as the person between the business and the IT problem. I really liked understanding what problems were being solved and then using new technology and seeing what we could bring to bear to solve the business problems.

    And I ended up working a lot with data and databases and a lot of large systems. I started at Accenture back when it was Anderson Consulting, then went into kind of a boutique company that worked on high end, large database deployments called Tanning Technology. And there, I realized I was doing—I later realized this was a step towards product management or I guess today you would call it product management. And ultimately, co-founded a startup, tried that for a few years around master data management, customer data integration; wrong market, wrong time—that’s maybe a story for another day. And then got into Embarcadero Technologies, which makes database tools.

    Alan Shimel: Mm-hmm.

    Philip Rathle: So, I worked there for about seven years, ended up heading up the database product portfolio, about a dozen tools for managing databases as well as a database interface database. Then learned about Neo4j, so, all of the cool stuff that was happening in data about 10 years ago, and thought, “I wanna be a part of this, kind of ground up, creating new ways of managing data.” And yeah, the rest is history. I’ve been heading up Product Management since then. I was employee number 30, and we’re at 550 growing way past 600 this year.

    Alan Shimel: Excellent. Very cool. So, I think we’ve laid a lot of the ground work, here. Phil, recently Neo4j announced a rather large raise. Why don’t you give us—a capital raise—why don’t you give us some information on that?

    Philip Rathle: Yeah, we did. So, as a company, you know, those of you who, probably every listener knows how it works with startups, right? You raise money, you put that money into primarily R&D, improving the product, and other areas of the business, and then you grow and, you know, at some point, there’s some kind of exit for us. We’re looking to be a big, independent company, and for that, databases are highly capital intensive, very complex technologies. It’s not enough to have just the database, you need tooling around it, especially when it’s a new kind of model.

    So, we just made a massive fundraise. We had already raised over $100,000,000.00 over the course of the years. This single fundraising event is $325,000,000.00, it’s the biggest fundraise ever for any database company in history. So, we’re pretty excited about it.

    Alan Shimel: For good reason—congratulations. You know, I’ve been involved in companies that have raised money, significant money over the years—that’s a lot of money, man. [Laughter] That’s a really big raise.

    Now, of course, you know, our audience says, “Whew, another big number.” We’re hearing a lot of big numbers lately. What does it mean for them? What does it mean for them, what does it mean for Neo4j to have that kinda capital? What’s the payoff, here?

    Philip Rathle: Yeah, I mean, it’s pretty much product, product, product with a little bit of a dash of regional expansion. So, on the product side, Neo4j recently launched—so, late 2019, we launched a self-serve version of the database as a service. So, it starts at $65.00 a month, credit card swipe. You can get a fully managed Neo4j database in the cloud and turn it on, it runs. You don’t need to worry about managing the database. And we launched an enterprise tier last year. Right now, it runs on, you can choose between GCP, AWS, of course, Azure and others are coming, and Work in Progress. 

    So, one area is to really keep improving the cloud product since the preferred way of consuming databases these days has flipped to cloud very, very fast over—you know, whereas just in the last couple years. Another one is graph data science. So, last—a bit over a year ago, we entered the, we expanded what Neo4j can do, which typically has been a developer oriented database with database drivers and more for transactional operational applications. 

    What we’ve been seeing over the last few years is that a lot of the kind of next step in machine learning and data science and AI is getting information about the connections in the data and in the network into your machine learning algorithm. And that whole art and science is graph data science, and we have a graph data science for a Neo4j offering, which is a graph data science library, together with a database, together with Bloom, which is a visualization and exploration tool. And that offering has really taken off in the last year, both in terms of demand, customers, and also, it’s become really mature in a short time. But there’s still a long way to go; it’s a brand-new space. 

    So, those are the two primary areas, and then I’ll add the third is just continuing to mature the database, continuing to make it scale, handling larger and larger amounts of data. By the way, as part of our announcement at our annual conference, we also a trillion-relationship graph.

    Alan Shimel: Wow.

    Philip Rathle: Just showing how, across 1,000 machines like—actually, trying to spin up this many machines on AWS, Amazon couldn’t provision the machines fast enough. [Laughter] We’ve never seen this kind of error.

    Alan Shimel: Right.

    Philip Rathle: But there’s no end to performance, scalability, security, more features, more richness, et cetera. So, those are the main areas, and like I said, a dash of regional expansion, really growing beyond—our primary markets have been U.S. and Europe with some customers in other parts of the world, but this will help us expand more in other parts of the world.

    Alan Shimel: Excellent. You know, I think one lesson that I learned during my career, and I think it’d be wise to share with the audience is—you know, there’s a latency time between the time you raise money and the time that money finds its way into products that are released, right?

    Philip Rathle: Yeah, that’s very true.

    Alan Shimel: Because, okay, now we’ve raised money, we gotta go hire people. We have the idea of what we wanna do, right? But, you know, we gotta flesh out those plans, we gotta get the people power, the technology process in place.

    When do you think we’ll see some of these great things that you’re talking about that, you know, this money raise has enabled actually come out to market?

    Philip Rathle: That’s a great question. So, prior to the fundraise, we were close to maxed out in hiring. And when I say close to maxed out, there’s a certain rate at which you can efficiently hire, and then beyond that, you kind of need to have people on their own off in the background, but that can’t contribute to the code base, because it’s not even structured to be able to have more than a certain number of people contribute. So, we’re—and that’s less true with new areas that we’re growing out. So, we’ve seen this one coming now for a few months, and so, we—that, plus just early in COVID, like, last April, we actually froze hiring for a bit and said we don’t know where things are going and we don’t wanna be in a position where we have to lay people off, that would just be horrible. Let’s just pause hiring.

    And then our business kept, you know, fortunately, doing well. And so, we started to wrap up, but I’d say—so, two answers, maybe. One answer is, you’re already starting to see the results of a really aggressive ramp in product and engineering. My team is doubling this year, and it’s close to doubled already. The engineering team is growing by about 50 percent. But what this also means is, we can keep hiring and we can keep—and whenever we have new products, new areas, we can form entirely new teams there. 

    So, I’d say the answer is now, but it also means that, on a steady state, continued basis, you’re gonna continue to see some really significant product releases from quarter to quarter, year to year.

    Alan Shimel: I would imagine. I would imagine—and beyond, with raising that kinda money, beyond. Anyway, Philip, we’re running out of time. You know what, did we mention the website?

    Philip Rathle: The website is Neo4j.com.

    Alan Shimel: That’s N-E-O—

    Philip Rathle: The number 4 and the letter J.

    Alan Shimel: Letter J—dot com.

    Philip Rathle: And you’ve got all kinds of info, resources for getting started. We’re huge on open source, we’re huge on education. And I should mention that our vocation has been not just making our own company successful—first of all, it’s about making users successful and making an impact in the world. But then not just making an impact in the world by making our product succeed, but making the category succeed. Because this is something that our founder penciled out on a napkin once upon a time. 

    So, we see ourselves in some ways as helping to grow the graph database space and just, you know, we celebrate when other companies go launch graph databases, because that means it’s good for the world. We’ve done a lot of work in the standards area and so on to make it more mature and better. So, you’ll see an open source version, there’s a free sandbox you can get started with, there’s a free version of our cloud product. Or, both on the developer side, on the data science side, lots of docs, good community out there, a Discord group, questions, QA on ________, you name it, it’s all out there. We’d welcome any and all of you listeners to check it out.

    Alan Shimel: Cool. Alright, we’re out. Philip Rathle, Neo4j—thanks for joining us on Tech Strong TV here today. Congratulations to you and the whole team on an outstanding raise, and now, let’s see you put it to work.

    Philip Rathle: Thank you.

    Alan Shimel: Alrighty. Be well. This is Alan. We’re gonna take a break here on Tech Strong TV. We’ll be right back with our next guest.

    [End of Audio]