devopsy.kinsta.cloud

Tag: developer

  • KubeCon 2023: CTO.ai’s Developer Control Plane

    KubeCon 2023: CTO.ai’s Developer Control Plane

    Speaker 1: This is Techstrong TV.

    Alan Shimel: Hey, everyone, we’re back. We’re here in Chicago at KubeKon. We’re wrapping up our day two coverage, our last guest for day two. We’ll be back tomorrow, though. But let me introduce you to Kyle Campbell, right?

    Kyle Campbell: That’s right.

    Alan Shimel: Kyle is the founder of cto.ai and we’re going to find out about cto.ai and a little bit about what they’re doing here at KubeKon. But before we do that, Kyle, first of all, welcome.

    Kyle Campbell: Thank you.

    Alan Shimel: Second of all, let’s hear a little bit about Kyle.

    Kyle Campbell: Yeah, thanks.

    Alan Shimel: Tell us your kind of journey.

    Kyle Campbell: Yeah, well, first of all, great to meet you in person. I’ve talked in the past. Yeah, so my journey, I’ve told this story before, a little unconventional. I grew up in Nova Scotia in Canada, a small town. Been on the internet since the age of eight and just had no interest in the formal past. So I’ve been building software from the early days of the .com boom. I may not look at it, but I got some of the scar tissue. But I was self-taught software engineer, so open source was the key to my success and good developer tools.

    Alan Shimel: Sure.

    Kyle Campbell: And I came up through the cloud and open source era and then started founding developer platforms in 2014. The first company I built was a developer platform, the real estate space. Zillow acquired it in about eight months, which was interesting.

    Alan Shimel: Very.

    Kyle Campbell: And then I bootstrapped a DevOps agency quite successfully and started to find that there was a lot of opportunity for next generation developer platforms, which led me to cto.ai.

    Alan Shimel: Excellent, man. What a great story too. You still up in Nova Scotia?

    Kyle Campbell: I’m not. I moved to the other side of the country. I live in British Columbia now.

    Alan Shimel: Good for you.

    Kyle Campbell: Love the outdoors. Spend a lot of time with my son camping, fishing, and trying to get outdoors and just enjoy the beautiful-

    Alan Shimel: Loving it. Yeah, no, it’s beautiful. I mean, not that Nova Scotia’s not beautiful. It’s brutal in the winter, but it’s a beautiful country, part of the country.

    Let’s talk cto.ai now. So look, I’ve founded multiple companies myself. Every founder I’ve ever interviewed or spoken with in 30 years, they don’t just say wake up and say, “Oh, I feel like founding a company today.” There’s kind of like Richard Dreyfuss in Close Encounters of the Third Kind, right? There’s something driving you like, “I got to do this. This needs to get done.” What was driving you that needed to get done here with cto.ai?

    Kyle Campbell: Yeah. I mean, as I described my past, a lot of my journey was self-taught and stand on the shoulders of giants. And really important thing for me was developer experience and ease of use and tooling early on in my career because that enabled me to really drive my competencies as a developer and keep up with these people that had computer science degrees and master’s and all these things, right?

    Alan Shimel: Certainly.

    Kyle Campbell: So when I thought about that and I also thought about where the future’s going and my passion for platform engineering, it became pretty obvious to me that what was missing in the market was a set of tools that were designed to embrace the future of developer experience and how AI can impact that from code to cloud.

    So in 2016 or so, I started experimenting with the idea, and through 2020, I decided to formally launch cto.ai as a developer control plane is what we call it. The way to think about it is it’s a modern CI/CD platform where teams can compose their workflow across build, deployment, test, and preview. And as they work with these tools, they can interact with them in the CLI Slack, GitHub, talk to these workflows through sort of a prompt based model. And we collect telemetry data that allows them to then measure that against DORA metrics, which is-

    Alan Shimel: It’s standard.

    Kyle Campbell: … yeah, it’s a standard and I think, frankly, an empirical way to measure the developer experience. We don’t try to measure developer productivity. We think that’s the wrong question to ask. The question is how do I measure the experience the developer has in the same way that I would measure the experience my users have with my product? And if what we’re doing is building platforms to enable developer experience, we need a natively integrated way to measure that. And we also think that that measurement is a way to pave a future between how the developers experience those tools and how AI can support the developer experience and enhance developers with, frankly, superpowers over time.

    Alan Shimel: So I was recently at a couple of user conferences, big DevOps companies, and it was interesting. Three conferences all use the same metric, which is that developers only spend about 30% of their time developing.

    Kyle Campbell: State of DevOps Report 2018, I think?

    Alan Shimel: But it was in there and it’s been kind of verified in several others. What do you think about that? Is it good thing, bad thing, something to be proud of, something we should be working on? What’s the deal?

    Kyle Campbell: Yeah. Well, I think a pretty practical point of view in that as a software engineer, my goal is to release features that customers love and drive business value. And if I can do that at a higher velocity while maintaining quality, I mean, I think that that is a way that I, as an individual, can contribute to the success of the business and the team.

    But I think what often happens is we get really obsessed about the individual developer. And I think any developer who looks at the idea that they only spend 30% of their time adding value isn’t going to like that very much. And I think we need to reframe the conversation away from developer productivity towards developer enablement and look at the tooling and the process that the developer experience promotes and how developers go from code to cloud and ask ourselves as a business leader, as a CTO who we work with a lot, is the tooling and the workflow that we’re supporting our team with and that developer experience, what they’re experiencing, is that driving results?

    And I think DORA metrics is an empirical way to measure that, obviously, through anecdotes and surveys, but what we’ve really innovated on is creating a vertical integration with the tool chain that allows you to measure that in real time and then iterate on the tools quickly. And some of the teams can really own that themselves. And what the businesses should be doing is zooming out and asking themselves, “Yeah, how do we maximize the amount of time developers spend without individualizing that consideration?” There’s an innerloop and there’s an outerloop. And what DORA is really good for is measuring the outerloop of how we work in aggregate of teams. We need next generation platforms that make this data and this context more accessible to business leaders.

    Alan Shimel: I don’t disagree with anything you said there, so we’re there with that. What about have you guys released any news or anything? What do you got going on around the show?

    Kyle Campbell: Yeah, so for the last while, we’ve been focused on really building a rock solid workflow and tooling to basically set the premise of that next generation composable CI/CD vision. We’ve fed that data into DORA Metrics. You can now report on DORA metrics in real time, save those reports, share them with whoever one-on-ones, retros, so kind of up to you. And we’re starting to experiment with some data science, but something that’s really exciting that we just kind of put into production last week. We haven’t really officially announced, so I guess this is a first is we’ve released our first kind of AI-

    Alan Shimel: Heard it here first.

    Kyle Campbell: … yeah, heard it here first. Our first AI feature set, which is AI code review. And what we’re doing is we’re letting teams start with these sort of base models for how they can leverage AI in the code review process to get developers’ feedback earlier. But then, we want them to be able to do is enhance that model to really customize it to their preferences on their own kind of hardware, to customize those models to take into consideration their own specialized requirements and obviously protect proprietary information that’s so core to their beliefs. So we launched that into production, announcing it here first.

    Alan Shimel: Really?

    Kyle Campbell: And we’re looking for early adopters who can kind of test it out and help us stress test it and figure out how do we further enhance the context of that model with some of the information that is derived in these DORA metrics as well.

    Alan Shimel: So here, we’re talking to this camera. Developers out there. You’re looking for them to kick this around a little bit. Kick the tires.

    Kyle Campbell: That’s right. Yeah.

    Alan Shimel: How do they do it?

    Kyle Campbell: You’re a developer and you’re interested in how AI can support your CICD workflow, specifically code review, come over to cto.ai, sign up, send a message. I’d love to talk to you personally, software development yourself, and love to hear what you think, what you think are the limitations and hesitations, and more importantly, how do you think we can enhance this with other use cases? Because I think there’s a whole exciting set of use cases for AI and software delivery that is going to help software developers.

    Alan Shimel: Absolutely.

    Kyle Campbell: We embrace-

    Alan Shimel: I had this conversation in an earlier interview today. Look, for people who aren’t tech, you show them an AI interface and some of the kind of parlor tricks it could do and they think it’s magical.

    Kyle Campbell: Yes. Arthur C. Clark.

    Alan Shimel: Right.

    Kyle Campbell: Yeah.

    Alan Shimel: Now, we know it’s not magic. It’s just putting one word in front of the other or one… but there are some uses that AI really shine certain. So it’s not meant to replace Google Search nor is it meant to replace software developers.

    Kyle Campbell: No.

    Alan Shimel: But it can empower software developers to be 10x more effective. And one of the ways it could teach you to code, right? You could put code in there and say, “Explain it to me.” It could review code, right? So either it’s writing code or it’s reviewing code. You shouldn’t have it doing both probably, but…

    Kyle Campbell: It’s true.

    Alan Shimel: Right. But if it’s using it for review, what a great use of the technology.

    Kyle Campbell: Yeah. We’ve seen that the number one thing that slows down time to market in feature sets for developers is waiting for code review. And while I’m not sure that it’ll be the same thing as you and I pair programming and getting into the nitty-gritty details, if we can make sure that every developer can get a code review instantly just by adding a label and then that we can tailor the quality of that code review with more context that is supported by how the team thinks about quality. I think this is an important example of how we can accelerate the needs of developers and the developer experience with AI that still leaves all the control in the hands of the developer to decide, “Is this rational? Is this a hallucination? Is this the appropriate time?” But the feedback loop that we can get from this is, I think, a really great beachhead and why we chose to start there.

    Alan Shimel: Absolutely. And we’ll see. I don’t think we scratched the surface yet on how we’re going to use AI, but this is certainly a real definitive first step that I see us going with, right?

    Kyle Campbell: I think so. And I think the call to action for other software vendors out there is to think about reinstrumenting what they’re offering to prepare for this because I think there’s a necessary hesitance around AI. I think rightly so. But if we can think about how we can take control of it, how softwares can take control of it. Software engineers are perfectly suited to be some of the first stewards of these problems. Just like we see AI and code generation, I think the next natural step is to support it within code review. And then we’ll start looking at how smarter deployments, smarter rollbacks, and obviously, operations can be enhanced with AI. And this place is a perfect example of people trying to solve that problem.

    Alan Shimel: Absolutely. Absolutely. And here’s the other thing. I’ve been in technology a long time. You don’t put these genies back in their bottles.

    Kyle Campbell: Right.

    Alan Shimel: Once they’re out, they’re out. And so like it or not, they’re here. And I think you could… if you can’t beat them, join them.

    Kyle Campbell: You can’t stop Moore’s Law. I mean, this the pathway that the world is on.

    Alan Shimel: No, you’re not. You’re just not going to stop it. That’s the way it is. It’s great stuff, man. I love the attitude. I love having you on here. Cto.ai. Again, if you’re looking to help Kyle out, kick the tires on this AI code review product they got coming on, it’s fantastic. So let’s check it out.

    Kyle Campbell: All right. Thank you so much.

    Alan Shimel: All right. Hey, Kyle, enjoy the rest of your KubeKon. cto.ai wrapping up day two in Chicago at KubeKon. We’re going to be here tomorrow morning. I think we’re on at 10. We’re only here till two tomorrow because it wraps early tomorrow. But we’ll be back with a full load. We hope you’ve enjoyed today. On Demand is available at Techstrong TV. And so if you missed anything, do check it out. This is Alan Shimel. We’re out.

  • Daytona Unfurls Developer Environment for On-Premises IT

    Daytona Unfurls Developer Environment for On-Premises IT

    Daytona has launched a development environment for on-premises IT environments that provides all the benefits of a cloud-based alternative.

    Ivan Burazin, CEO of Daytona, said the company’s namesake alternative to platforms such as GitHub Codespaces enables DevOps teams to retain control and better secure the intellectual property created by developers while simultaneously making them more productive.

    The Daytona platform enables developers to instantly start coding by accessing a development environment based on a set of preferred tools and frameworks, he added. Platform engineering teams looking to scale DevOps workflows can then take advantage of fine-grained access controls and advanced security features across developer environments spanning hybrid IT environments, said Burazin.

    He added that the ability to centrally manage development environments is designed to appeal to organizations attempting to improve developer productivity in highly regulated industries that are now allowed to use cloud-based alternatives. This is becoming a more pressing issue because as regulations become more stringent, organizations are trying to move source code off developers’ individual laptops that are more easily breached than a server environment, noted Burazin. The challenge is striking the right balance between enabling flexibility and centralizing the management of developer environments, he added.

    Of course, many of those organizations could opt to construct their own internal developer portal (IDP) to accomplish the same goal, but in most cases, that would prove expensive to both construct and maintain, noted Burazin.

    The same core team that created Codeanywhere, a cloud-based integrated development environment (IDE), is also behind the development of Daytona. The issue that organizations previously encountered when attempting to shift development to the cloud is that, in addition to regulations that forbid it, developers could not easily customize their environments. The Daytona platform is designed to enable more flexibility without sacrificing control, said Burazin.

    Arguably, organizations are more focused on developer productivity today than at any time in recent memory. Developers typically spend as much as 70% of their time maintaining environments versus writing code, so there is a clear need to reduce the level of toil. Reducing that toil doesn’t guarantee developers will spend more time writing code, but it does create the opportunity. DevOps teams are, as a consequence, being asked to eliminate as many of the bottlenecks that conspire to reduce developer productivity as possible in furtherance of that overall effort.

    Developer productivity issues are not going to be resolved overnight. Developers will not always be immediately inclined to give up control unless it’s conclusively proven that a platform engineering team will be responsive to their needs. The cultural challenge is finding that fine line between reducing friction while still giving the developer the sense that they can add a tool to a workflow that enables them to innovate.

    One way or another, change is coming to DevOps workflows. Each organization will need to determine how quickly that change will occur, but in the absence of any meaningful change, it’s clear they won’t be able to compete against rivals capable of producing higher-quality code faster.

  • Nixpkgs: How to Build Better, More Open Software

    Nixpkgs: How to Build Better, More Open Software

    The world we know today depends in fundamental ways on open source software. As providers of this software, package managers serve their respective ecosystems from their centralized, opaque hubs in ways that are sometimes difficult to reconcile with the open source nature of the software they’re serving. I want to talk about the extent to which that’s true and discuss Nixpkgs—the largest open source package repo in existence today—in this context.

    People sometimes focus on Nix the language or Nix the package manager, but it’s the Nixpkgs package collection (pronounced ‘nix packages’) that is core to the success of Nix. But it’s more than that: We can also understand Nixpkgs as a powerful tool to ensure the continued growth, transparency and resilience of open source software, first and foremost because it democratizes the process of curating this software and its build instructions. It is the living compendium of the efforts of thousands of contributors who both stand on the shoulders of giants (the software they’re packaging) and provide the ladder for others to climb further.

    A Purely Functional Grimoire

    The Nix package manager is different from other package managers in several ways, but one key aspect is that Nixpkgs contains only recipes to build packages, not the packages themselves. This is a fundamental idea to Nix: If we build a package reproducibly, we can separate the specification of a build from its built output. When we build a “recipe” for a package, we determine the output hash of that build–without actually building it. This means you don’t need a cache of build artifacts: Recipes are enough to rebuild your software world.

    Nixpkgs is just that: The big fat recipe book of open source. Anybody can view it, fork it, patch it and use Nix to inspect the largest set of free software recipes in the world. Anybody can create a cache that uses those build recipes (or your own!) to publish packages: The default cache is the official one run by the NixOS Foundation at cache.nixos.org.

    Derivation and Substitution

    So what happens, in practice, when I download a package from the Nix cache? When you query for a package, the CLI creates the specification (we call it a derivation) of the package to be built. This derivation includes a hash: The summary identifier of all source code, patches, build flags and hashes of the dependencies themselves.

    With that output hash in hand, Nix the package manager just asks whether that path exists in its cache. If it does, Nix fetches the contents of the package you asked for; if it doesn’t, Nix will build the package locally. We call this move substitution: Creating a derivation from a Nixpkgs expression gives you all the information you need about the output of the build, so the cache can substitute a binary with that output hash if we have it. Of course, if you end up having to build the package locally, it will result in the expected output hash anyway.

    Democratizing Package Curation

    The term ‘lingua franca’ refers to the dialect spoken by sailors across the Mediterranean in previous centuries, and it was the shared language with which one could communicate and do business from Morocco to Greece. Today we package software for independent islands, each with its own rules, languages and cultures and, until Nix, we had not found a way to “speak” across the many ports-of-call of our software archipelago. What this allowed for is packages being available on more platforms, which drew more developers in, which resulted in Nixpkgs becoming more complete, and so on in a positive feedback loop.

    Nix is a cross-platform, cross-architecture package manager. If we had to design the next-generation package manager, this would probably be at the top of the requirements list, yet few package managers enable this today. Each of the teams at RedHat or Debian maintains their own packaging format and package cache and has a fixed team of vetted contributors to keep packages secure and up-to-date. This results in few who effectively contribute to packaging open source projects and do so for one of many platforms.

    An open book of package recipes written in this lingua franca results in another surprising effect: Nixpkgs maintains more packages than anybody with an order of magnitude fewer maintainers [1]. It makes sense: Since it’s a common language, people can review the packaging code for packages they don’t necessarily know intimately, which in turn leads to contributors and reviewers cycling through updates faster and providing more up-to-date packages, which reinforces the previous feedback loop from earlier.

    Even if the rest of the Nix ecosystem stalled out, we’d still have this incredible resource in Nixpkgs to curate and share open source software; flox sources its packages exclusively from Nixpkgs for this (and many other) reasons. As of February 2023, there were around 6,000 contributors on the Nixpkgs repository and growing.

    (via (repology.org)[https://repology.org/repositories/graphs])

    Stay Unstable, my Friends

    Nixpkgs embodies a new model for package management and one that opens it up to more people to contribute. Providing cross-platform and cross-language packages is obviously attractive, but it’s the fact that it provides a lingua franca for software packaging that makes it a pillar of open source moving forward. Because Nixpkgs unifies a broader swath of software and makes it easy for anybody to contribute new packages and fix existing ones, it continues to grow and serve the open source community: A positive feedback loop propelling the existence of a more free and open world of software.

    References:

    Nixos.org

    https://reproducible-builds.org/

    https://wiki.debian.org/SecureApt

    https://rpm-packaging-guide.github.io/

  • A Supportive Engineering Culture is Key for Recruiting and Productivity

    A Supportive Engineering Culture is Key for Recruiting and Productivity

    Skilled engineers are in high demand, as their ability to build new products quickly is extremely valuable for businesses competing in a tight market. Strong engineering talent allows businesses to increase productivity and meet customer demand faster, which is essential for long-term success. However, in the current economic context, companies are being forced to reduce spending while trying to achieve the same results. This means engineers have fewer resources available to achieve their goals–increasing the pressure on their role. At the same time, engineering talent remains a valuable asset; in the UK, for example, just under half (49%) of engineering businesses struggle to find workers with a sufficient skillset for the job.

    So, businesses must find a way to attract and retain talented developers while ensuring they feel supported through challenging times. A vital step to achieving this is creating a workplace culture that promotes developer satisfaction. Below, I explore how businesses can create a positive working environment for engineers, make transparent decisions and ensure their teams feel supported – to create a culture where developers want to stay and supercharge business performance.

    The Key Pillars to Creating a Positive Culture

    Leaders of all stripes are tasked with ensuring that their employees are supported, engaged and motivated to do their best work. However, the route to achieving these outcomes can sometimes seem vague and elusive.

    Research shows that there are several key pillars for creating a strong organizational culture. The first is psychological safety. Developers must have the freedom to take risks without the fear of being criticized. Software engineering is an industry in which professionals must be creative enough to be able to create something out of nothing, so it’s important that companies ensure developers have the space they need to experiment and try new approaches.

    Secondly, dependability is essential. Developers are more likely to produce high-quality work in a timely manner if their employer believes in them and their abilities. Developers often face immense pressure to ship functional code on a daily basis; feeling trusted can give them the confidence they need to take on more responsibility and drive projects forward. Alongside this, it’s the leader’s role to ensure developers have a very clear and structured mission. They must know exactly the goals they’re aiming to achieve, the part they’re playing in reaching these goals and the plan for execution.

    Finally, ensuring developers understand the meaning of their work and are fully aware of the impact it has is an important pillar for cultural success. Sometimes, software engineers spend a huge amount of time on small fixes and updates that may seem insignificant. Business leaders must make efforts to communicate the importance of developers’ work and the value they bring to the business. If developers feel valued and rewarded for their difficult work, they will be more motivated to commit themselves to their mission and stay at the organization.

    When engineering teams are happy and supported, they are also more likely to speak highly of their organization and encourage other developers to join the team, helping the company build a strong pipeline of engineering talent.

    Making Transparent Decisions Using Concrete Data

    Transparency is another crucial element to ensure engineering teams are happy and motivated to do their best work. Developers can’t do their jobs effectively if they don’t have insight into the workflows on their teams or don’t have context into decisions that impact their ability to perform effectively.

    With tools such as software delivery intelligence platforms, developers can see the positive impact of their contributions and how they’re helping the business succeed. These tools also enable engineering managers to rely on concrete data to celebrate developer progress on previously invisible work and show how their work contributes to wider business success. This helps developers gain more autonomy to achieve the company’s goals, and eventually create better business outcomes.

    For example, Manulife adopted an engineering analytics tool to create an engineering culture that is based on trust and transparency. The concrete data generated is used to analyze engineering processes and highlight talent that may have remained hidden without a data-driven approach. Data also helps managers decide who within the team deserves to be promoted based on precise insights into everyone’s contribution.

    Use Data to Help Overcome Complex Challenges

    The mission of developers is constantly evolving, with new technologies, programming languages and methodologies changing very quickly and making the job more complex over time. It is up to engineering leaders to help their teams remove potential barriers that are blocking them from being successful in their roles.

    With data-driven insights, engineering leaders can better support and uplift their teams. Data about engineering workflows allows leaders to understand exactly where bottlenecks exist and remove them quickly before slowing their projects down. This means that employees can learn what worked–or didn’t work–from one project to the next, as well as develop the necessary skills to remove roadblocks to future success.

    Timelines and targets for projects set by business leaders can also then be tailored to each person’s abilities and based on concrete data that show where they can boost efficiency to ensure everyone produces their best quality work and remains engaged and motivated.

    In the current technology industry, employee culture is one big factor determining where developers wish to work. In a context where engineering talent is in high demand, it’s the job of every organization to create a stellar work culture to attract and retain talent.

  • How Good Developers Become Good Engineering Managers

    How Good Developers Become Good Engineering Managers

    Being a good engineering manager means that, to some degree, you have a natural affinity for the work involved. And if you’re a good developer, you likely already have some of those innate skills and attributes.

    But simply having the skills doesn’t mean you’ll enjoy being an engineering manager. For example, the nature of the role is that you’re no longer the one writing the code. I’ve known great architects who became great managers, but they ended up going back to being individual contributors (ICs) because they missed working with nitty-gritty coding details and the joy that brought them.

    It helps to take a realistic inventory of your skillset and preferred working style before making the transition. There are certain competencies you should have completely dialed in, and other tips to know that will help your transition.

    Basic Skills of a Good Engineering Manager

    If you’re a good engineer, you likely already have the following foundational skills that also make a good engineering manager. With these in your toolbox, you’ll have a better chance at success.

    1. Attention to detail. The buck stops with the engineering manager. You need to know when to push and ask questions and how to set outcomes for your team’s accountability. If you work loosely as an IC and need reminders about your own work, you probably won’t do well as a manager.
    2. Amazing personal planning skills. As an engineering manager, you’ll be planning an entire team’s work. If you’re terrible at planning your own work, maybe the engineering manager role isn’t for you.
    3. Speak intelligently about broad technical topics. If you’re a good developer, this shouldn’t be a concern. You’ll need this knowledge to advocate for your team and justify their work to upper management.
    4. Team player attitude. Even as an IC, you should have a collaborative, team-oriented mindset. It goes without saying that you’ll need this when leading others.
    5. Interest in managing others. This might seem obvious, but if you don’t have a natural interest in managing others and in the people side of engineering, you’ll be unhappy as an engineering manager.

    Making the Transition From Developer to Engineering Manager

    If you can do all of the above and are ready to make the transition from developer to engineering manager, here’s what to be aware of as you navigate that process. These tips aren’t meant to deter or discourage you but rather provide a helpful dose of reality.

    Satisfaction in your work takes new forms. If you’re an excellent developer, you know the value you bring and likely get personal satisfaction out of the contributions you make. As a manager, it’s harder to see the direct value you provide. Shift your mindset and realize that now, you’re the one enabling developers to do that rewarding work, and that’s the value you provide. You’ll learn to take enjoyment and fulfillment from your team’s accomplishments.

    People take up a lot of your time. What most frequently catches new engineering managers off guard are the people-related aspects of the role. They’re not prepared for how much time managing people takes. I’ve seen it over and over again where a new manager thinks they can still set aside time to code, but you really have to prioritize learning the craft of management.

    Remember to communicate upward. In the absence of information, it’s human nature to assume the worst. When I was at BitBucket, I assumed that because I had always done a good job, the leaders would think my team was crushing it. The opposite was true; leadership thought everything was on fire.

    I started sending updates every two weeks to various levels of management, and that’s when I learned the power of upward communication. I realized I needed to sell my team’s work, contributions and challenges. Learn what higher-level leaders are looking for and then give them that information regularly.

    Context-switching is part of the job. As an IC, constant context switches kill your day. But being an engineering manager means dealing with endless context-switching. Going to multiple meetings is baked into the job, and you’ll still have to find ways to block off time to get other work done.

    Shield your team. You also need to be prepared to help protect your team from context-switching. You might learn information that’s important to your team during your many meetings, but not all of that needs to be communicated with them. Filter out the junk so it doesn’t rain down on them and they can maintain their flow.

    Being an engineering manager can be a rewarding role that has a considerable impact on an organization. The more prepared you are for the realities of the role and its differences from being a developer, the more you’ll enjoy it and the more successful you’ll be.

  • Loosely Couple Your Schema and Keep Data Separate

    Loosely Couple Your Schema and Keep Data Separate

    In software development, one of the primary goals is to write clean, maintainable and extensible code. To achieve this, developers often strive to design systems that are loosely coupled and separate concerns into smaller, more manageable components. Loose coupling refers to the practice of designing components that are not tightly dependent on one another, allowing them to be modified or replaced without affecting the rest of the system. Separating data into distinct layers, such as a database layer, further contributes to this modularity by allowing developers to focus on writing business logic without worrying about data management. Together, loose coupling and data separation provide a solid foundation for building scalable and adaptable systems that enable developers to concentrate on writing high-quality business logic. This approach also lends itself to platform engineering best practices, a popular methodology that is being implemented or considered by developer teams today.

    Loose coupling is an important concept in microservices architecture because it helps to achieve the goal of creating modular, scalable and easily maintainable software systems. In a microservices architecture, individual services are designed to be independent, self-contained and loosely coupled, meaning they can be changed, updated or replaced without affecting other services or the system as a whole.

    There are several benefits to loose coupling in microservices:

    • Scalability: Microservices architecture allows scaling of individual services independently based on their needs, and with loose coupling, the interdependencies between services are minimized, allowing for better scalability.

    • Maintainability: With loose coupling, it’s easier to make changes to individual services without disrupting the system as a whole, reducing the risk of introducing bugs and making maintenance tasks easier to perform.

    • Flexibility: With a loosely coupled architecture, services can be developed, tested and deployed independently, enabling teams to work in parallel and make changes to the system without impacting each other.

    Platform engineering plays a crucial role in ensuring that microservices are loosely coupled. A well-designed platform should provide a standardized set of APIs, protocols and interfaces for service communication, as well as tools and frameworks for building, testing, deploying and managing services. This enables service teams to focus on developing their services rather than worrying about the underlying infrastructure and communication protocols.

    Moreover, platform engineering can also ensure that there is a standardized and centralized approach to security, monitoring and logging, which are critical aspects of microservices architecture. By providing a consistent and standardized approach to these areas, platform engineering can help ensure that individual services remain loosely coupled and can evolve independently, without negatively affecting the system as a whole.

    It’s All About Keeping the Schema Separate

    In platform engineering, the concept of keeping the schema separate typically refers to separating the schema of a database or data store from the application code that interacts with it. This separation allows for greater flexibility and scalability in the platform, as changes to the schema can be made independently of the application logic. By decoupling the schema from the application code, developers can avoid introducing breaking changes to the platform, and can more easily evolve and scale the platform over time.

    Separating the schema also allows for different parts of the platform to interact with the same data store using their own specialized schemas, without interfering with each other. This is particularly important in large and complex systems, where different teams may be working on different parts of the platform concurrently. By keeping the schema separate, teams can work independently and avoid stepping on each other’s toes, while still interacting with the same underlying data store.

    In summary, it’s all about the schema – don’t share databases and tables. The separation of data is fundamentally driven by that desire for loose coupling. Microservices should be as independent as possible from each other from design to development to production. Why? Efficiency. Disparate teams can change data structures and will not affect others versus the more traditional approach where we do share and it’s more cumbersome and involves more coordination between teams.

    Keeping schema separate in platform engineering is important for several reasons:

    ● Scalability: Each microservice owns its private schema, which enables the platform as a whole to scale more effectively. As the platform grows and evolves, it may become necessary to change the data schema without necessarily changing the underlying code. Separating the schema makes it easier to make these changes without disrupting the rest of the platform.

    ● Maintenance: Separating schema from code makes maintenance easier. It allows developers to make changes to the schema without needing to change the code, which reduces the risk of introducing bugs or errors.

    ● Collaboration: Separating schema from code makes it easier for multiple developers to work on the platform at the same time. When developers work on the code and schema separately, they can work independently without interfering with each other’s work.

    ● Flexibility: Separating schema from code provides greater flexibility for integrating with other systems or applications. It enables the platform to adapt to changing requirements or to interface with different data sources more easily.

    Separating Schema From Code is an Essential Practice

    By hiding data from service components from other service components, It provides guardrails and “just works.” While it does provide a form of coupling, it’s through messaging (typically asynchronous). Platform engineering encourages good design patterns from the start versus having to be more aware of these things. It’s a lot easier for especially newer developers that are not familiar with some of these architectural patterns and can get themselves into trouble.

    Overall, separating schema from code is an essential practice in platform engineering that provides scalability, maintenance, collaboration and flexibility benefits. It allows developers to build more robust and flexible platforms that can adapt to changing business requirements and technical environments.

  • How Application Developer Turnover is Shaping Security

    How Application Developer Turnover is Shaping Security

    Observing the developer job market for the past two years reveals a surprising truth that has a profound impact on technology organizations. In the “new normal” of hybrid work, life moves fast and there’s no sign of slowing down. Since 2021, we’ve witnessed all-time record demand for software engineering developers. The emergence of privately held large and medium technology startups, fueled by a fear of missing out (FOMO) driven venture capital market, created vast growth opportunities for software engineer developers in particular. From developing and deploying new applications to scaling your business through multi-cloud environments, cloud and cloud-native companies are everywhere. Not only are cloud and cloud-native companies receiving enormous funding rounds, but worldwide spending on public cloud services is forecasted to grow 20.7% to a total of $591.8 billion in 2023.

    This dramatic cloud growth is primarily driven by the warp speed developers are operating. Thanks to developers, every company is now a digital company and every industry is reaping the benefits of a cloud presence. A retail organization can now build and deploy an application for the holiday shopping season within hours, and workforces can access applications from their kitchen, local coffee shop, or halfway across the world.

    However, as with any new, widely adopted technology, cybercriminals look to target and exploit organizations in their early cloud days as they begin to grasp and understand the technology.

    It was reported earlier this year that the recent exposure of roughly a terabyte of Pentagon emails was likely due to a cloud configuration error. Cloud attacks are on the rise, and there is a major disconnect about who is responsible for keeping the organization safe. Developers who are told to build with speed don’t want their applications slowed down or user experience to be altered by security protocols. Still, accelerating application velocity has security teams caught between remaining responsive to application teams and securing an increasingly complex cloud environment. A major problem is bubbling up, and if security and developers teams cannot come together, the result can leave organizations vulnerable.

    Developers and Turnover

    In a recent survey, 75% of respondents reported a higher-than-usual turnover rate in DevOps. Similar to the well-documented cybersecurity talent shortage, multiple workplace stressors appear to be taking a toll on developers. The same survey reported that 38% of enterprises deploy code to production or release to end users every day, with 17% deploying multiple times a day. To put this into perspective, just two decades ago, developers were shipping limited updates one to two times a year.

    Given the pressure to deliver new code every day, combined with security teams’ pressure to ensure that code is secure, it’s no surprise that we’re witnessing higher-than-normal developer attrition. Instead of pointing fingers at developers for unsecure code, security teams need to create tools and training that do not impede their number one priority: Shipping new code. We’re in a developer-friendly job market and organizations cannot afford to lose top talent because they are overburdening them with security tasks. Developers will find greener pastures.

    Shifting Left

    By now, everyone in the industry has heard of shifting security left—the practice of moving security to the earliest possible point in the development process to ensure code is secure and limit timely and costly remediation. In practice, this means bringing security to where the developers operate so vulnerabilities and misconfigurations are caught early when they are easy to fix.

    This approach has brought about a deeper level of engagement between application developers and security teams, especially in the design phase. However, while developers might now be embracing tools and processes that help them code secure applications, there’s still a limit to how much shift left responsibility they can and want to handle.

    Path Forward

    Shifting security left is certainly a recommended starting point, but simply equipping developers with DevSecOps tools isn’t a winning approach. Developers must be trained and have the desire to effectively use the tools. Like the old saying, give a man a fish, and you feed him for a day. Teach a man to fish, and you feed him for a lifetime. The same holds when shifting security left.

    To ensure developers and security teams understand their security responsibilities within the organization, consider implementing the following:

    1. Define a clear shift left strategy with developers and security teams
    2. Provision the necessary security tools that effectively protect the organization but don’t create roadblocks for developers. Additionally, more tools don’t necessarily mean more security, so be strategic.
    3. Train developers to be more mindful of cybersecurity issues. Most developers haven’t had formal cybersecurity training, so it’s up to organizations to make sure that particular skills gap is closed

    DevSecOps is as much about changing culture and approaches as it is about acquiring new tools. Developers will, unfortunately, come and go, but it should never be because security teams are overburdening them.

  • 5 Key Performance Metrics to Track in 2023

    5 Key Performance Metrics to Track in 2023

    It’s never a bad time to adopt healthy new habits for the mind and body, and for developers, it’s always a good time to consider new best practices at work. As 2023 rolls on, I’ve been thinking about some key performance metrics that help to ensure critical applications and websites perform optimally throughout the year—especially during unpredictable spikes.

    To see why this is important, just look at the problems Ticketmaster had in November 2022 when Taylor Swift’s concert tickets went on sale. Fans of the pop idol (including me!) swamped the Ticketmaster site and brought the system to its knees, resulting in frustrated customers and a wave of bad publicity for the company.

    These usage spikes and the problems they can produce are not uncommon. The holiday sales period, starting with Black Friday and Cyber Monday, is notorious for technical issues that can cost retailers millions due to lost sales and other problems. And it’s not just retail. A few years ago, the Internal Revenue Service (IRS) had to extend its tax deadline after a rush of last-minute filers knocked its systems offline.

    Fortunately, developers can mitigate these problems with the right planning and practices in place. There are key performance metrics that developer teams should monitor, especially in times of peak traffic, to avoid application crashes, long wait times—and burnout for the teams who have to fix these problems when they arise.

    Here are the five performance metrics developers should be tracking in 2023—especially on their busiest days of the year.

    Web Vitals

    The largest contentful paint, or LCP, measures the load speed of a webpage. Having a fast LCP is a clear sign that a customer’s experience is optimal as they toggle between pages. In tandem, the cumulative layout shift, or CLS, scores how a user is experiencing unexpected layout shifts. This plays out on busy shopping days: Typically, these pages have ad and sales notifications that affect a brand’s main page layout, potentially causing shoppers to experience unexpected shifts. This can reduce their ability to shop and becomes a hindrance if a developer cannot quickly access and address the issue.

    Error Count Vs. Error Rate

    Error counts typically increase alongside site traffic as more people flock to a website—that’s a natural occurrence. The error rate is more telling because it reveals if a greater proportion of users are experiencing issues with your application or website. Developers who see an increase in this metric should investigate and take action.

    Mobile Monitoring

    More and more consumers are using their mobile devices to shop and manage all aspects of their lives. For example, mobile accounted for 45% of sales during the holiday shopping period in 2022 from October to December. Mobile performance metrics are critical to understanding what is happening on your users’ devices during these busy times. Vitals like frozen and slow frames or cold and warm starts when an app opens provide visibility into how fast views are loading for your users. As became obvious with the Taylor Swift incident, a slow shopping experience can lead to an outcry and a lot of upset customers.

    Outside of vitals related to mobile performance, it’s important to monitor mobile changes through application release health. Developers can see this in the rate of crash-free sessions and crash-free users as traffic to an application increases—which helps to identify abnormalities in the overall health of the app.

    Slow Database and HTTP Ops

    During sustained traffic spikes, database queries and HTTP requests that take too long to execute harm the user experience. Hitting a slow checkout process, or seeing the dreaded spinning beach ball after adding a product to the cart, confuses shoppers. They don’t know if they should refresh the page, and often they will abandon a purchase completely.

    What’s more, if a developer is working on a backend framework, slow database and HTTP ops could also be a sign of an N+1 query problem. At that point, an application is making database queries in a loop and causing performance issues for the purchaser.

    User Misery

    This is a homegrown metric we use at Sentry; tracking user misery helps developers understand a customer’s experience with an application. It is a ratio of unique users who have experienced load times at 4x a configured threshold in a Sentry project and thus serves as a proxy for customer frustration. With this user misery score, our developers can see which transactions have the highest negative impact on users and prioritize fixing them.

    Providing a smooth end-user experience is table stakes for businesses throughout the year, but especially during high-pressure, culturally-driven spending moments. In any of these scenarios, developers behind the scenes often work long and odd hours to address performance issues and ensure the most seamless experience. You can’t prepare for everything that can possibly go wrong, but you can be equipped with the right tools and metrics to quickly identify and remediate any problems that occur.

  • Don’t Make Big Tech’s Mistakes: Build Leaner IT Teams Instead

    Don’t Make Big Tech’s Mistakes: Build Leaner IT Teams Instead

    One of the biggest narratives in the U.S. concerning the current wave of tech layoffs is the fact that ‘traditional’ sectors have a prime opportunity to snap up new tech teams and IT talent. This has always been a significant challenge for sectors such as banks, insurance and manufacturing that compete with big tech for talent. In 2022, a majority of CEOs (71%) identified skills and labor shortages to be that year’s biggest business disrupter.

    And they were right to be worried. According to Salesforce, three in four workers do not have the digital skills needed by businesses to prepare for the future of work. The temptation in this environment for many traditional organizations is to enter the labor market to build larger tech teams to help drive their modernization programs. However, this may be a mistake.

    Building large, expensive, bloated tech and IT teams is not always the answer. It’s critical that traditional companies learn this lesson from big tech.

    Instead, traditional organizations should consider other avenues to drive their digital transformation ambitions with their lean resources. What does this look like?

    What is the Real Challenge you are Trying to Solve?

    Since COVID-19 hit, U.S. companies have been forced to reevaluate how they work, with remote working becoming commonplace–even for companies who couldn’t imagine themselves working remotely.

    And the challenges have grown as U.S. businesses seek growth to recover lost gains from the last few years. Evanta’s 2022 CIO Leadership Perspectives study found that CIOs rate data and analytics as their second top priority within the IT function. It is clear that business leaders see advancing organizational use of data, analytics and automation as key to reaching enterprise objectives.

    For many organizations, the biggest challenge they face is the unprecedented amount of data generated. Critically, more than 80% of all new data is unstructured and messy. This makes it difficult for data-driven companies to accurately review, gather, analyze and rely on what is mission-critical information.

    Traditional industries such as health care, banking, insurance or the public sector, have even larger concerns because data is increasingly critical to how they build value and deliver services for customers. So, it is important for their IT teams to create a single source of truth and data governance framework across their organizations to ensure the right people and senior leaders have the most accurate information to make smarter decisions.

    For instance, in the manufacturing industry, no-code database platforms can be leveraged to structure daily operational data and visualize insights through digital dashboards for quality assurance, continuous improvement and health safety.

    Additionally, at Baserow, we work with treasury customers at some of the biggest international banks that use no-code platforms to keep track of local legislation around the world by centralizing information in real-time databases available to relevant teams, including compliance.

    Modern organizations depend on their IT teams to accomplish a lot of their goals. So, it is important to provide them with the tools to focus on more complex development projects. No-code tools free your non-technical teams to address their pressing needs themselves without having to wait for budget or resource availability in the IT department.

    Have you Also Considered Upskilling Your Current Team?

    We know from the Salesforce survey that the vast majority of U.S. workers lack the digital skills to support their organizations’ ambitions and needs. This suggests that there is a wider issue within most companies at large. Do businesses need to prioritize upskilling their current team instead?

    A U.S. Gallup report found that 72% of workers are overwhelmingly interested in pursuing additional digital training to prepare them for an increasingly digitilized workplace. Is there room for no-code technologies to supplement the digital capabilities within companies?

    Enterprises are already expanding the responsibilities of application development outside their IT teams as well as increasing the productivity of their existing IT resources by automating and abstracting repetitive developer tasks. Gartner predicted that by 2025, 70% of apps will employ low-code/no-code technology. This trend aligns neatly with the business needs, such as the development of organization-specific applications and tools, because every business’s requirements are different.

    Furthermore, no-code/low-code technology enables companies to drive innovation and shorten the idea-to-market cycle times. The technology’s pre-built components make it easier for end-users and workers to intuitively assemble the templates that solve their complex needs.

    In Conclusion

    U.S. companies that need to fill the IT skills gap should consider how technology can be leveraged to boost productivity and accelerate the speed of innovation while accepting that new technical resources will continue to be expensive and scarce.

    These organizations are better off taking the lead in the democratization of IT within their teams. Only then can they keep up with the speed and agility required in today´s competitive environment.