devopsy.kinsta.cloud

Tag: developers

  • SlashData Report: Developers Influence Tool Selection

    SlashData Report: Developers Influence Tool Selection

    SlashData has just released its “State of The Developer Nation Report for Q3 2017.” Here at Heavybit, a program that helps developer and infrastructure startups take their product to market, we look forward to SlashData’s reports that allow us to keep up to date on the shifts in the developer landscape.

    In 2013, Stephen O’Grady published, “The New Kingmakers: How Developers Conquered the World,” sending shockwaves through the ecosystem about just how important individual developers are when it comes to making technology purchasing decisions in companies.

    SlashData provides compelling data that backs up O’Grady’s book, showing how influential developers are within an organization and how targeting individual developers at organizations is a viable strategy for enterprise sales. The research only includes professional developers in organizations who buy tools or components and exclude lone wolves—developers who work on their own and where a purchase decision is purely personal.

    Key report highlights Include:

    More than 87 percent of developers with a leadership function, no matter how small, as well as two-thirds of front-line coders, are somehow involved in purchasing decisions.

    For developers that are not in a top leadership position, between half and two-thirds are in a position to make recommendations or influence decisions. In particular team leaders (i.e., senior developers) are big influencers (68 percent).

    Thirty-nine percent of developers acquire tools for their personal use.

    The ability for developers to pick and use their own tools shows that they have the power to make their own decisions and a budget to spend (even if it is small). It’s through this personal use that a tool can get introduced to an organization and is the first step in evangelizing the tool to the greater developer community.

    Up to a third of mid-level developer leaders (team leaders and product managers) hold the pen when writing specs for tools (32 percent and 31 percent, respectively), or even get the final word on which tool to adopt (30 percent and 27 percent).

    This fact is even more proof that individual developers at all levels matter. “Even when in a majority of cases developers still need management to sign off on purchases, the days are over when an inadequate tool could be pushed from the top down because it appeared attractive to CFO’s or top brass,” the report states.

    Developer influence, when it comes to making technology decisions in companies, continues to grow. To harness this shift, companies need to focus on creating tools that developers will come to love and evangelize on your behalf within their organization.  

    — Malia Powers

  • OutSystems: Low-Code Apps Require Sound DevOps Processes

    OutSystems: Low-Code Apps Require Sound DevOps Processes

    As more IT organizations embrace low-code application development platforms, many of them are discovering that accelerating the rate at which they develop applications can be too much of a good thing.

    António Barros, product manager for OutSystems, says most of the organizations that have adopted the company’s namesake low-code platform are just now coming up to speed on automated testing. Barros estimates that only 20 percent of the organizations using the OutSystems platform are also employing automated testing tools. The majority of OutSystems customers continue to rely on manual testing even though it’s apparent low-code platforms will produce applications much faster than manual testing can keep pace with. OutSystems has identified five stage of DevOps maturity, he notes, but none of its customers has achieved that level of DevOps maturity.

    To help them get a little further down the DevOps path, OutSystems, at its recent NextStep 2017 conference, announced an extension of its partnership with Indigo to provide tighter integration with the OutSystems platform. In addition, OutSystems plans to make available digital transformation playbooks, including one for DevOps, beginning in 2018. Outsystems has already integrated its platform with the Jenkins automated server to provide continuous integration and continuous development (CI/CD) capabilities.

    Barros says it’s not only critical to engage in continuous testing, but the application needs to be designed to make it easier to isolate user interface, application programming interface (API) and unit testing. Otherwise, automating the testing process becomes more difficult than it should be, says Barros.

    Developers for years have resisted low-code platforms, which automatically generate much of the code that needs to be written for an application. Some fear the platforms can’t build applications that scale. Others have simply decided they prefer to write code themselves. But these days, organizations are under more pressure than ever to deliver applications faster, which has led many organizations to rely more on low-code platforms such as OutSystems.

    It’s not clear just how popular low-code platforms will become. OutSystems revealed at its conference it has signed up 122 new customers in the last quarter alone. On average, it now takes organizations eight weeks to build and deploy an application on the OutSystems platform, with new release cycle updates occurring every 48 hours, says OutSystems CEO Paulo Rosado.

    Michael Facemire, an industry analyst with Forrester Research, told conference attendees that adoption of low-code platforms is now a matter of economics. The starting salary for a developer in the San Francisco Bay area is $145,000, says Facemire. As the salary of a developer with a few years of experience now reaches more than $200,000 in the Bay area, companies in that geography will have no other option but to employ low-code platforms, he says.

    Obviously, developers might be less expensive in other regions. But Facemire notes that no matter what the region, developers are in short supply. That need for speed, he adds, is also why organizations are rapidly adopting containers and serverless computing framework. The challenge IT organizations will face is finding a way to put quality control processes in place to don’t compromise either the speed of the development process or the quality of the end user experience, he says.

    By the same token, however, no one really appreciates code that arrives late, no matter how great it might be.

    — Mike Vizard

  • DevOps: Time for Developers to Dig In

    DevOps: Time for Developers to Dig In

     Speaking to developers and project managers at the Gartner Catalyst Show, Jenkins World and Atlassian Summit revealed some insights about how most IT specialist perceive DevOps. Yes, I had a busy end of summer schedule attending events and talking to a wide range of developers and project managers about DevOps.

    two dogs digging together in one hole in the ground

    What I learned is that most people have not yet figured out what is meant by DevOps. What most people told me is that they are doing DevOps explaining they have a CI/CD pipeline. When questioned further about how they perceived DevOps, they believed DevOps was about creating a CI workflow that developers and sometime-testing teams could execute to automate builds, deploys and testing. One of the most ironic of these conversations was project managers making the claim that their CI server performed builds and deploys.

    Well folks, DevOps is not about a CI/CD pipeline. And contrary to popular beliefs, your CI server does not perform builds and deploys any more than it performs automated testing. Your CI/CD pipeline orchestrates build and deploy processes in the same way that it executes an external tool for test automation. The ugly truth is that your CI/CD pipeline is executing one-off build and deploy scripts, the same scripts used in your waterfall practice, but just triggered by your CI/CD process. These scripts cannot easily adapt across the continuous delivery pipeline. Testing and production environments are configured differently. The build and deploy scripts must be tweaked, maintained and debugged in the same way they were during waterfall practices. Once ready, they are then called by your CI/CD workflows. DevOps is different.

    DevOps demands that your developers and operation teams share and collaborate from the earliest phases of a project. In essence, this requires that more of the operational duties be shifted left and directly onto the shoulders of developers. This is precisely were the responsibility and control should be. Operation teams need knowledge and transparency so they can communicate infrastructure requirements early and often to the developers. Eventually what we will see is a decline in the number of operational resources. A “release engineer” who managed production deployments said, “If we define our DevOps process correctly, my job will not exist in the future. Instead, automation will replace the manual Ops scripts, allowing Dev teams to do 70 percent of my work.” And that, my friend, is real DevOps.

    Developers, Dig In

    So it is time for developers to really dig in to DevOps. What I mean is developers must begin walking away from automating most of the processes by a one-off script and begin incorporating tools that perform DevOps tasks from continuous deployment, security scanning and performance monitoring. And different platforms need different tools. Project teams will need the power to select the tools that work for them. The days of “legacy” purchases of a single tool for the enterprise is long gone. High-dollar operational tools are high-risk and incur technical debt when developers choose to use a different solution. DevOps means giving development teams more control and purchasing authority for automation tools that can be managed at a project level, not an enterprise level. These tools must support a process where the developers work on the declarations followed by testing and production consuming the process with a good amount of transparency. Scripts cannot achieve this, even when triggered by CI.

    Our future is becoming more and more complex. Most companies I spoke to still managed over 70 percent of monolithic applications. The writing on the wall is in the 30 percent. With microservices and containers, operational tasks are going to become more complex, not less. The sooner developers can become the core owners of operational tasks, the easier it will be to adopt a microservices architecture. Managing microservices will not be the job for the operations team. It will need to be done by the developers who have the intimate knowledge of their application.

    To mature from CI/CD to DevOps is going to require a close look at the engine under your CI/CD hood. When developers and project managers begin to really dig in and see how the engine is running, it will be clear that achieving DevOps is a bit down the road. And there is urgency in this process. Leaner and better ways of writing and releasing code is upon us. Most of us are just not ready to toss out some of the old habits from waterfall practices to get there.

    Some departing thoughts:

    • Acknowledge that CI/CD is a good place to be, it is just not DevOps.
    • Look carefully at your underlying processes that are called by CI/CD. How well do they support your production environment?
    • Can your CI/CD workflow at Dev support a production deploy? If not take the time to understand why.
    • Are your developers working late to get a code update to production?
    • Do you have a clear continuous feedback loop that shows where a code update is running, and on what exact endpoints?
    • Do your developers know what infrastructure updates are going on in production today?
    • How do your developers communicate infrastructure updates to production teams? Can they do it alone?

    The time is now for self-examination and a clear honest appraisal of just how automated your core operational tasks are. If they are built upon one-off scripts, you should assume you have some work ahead to really begin moving to a DevOps practice that can be declared by developers, consumed by all runtime environments and is transparent to all.

    And remember, microservices are chasing you.

    About the Author / Tracy Ragan

    Tracy Ragan is COO and Co-Founder, DeployHub. She has extensive experience in the development and implementation of business applications. It was during her consulting experiences that Ms. Ragan recognized the lack of build and release management procedures for the distributed platform that had long been considered standard on the mainframe and UNIX.  In the four years leading to the creation of OpenMake Software she worked with development teams in implementing a team-centric standardized build to release process. Connect with her on LinkedIn.

  • Software Security is an Engineering Problem

    Software Security is an Engineering Problem

    We live in a world where those who sell products are generally held accountable for defects. Automobile manufacturers, pharmaceutical companies and toy makers are among the businesses that have been handed fines and tarnished reputations for distributing items that were later deemed ineffective or dangerous.

    One notable exception to this “consumer first” ethos is the traditional shrink-wrapped software world, where responsibility for product problems is hazy. The vendors that develop and sell software and the customers who purchase and install that software are in a constant battle to shift security liability. The vendors strive as best they can to reduce and remove product vulnerabilities and offer secure configurations while customers deploy, configure and operate the software in very different environments, with very different configurations, which can produce very different security outcomes. Because no customer can (or should) assume receipt of 100 percent vulnerability-free software, they must always take on the primary burden of protecting the environments where that software lives to offset the risks—which typically falls to the customer’s IT and security teams.

    Although there are excellent arguments for both sides, the current situation leaves both parties wanting—and needing—a higher level of assurance in both the development and runtime settings.

    Software Delivery Has Changed (Along with Security Responsibility)

    The good news for software customers is that software delivery is rapidly moving away from an on-premises install approach to a cloud-based subscription model where applications are delivered via the internet. Upfront costs for hardware and software, as well as the expense of personnel to install and manage that software, are replaced with business-friendly, predictable, subscription-based operational costs. Software is easy to launch, upgrades are uncomplicated and scaling to meet changing business needs is simple.

    Customer expectations are high for software-as-a-service (SaaS) solutions. Purchasers demand an elevated level of innovation and want new features more often. They assume built-in support of global information security and privacy regulations such as the European Union’s General Data Protection Regulation (GDPR) and various U.S. regulations, including the Health Insurance Portability and Accountability Act of 1996 (HIPAA) and PCI Digital Security Standard (PCI DSS). SaaS customers also expect the vendor to take on larger responsibility for any potential software vulnerabilities that might negatively impact the customer’s data and operation.

    The new reality is that deploying software in the cloud significantly changes the paradigm of how software security should be treated at the vendor level. This means companies now have to think critically about security as an engineering problem as the balance of the liability burden shifts focus to the cloud provider’s engineering practices to ensure constant delivery of a secure and trusted cloud service, rather than just secure software.

    Now is the Time to Rethink Software Security

    Cloud-based application delivery brings together under a single company umbrella the ones that make the software and the ones who run the software. Informing software development of runtime behavior and vice versa can have significant advantages in enhancing the overall security of the software by offering a closed-loop system. In such a perspective, the rapid pace of software releases work in favor of enhancing security.

    Shifting security to an all-around engineering responsibility requires the elimination of traditional work silos. Geared-for-speed developers must receive precise information early, allowing them to correct security problems during the initial coding process. DevOps team members also must assume security responsibilities. And security engineers who understand the attacker mindset, security risks and compliance requirements must provide guidance at all points in the software development process.

    Change is never easy in any organization, so transforming disconnected teams into one in which security is an across-the-board responsibility is difficult. Managers must be careful to not alienate team members who already have full plates. And although we’re starting to see some shifts in the educational model to one in which professionals are taught early on how to seamlessly integrate coding and security, we have a long way to go.

    While we wait, the most effective way to overcome the chronic problem of heavily siloed teams is to provide the entire software engineering team with a means to quickly identify issues between software development, its deployment and runtime across all the components. These tools must give users the ability to:

    • Scan and provide real-time analysis that integrates source code analysis and its runtime behavior. After all, they are two sides of the same coin;
    • Obtain accurate and full visibility into the security impact of changes from build-to-build;
    • Ease the audit analysis process;
    • Reduce the number of false positives; and
    • Allow developers, DevOps, and Security to collaborate to enhance the security of the environment.

    The goal? An ability to fix software mistakes, weaknesses and vulnerabilities at the beginning of the development process instead of post-deployment, when liability, revenue and reputational risks are the greatest.

    Consolidating today’s siloed security practices into an integrated engineering effort is the only way to ensure the appropriate level of security at the appropriate time in today’s fast-moving cloud-based software industry. This sort of shift in thinking and processes will give customers less security worries and better products. Development teams will become more effective and efficient. DevOps will have real-time visibility into how each release impacts the infrastructure. And Security will know how each release impacts both security and compliance requirements. SaaS companies will reap the financial benefits that come with strengthened industry reputations for rapid innovation capabilities and high levels of software security. It’s a win-win-win for everyone.

    About the Authors

     Manish Gupta is founder and CEO of ShiftLeft Inc. He is a 20-year veteran of the security industry and has led product development, management and strategy at FireEye, Cisco Systems, Intel, McAfee and Redback Networks.

     

     

    Craig Rosen is vice president and CISO at AppDynamics. He has held leadership roles for more than two decades in the security industry, including at FireEye, Pacific Gas & Electric, TDI and CGI.

  • JAMstack Radio Podcast: Benefits of an API-First CMS

    JAMstack Radio Podcast: Benefits of an API-First CMS

    Welcome to JAMstack Radio, a podcast that discusses the JAMstack, a new way to build fast and secure apps or websites. JAMstack Radio is hosted by Brian Douglas, Developer & Advocate at Netlify. In this episode, Brian is joined by Stefan Judis and Rouven Wessling to discuss Contentful, a developer friendly API-first CMS.

    They explain the benefits of decoupling your website’s content from its presentation, including better cross-platform support. With Contentful, you are completely free to write your site in the language you want, while the service takes care of the CMS backend, storing and versioning your data, and backing everything up.

    This podcast is brought to you by Heavybit, a program that helps developer product companies take their product to market.

    [soundcloud url=”https://api.soundcloud.com/tracks/329732086″ params=”color=#ff5500&auto_play=false&hide_related=false&show_comments=true&show_user=true&show_reposts=false&visual=false” width=”100%” height=”175″ iframe=”true” /]

    Brian Douglas: So, welcome to another installment of JAMstack Radio. On the line, we’ve got Stefan Judis.

    Stefan Judis: Yeah, I am Stefan, I am a developer evangelist at Contentful, based in Berlin. And I’m really excited to be here.

    Brian: Cool, and then we also have Rouven Wessling.

    Rouven Wessling: That’s almost correct. Yeah, I’m also a developer evangelist, but I’m also an ecosystem developer, so I don’t only just get to talk about Contentful. I actually build part of the product, mainly unveiling the PHP SDK. I also have my fingers in some of the other ecosystem projects we do—stereo saturators, family integrations.

    Brian: Cool. The audience definitely knows what Contentful is, because you guys have come up in conversation at least three times in previous episodes. But for those who maybe just found this randomly, do you want to explain what Contentful is?

    Stefan: I can take it. So Contentful is a headless CMS. So what this mean is that we offer an API, where you can retrieve your content, and on the other hand, we offer a web app where editors can edit their content. That’s basically a headless CMS, and we’re API first.

    Brian: Yeah, we’re big fans. At JAMstack, we love separating concerns. And the reason why Contentful keeps coming up is because we keep talking about front end solutions, to talk to APIs, and since you guys are API first, you fit right in the middle of the jam. It’s like a match made in heaven, really, for anybody who wants to have something super fast, but not worrying about collecting and maintaining their own infrastructure.

    Rouven: There was this amazing quote from a user we had just recently.

    Contentful is a CMS where the user gets the full CMS they want, but the developer doesn’t have to deal with us at all. 

    It’s just an API, it’s like any other API you would ever use, it’s just the same. You don’t have to think like a CMS, like it worked just like a Drupal, like a Adobe experience manager of all the gruesome things out there. It’s just an API. And that’s really what we want to be, we just want to be the API for your content.

    Brian: Cool. Yeah, I really like the approach too, because I literally have recently got into WordPress, for some things we need to test for work, and I’ve never done WordPress development before.

    Rouven: How was it?

    Brian: Not very good. Just the getting up and running, and getting you environment set up to host your own WordPress setup, it’s kind of tedious. Especially if you just want to manage a small amount of content to test out a small side project. It doesn’t seem realistic to reach for WordPress right out of the gate, which is why I love the solution of Contentful. You call it a CMS, so do you have an admin interface where users can actually interface and add data too?

    Stefan: Yeah, so we’ve got the Contentful app. This is the place where content managers or editors set up their content model first, so it’s completely flexible, you’re adding there, define what your content types are, you then connect items. And then you’ve got the actual editor interface, and this is where then the data editing happens.

    Brian: Okay, cool. And then, the data itself, we’re talking JSON, I assume?

    Stefan: We’re talking JSON.

    Brian: Yeah, which is very cool. So my last job, we built a growth hacking project to get people to find our site eventually. And because it was a growth hacking project, I worked really closely with the marketing team. So what I did is I built a CMS with Google Sheets. I built a quick scraper for Google Drive, to actually take that data and add it to a Postgres database to then serve to my front-rend react app.

    So I spent about, maybe 1 1/2 weeks researching it, and I spent about another a few days to actually to implement it, and I walked away feeling very proud of myself. And the very first day I showed it to everybody in the team as like, “hey this is what I built, now the marketing team can go into the Google Drive and add data and it shows up on our website that I built for them, this works perfectly.”

    And the first question that came up with the engineering team was, Why didn’t you use Contentful? And I was like, “Content-who? I’ve never heard of these guys, I don’t understand.” And then as soon as I got back to my desk after feeling very defeated, I actually Slacked and was, “Ah yeah, if you told me about this like two weeks ago, I would have used it, but I already built this, and I don’t want to do any more work, so I’m like kind of upset.”

    Rouven: But what you built sounds amazing, because you got the core thing right. You decoupled your presentation.

    Brian: Yeah.

    Rouven: The react app is about how you displayed the content from the part where you store the content. So you can switch out either end without having to redo the entire project. Now if you worked with WordPress, and you have a WordPress theme, and you have all the content in a WordPress database, you’re stuck, it’s never going to get out of there, unless you put in a lot of manual work.

    If you want to add a mobile app, or Alexa Skill, or whatever else you want to build for your content, and you want to reuse the same content, good luck in WordPress. 

    That’s not going to fly. I mean, it’s possible, don’t get me wrong, it certainly is possible, but it’s not going to be fun, by any means. And just grasping that and actually doing that right, is where most people who build websites today actually fail. Because they are thinking in a web-centered world, and everything they design is made for the web, and for nothing else.

    Brian: That’s very true.

    Stefan: You need a mental shift though. With WordPress, you have the holy WYSIWYG editor, where people start messing around with all the content, but when you want to go cross-platform, then you’re stuck. When you have markup in your text blocks, there is no way getting out then again.

    Brian: Yeah, and especially if you have to, like you mention, handle multiple platforms, and now we have the Amazon Echo which has no actual visual interface, like it’s just reading data from strings. It makes a lot of sense to decouple your markup from your API so that way you can present that nicely.

    Stefan: And also you have much more choice of not just reusing content, but also how you have your editors think about content. Like my absolutely favorite feature of Contentful is the images API, where you can seriously resize and mitigate images on the server. And how many people implement CMSs with structured fields where you have to upload the same picture five times at different sizes for different viewport sizes.

    The editors hate it and the developers hate it but people still do it. But the much smarter solution is to resize images on the server and cache them in the CDN.

    And I’ve even seen people with Contentful implemented that way, because they are so stuck in the mindset that they have multiple image fields for different sizes, while it’s much much easier to just use the power of the platform, and just resize images on the server and be done with it.

    And to think more in abstract terms about your content, independently of the platform where you show it, is a mind shift people just have to make. And once you’re there, a whole world opens up to you of just reusing the content you have. The most amazing thing companies usually produce, is content, like this podcast, it’s content, and it’s actually really good content, cause people learn from it, people study it. Bringing the content to multiple places to be more accessible to people, is an incredibly important thing to do just to get it out there. And making that mind shift is so important.

    Brian: Reuven, at the top of the show you mentioned that you also, not only that you’re an evangelist, but you also implement some SDKs. Contentful, do they have SDKs for multiple platforms as you mentioned?

    Rouven: Yes we do. So I have the probably unpopular with this audience job of maintaining the PHP SDK. So if you’re coming from WordPress, I’m the guy who’s going to get you on Contentful. But we do have SDKs for obviously JavaScript, for all the mobile platforms, Swift, Objective C, Java for Android, Python, Ruby.

    All the common languages used on the web on mobile, we have covered. I think it’s right now eight in total, so whatever platform you’re on, we can probably get you hooked up on SDK so you don’t have to actually write raw API interactions ever.

    Brian: Cool. Yeah, that’s like the best thing about, one, separating your concerns, separating your markup, but also being able to use like, these SDKs for example, if I wanted to create a react app or ingar app, or something fancy. If I can just drop in a quick connection to that API and reuse that same data, that kind of unlocks a real world, so it makes sense how Contentful really is serving the space really nicely.

    Stefan: Yeah, you just have to drop in the JavaScript SDK and then you’re already flying.

    Brian: Yeah. Apologies if I broadside you guys with this question, but do you guys have search integrated in there as well? Like if I had a long list of content that’s being implemented, is there an easy way to pick certain items for certain situations?

    Rouven: We do have a full text search. It’s not Algolia-quality search, so if you want a really powerful full text search, use Algolia with us, it has actually quite a nice integration and there are some open-source projects to do it.

    Brian: Oh nice.

    Rouven: But if you have just basic search needs, we actually run off Postgres, we have I think like a dozen different search operators. Including like geographic searchers, range searchers, bounding box GU search, very fancy, but the most important thing is full text search across all the content users have to get you to the result. Whatever humans actually type in, not just computers do.

    But the powerful thing about Contentful is, we really just want to do one thing very well, and that is content and managing your content, and if you want another thing to do with your content, somebody else may do it better than us. So if you want really really good search for your website, or your app, or your Alexa Skill, or whatever else, use Algolia.

    Brian: Yeah.

    Rouven: Push your content from Contentful to Algolia. If you want AB test, use Optimizely.

    We want to really get away from the monolithic view of “your CMS does everything and everything lives in your CMS.

    But you use the most powerful, most advanced tool for whatever you want to do, and you just store your content with Contentful, and you integrate with all these tools. So you’re not locked into using whatever happens to integrate with Contentful, you just use whatever is the coolest thing right now.

    Brian: I’m glad you mentioned Algolia, because an episode that hasn’t been out yet, they’re actually on, I believe it’s the last episode, so the one before this will be about Algolia, which is great because all they need is JSON, which makes sense, if you guys have integration to pass your Contentful data straight to Algolia. It’s amazing how many APIs here work for each situation, and how perfect the JAMstack works.

    Stefan: You just have to connect pieces, right?

    Brian: Yeah.

    Stefan: We’re really reaching that state, and that’s quite nice.

    Rouven: It’s an amazing world we live in right now. I’ve been, the past couple of months, to a couple of API conferences. It sounds absurd on the surface, it kind of is to be honest, but like just discovering how many APIs are out there to do all the things, it’s a great world we live in.

    Brian: Yeah. Do you mind sharing what API conferences were you just at?

    Rouven: The last one I was at was actually APIdays Australia, in Sydney.

    Brian: Okay.

    Rouven: Before that it was APIdays Paris, Nordic APIs. So around the world, mostly US, Europe and Australia, there is a conference where it’s API implementers usually come together, exchanging best practices about how to best implement an API and document an API and get developers started with an API. How to make the first API call as easy as possible. This is incredibly important for us, because

    Entering your content through the web app, editors know how to do that.But integrating content through an API entry each occasion is something new for most developers. 

    So we just exchange knowledge there about actually getting started. And it’s amazing to see how many companies now open up their data. I met somebody from Lufthansa. Lufthansa, the big airline, now has an API, where you can carry flight information. I think it’s the first big airline to do it, and it’s amazing that these old like 19th century companies are starting to open up their own data for the world to use, for us developers to use.

    Stefan: Yeah, this is exactly what we wanted right? You mentioned the react use case. For example, I’m storing the information for my personal website also on Contentful. And just because I can and I have an API available, I built myself a little app that schedules all my travels. Because I do a little traveling, these are the events, and that wouldn’t be possible without an API, that’s really sweet.

    Brian: Yeah, it’s super nice. I really like the catering to, I don’t know if you want to call them non-technical users as well, but having a marketing team or an editing team that’s pushing out content while the rest of the app’s actually being built. That was one of the big use cases why I opted for building my own API-first CMS with Google Drive. It was because we were building a proof-of-concept for a project, and at the time we needed a lot of content for it.

    So we couldn’t wait for the app to be built while we were collecting content, so we started adding content to the Google Drive to eventually port it over manually. So I was like “hold on, we have the content already, let’s not double work,” so I ended up doing way too much work to get this thing to work. But if I’d used Contentful, we would have had all the content being populated as the app was being built, and then click a button and then it populates all our data. And I liked your use case for traveling too as well.

    Stefan: It works pretty well for me.

    Brian: Yeah. I love providing data. Every other episode I talk about this app, but I built an app to tell me whenever there’s a home baseball game. I don’t know if you guys have baseball in Germany.

    Rouven: Rarely. It’s not really, rumor is there is a team someplace in Germany, one, just one.

    Brian: One team? At least you’re keeping up with everybody else in the Western ecosphere. Yeah, so for the listeners who don’t know what baseball is, it’s basically where grown men stand around and watch one tiny white ball get thrown around for hours.

    Rouven: That’s so how it feels to me.

    Brian: Yeah, there’s beer involved, and hot dogs.

    Stefan: I went to a single game in my life, I was bored to death.

    Brian: Well, if you guys ever make it out here to the SF, I’ll bring you guys to a baseball game, and we’ll get drunk.

    Rouven: I think we have a deal here.

    Brian: All right.

    Stefan: I’m up for the drunk part, I’m not sure if baseball’s necessary for that.

    Brian: All right, well we’ll go next to the stadium to a bar and listen to it.

    Stefan: That’s a deal.

    Brian: So the app I built basically is just scraping ESPN data and making sure I know whenever there’s a home game for the Giants, and it just will ping me and tell me whenever not to go to the city, and avoid any sort of baseball traffic. So it sounds like I could use Contentful for that, just to collect all my data for me, and then be able to ping me with the notification, just a serverless function.

    Stefan: These kind of stories make me so excited, that we’re kind of developers and we’re able to build these kind of things. Sometimes I’m like “holy shit, this is amazing.”

    Brian: Yeah we’re at a really awesome time to be a developer, because a lot of the, not all the problems have been solved, but a lot of the hard problems have actually been solved for us, so we can just use our knowledge and collect all these APIs and collect all these use cases.

    Rouven: Just an API callable year. Google and Amazon too have been opening so many amazing APIs for natural language processing, AI context-based language meaning, image recognition, video recognition just opened by Google. I mean, it’s not open source, open source would be nicer, but just an API call away, and maybe three cents charge per API call is a breadth of knowledge, that at least I am incapable of developing by myself.

    Like I can’t build a natural language processor, that’s definitely not my level of skills. But I can just access one that’s really really good by making an API call. And that ability, as a developer, is amazing.

    Brian: Yeah. Even things like VR, and we talked a little bit about the Amazon Echo as well. People have already solved those problems, and you just drop in a JavaScript library to do VR on your phone, or language processing from your phone or other third-party device. It’s amazing and also scary. Because you know, everything’s super easy and attainable to do, so the possibilities are almost endless for the good and bad.

    Stefan: There’s always stuff to go, right, always.

    Brian: Cool. And sorry, Stefan, do you also do code at all, at Contentful?

    Stefan: I was developing many front-end stuff in JavaScript before and now anchor the developer relations team, which means I’m responsible for blogging and hanging around and chatting with people. Coding these days is side projects and try-outs.

    Brian: Okay, cool. Well I’m sure Contentful gives you access to nice stuff for your account so you can continue to build your scheduling applications.

    Stefan: I’m building all this stuff with Contentful now, I need an API, okay let’s do that.

    Brian: Yeah yeah. I feel the same way Netlify, like I’ve had three sites before I was a Netlify employee, and now I have like 50, just because I can. I don’t need to think about whether or not I need to pay for something. For one, if you’re a developer, Netlify’s pretty much free, but for two I get to try out every new feature that Netlify has, so I’m just rolling out always. If I think of something, it goes on Netlify. And hopefully I’ll do the same thing with Contentful, coming up pretty soon.

    Stefan: I tried Netlify last week the first time, and I was like, “That is nice.”

    Brian: Yeah.

    Stefan: It was really easy.

    Brian: Yeah, we try to make it easy.

    Stefan: HBB2 and CDN and yeah it was really good.

    Brian: Cool, I’m glad you liked it. So speaking of which, examples. I was trying to rack my brain, you guys have any good examples of use cases for Contentful working at companies?

    Rouven: All right, let me start. They’re not actually doing static sites, but it’s my favorite customer ever. They’re right up in London, it’s the British Museum, world-famous institution, very like old school dusty name. It’s the British Museum, they have the mummies.

    Brian: Could you do it in a British accent?

    Rouven: I can’t. You can tell my accent is horribly German and I lived in America for a year, and I still haven’t gotten over a German accent.

    Brian: Well, I don’t know if you watch any 1950s and ’60s movies, but all Germans are British in movies.

    Rouven: I haven’t heard that one. So before I left for the U.S. I was an exchange student they told me that in American TV Germans are either portrayed as Bavarians or as Nazis, so don’t be offended. So I always wanted to be the Bavarian. So the British Museum, dusty old mummy house, but they actually have an amazing digital agenda, they have a digital director.

    And Contentful is their default CMS now for all things. And they’ve done some amazing things, which is actually my favorite use case, because something nobody thought of, when they designed Contentful. Audio guides, like the things you rent for 5 pounds, and walk around the museum and they tell you about the exhibit.

    Brian: Yeah.

    Rouven: These are running off Contentful.

    Brian: Wow.

    Rouven: So all the audio files, the pictures, the text, they are stored in Contentful. The curators and the editorial team, they add to it throughout the day. So the British Museum is a very old building, with very very thick walls, which is harder for wifi, so they don’t have wifi throughout the building. So they just have wifi at the booth where they rent out these devices. And they use our sync API to sync down the entire content onto the device.

    And then you take the device, you go through the building, and you have all the content available to you, without actually connecting to Contentful. And it’s my absolutely favorite use case, because I’ve been there, I’ve used the audio guides, they’re incredibly well done, they’re informative, they have pictures. The pictures are changing while the audio’s playing, they have like the family quiz version where you have tasks to do with your kids, like do pantomimes, or makes sounds and stuff.

    So it’s incredibly good content, it’s really well presented, and using our technology to enable it. So this is my absolute favorite. They do some other things, they have some websites running, or some signage systems, but these audio guides are so amazing. If you’re in London, go to the British Museum, take five pounds, it’s going to a good cause, partly to us, get an audio guide, just experience it. They’re incredibly well done, and it’s using the same CMS you can use to call your block. Just think about that. This is a big institution with a lot of money, and they’re using the same product you can use for free to call your block. That’s my absolute favorite.

    Stefan: Yeah, speaking of which, just to make that clear. Contentful is actually a set of APIs, so it’s not just only retrieving data, we also have a read-write API, so when you need your own interface, or a simplified version for that, that’s also available, and then also preview API. So when you for example don’t want to publish stuff but want to build an environment to see edited things, this is also included.

    Brian: Okay.

    Rouven: And the Images API, my favorite API to downscale images. Also one of our APIs.

    Brian: I’ll have to check that out. And then as far as Contentful, you guys have like a trial period or a free plan to check out?

    Stefan: The free plan is up to 50,000 API calls per month.

    Brian: Oh, okay that’s totally doable.

    Stefan: I think so too, that this is enough for a side project.

    Brian: Yeah, and if you go over that then you probably want to talk about investment, and maybe getting some VC funding or something.

    Stefan: Exactly. This is the normal reaction that happens then. When you have the problem that you’re dragging 50,000 requests, you have different problems than.

    Brian: Yeah, than paying for Contentful.

    Rouven: Especially since 1,000 more API calls are just 15 cents. So if you pay us 15 cents, you get to keep using the API.

    Brian: Yeah.

    Rouven: We’re very cheap for Germans. We’re easy, just give us some beer, we’re good.

    Brian: Cool, I’ll keep that in mind, I’ll send a case of my finest Budweiser.

    Rouven: Um, Czech Budweiser you mean, not American, right?

    Brian: Well, Budweiser is not the best beer, it’s not our best export.

    Rouven: The Czech one is really good, it’s also called Budweiser, but it tastes way better.

    Brian: Cool. I really appreciate you guys talking about Contentful and kind of giving us the rundown, but for now, let’s go ahead and transition to picks, if you guys down mind. So yeah, these picks are jam picks, anything that you’re jamming on, things that are keeping you goin, that you like to do to keep your work going, or just even outside of work. So technical or non-technical are totally fine. But I’ll go first.

    My first pick is Webpack. Webpack is a build tool. A year ago I barely knew what it was. One of my lead engineers, actually Christian Schinzler, I worked with him, and he was a big Webpack fan.

    So Christian was actually on Episode Number One on the JAMstack Radio, so for anybody who remembers that far back, but yeah he was a big Webpack fan, I didn’t know anything about it. And I finally got around to actually learning it well. So I highly recommend checking out the webpackjs.org docs, because they did a really good job of making sure it was clear.

    Rouven: Version two is much better, right?

    Brian: Yeah, version two is much better, you can actually understand what’s going on, it’s very clear. So I’ve been doing offline plugins, service workers, I’ve been optimizing our build to make sure it’s not crazy heavy-weighted and stuff like that. And just from a little reading I was able to improve our bundle size and stuff like that. So highly recommend Webpack.

    My next pick is actually Service Workers, too, just roll right into that, there’s a cool offline plugin for Service Workers, which I’ve implemented in Netlify. It’s not in production yet because I’m trying to figure out some crafty ways to offline our docs to as well, currently just our homepage is offlined, so hopefully when you lose network requests on your phone or your desktop, you’ll be able to at least read our docs.

    And all that will be cached, will which be super awesome, so check that out, Webpack and Service Workers. My final pick is Luke Cage, which is a Netflix series. So I don’t know if you guys get all the Netflix series in Berlin, do you?

    Rouven: Yeah, most of them.

    Brian: Most of them? Well Luke Cage is made by Netflix, and it’s a Marvel series. He’s an African-American super hero, and he came out during the jive-talking ’70s, and that’s where he kind of was birthed in the comic books, and they’ve modernized him and put him into a Netflix series, Marvel series, and they’ve also brought along some of the ‘70s undertones too, which is really cool, a lot of the music, a lot of the stuff that came from that comic book.

    I’m very impressed that they were able to intertwine it and not make it as cheesy as the comic books were. So I just finished it, it took me a while to finish it, cause I have no time to watch TV as of late. I do a lot of coding and a lot of side projects, so I just don’t find the time. So I just finally finished the last couple episodes, so I just wanted to recommend, anybody who hasn’t seen Luke Cage, check it out, and then maybe read the comic and try not to laugh. Stefan, since I introduced you first, you have some picks you want to share?

    Stefan: Sure. One question though, have you SW Toolbox when you play around with Service Workers?

    Brian: I haven’t. What’s SW Toolbox?

    Stefan: That’s actually a Google library. And I did a Service Worker implementation six months ago, and it took a while to get my head around it, because you have to hook and to activate and fetch and all that stuff. With SW Toolbox, it’s five lines, reconfigure it, you can say cache first, network first or whatever, and I implemented it and it was like “Wow, I lost a lot of time here.”

    Brian: Yeah, I definitely did spend at least a week on just tinkering and manipulating different things and fetching and pushing.

    Stefan: Check it out, you will see that it’s not that hard then.

    Brian: Okay, cool, I will, I definitely will check it out.

    Stefan: Okay, well cool. My first pick is Unsplash, this images API, where you can get beautiful images under Creative Commons License, for free. That’s awesome, I love that. Especially when I do conference talks or something, I spend a lot of time just looking for beautiful images, so this really helps me out.

    And actually, three days ago I came across an app that is called Zoomi, or Zoom-eye, which is basically a MacOS app which queries a lot of image services that are all Creative Commons, so I now just have to type in a phrase and I can choose out of 100 beautiful images what I want to use for my presentations, so this will definitely save me some time, I recommend it.

    And extra today, when I was younger I was a little more into music, this kind of disappeared when I started coding. But accidentally I came across a band I used to listen to, which is Marot Rockel, it’s a little bit funky electronic music, and I listened to them the last time six years ago, and I discovered that they released three more albums, and I had a dancey day today, it was really good.

    Brian: A dancey day, it’s too bad this is an audio medium.

    Rouven: Stefan really cares about his stock images. Stefan is our slide quality obsessed evangelist. Whenever I had to do any presentation I had my slides to Stefan, and now I get a list, a Google Doc two pages long with all the things wrong with them back. And after I go through it they’re going to be perfect, but he’s really obsessed with the quality of his slides, and he’s great at it, so he loves his images.

    Stefan: It has to be pretty, right?

    Brian: Yeah. I was going to say I was going to look up some of your talks so I can try to mimic some of your talking in slides. Most of my slides I throw it together last minute, so you probably won’t be proud of mine.

    Rouven: Yeah, me too, Stefan is different, Stefan is amazing.

    Stefan: Well thank you.

    Brian: Reuven, you got some picks for us?

    Rouven: Yeah, of course. I’m late to the party, because I got late to Germany, but I’m really in love with the Alexa API right now. I’ve done two skills right now, I’m working on my third one, all of them using Contentful, because “ooh multi-platform.” I get to use content in a non-visual way, which is breathtaking to me still. But even beyond that, just the quality of the interaction, but also the work that goes into the interaction, and actually having to think about, “Hey how do people want to interact with something where you can’t click” is to me just fascinating.

    So I love the API, I love that I can use it AWS lamda, so I don’t have to run a server anywhere, I don’t have to think about hosting it anywhere, I just upload it to Amazon, and Amazon takes care of all the crap for me, but also the thought pattern, having to evolve from, I’ve always done either PHP band dev, or JavaScript phone-in dev, so it was always very visual, and having to think about in a different fashion is to me super engaging right now.

    And the other thing I was raving on today is, for lunch today with a co-worker I made the most awesome omelets ever in our brand new kitchen, we just moved to a new floor with my team. I just love food and today’s omelets were amazing and we had some good wine for it for lunch, and it was the best lunch I had for a long long time. It was amazing. Shout out to Linn for actually having the idea of making us omelets today.

    Brian: Nice. You can’t underestimate a good omelet. Anyway, guys thank you very much for coming on, jamming with me and talking about the JAMstack, talking about Contentful. I’m very intrigued, I’m definitely going to be using this for some future side projects, and getting real used to it.

    This podcast is brought to you by Heavybit, a program that helps developer-focused companies take their product to market. Throughout the program, founders work on developer traction, product market fit and customer development. If you like what you hear, be sure to subscribe to Heavybit’s Podcast Network on iTunes.

    — Malia Powers

  • NodeSource, AppDynamics Ally for Node.js DevOps

    NodeSource, AppDynamics Ally for Node.js DevOps

    The prevalence of Node.js as the most widely deployed instance of JavaScript is putting a lot of pressure on DevOps teams to find ways to automate the management of Node.js applications. To provide the visibility needed to achieve that goal, NodeSource has announced it has integrated its N|Solid development environment with the application monitoring tools developed by AppDynamics.

    Pravin Halady, product manager for NodeSource, says one of the things that differentiates N|Solid, which is based on Node.js, most is the amount of telemetry data NodeSource is able to collect from Node.js applications. The alliance with AppDynamics provides a mechanism for DevOps teams to now tap into that data to better manage those applications alongside the panoply of existing applications they need to support and manage, says Halady.

    AppDynamics already supports Node.js. But Halady says the alliance with NodeSource provides deeper visibility into, for example, asynchronous activity. Advanced metrics can be used to augment the basic Node.js metrics that AppDynamics currently provides to better ensure service level agreements (SLAs) are both achieved and maintained, Halady says.

    Adoption of Node,js exploded across the enterprise once it became feasible several years ago to deploy JavaScript code on both the client and the server. That capability in many ways fulfilled the early promise of Java in the eyes of many developers. But DevOps teams have been slow to include support for JavaScript applications within their DevOps processes, mainly because of a lack of visibility. Now that the base of Node.js applications in the enterprise has reached a level of critical mass, tools vendors such as AppDynamics are looking to provide that visibility by expanding the scope of support they provide for Node.js.

    Halady says NodeSource chose to partner with AppDynamics in part because in the wake of AppDynamics being acquired by Cisco Systems, it’s clear that NodeSource needs to participate in the end-to-end visibility platforms that larger companies are starting to build. Cisco has already made significant investments in a Cisco Tetration analytics application in addition to committing to imbuing the IT infrastructure environment it provides with machine-learning algorithms. That data will then be combined with data about applications collected by AppDynamics to identify the source of any potential performance issue faster. NodeSource, adds Halady, will look to foster similar agreement with other providers of application monitoring tools.

    In the meantime, DevOps team can take heart in the fact the Node.js Foundation is making providing visibility into DevOps environments a higher priority. The number of Node.js applications in the enterprise is only expected to increase. There’s a high correlation between usage of Node.js and containers that collectively are being used to facilitate the development of microservices. As those microservices bgin to replace monolithic applications developed originally developed in languages such as Java, the size and scope of the DevOps challenge exponentially increases.

    Application development in the enterprise is going to be polyglot well into the next decade. Today there are more programming languages being actively used in the enterprise than any time in history. In fact, the days when IT operations could mandate what technologies developers can employ are long over.

    — Mike Vizard

  • VMware Cloud on AWS Will Bring DevOps Issues to a Head

    VMware Cloud on AWS Will Bring DevOps Issues to a Head

    Now that instances of VMware are starting to become available on Amazon Web Services (AWS), a DevOps ecosystem is starting to emerge around the management of applications workloads moving into and out of AWS.

    Among the more than 4,500 applications that VMware claims already support VMware Cloud on AWS are those from Blue Medora, CA Technologies, Chef, Cloudbees, Dynatrace, Jfrog, Puppet, SaltStack and the Ansible arm of Red Hat.

    VMware CEO Pat Gelsinger told attendees at the recent VMworld 2017 conference that VMware and AWS are just now scratching the surface of what they plan to accomplish together. AWS CEO Andy Jassy added that, unlike other alliances, which consisted mainly of marketing, engineers at both companies are working closely together to deliver VMware globally across all AWS availability zones. At present, VMware Cloud on AWS is available in the AWS availability zone hosted in Portland, Oregon. The service soon will be extended to the east coast of the United States, followed by rest of the world over the course of 2018.

    The availability of VMware Cloud on AWS should force several DevOps issues within enterprise IT organizations that have deployed the VMware stack of software mainly on-premises. Many of those organizations have also been employing public clouds such as AWS. But because those environments are based on different hypervisors, it was never easy to migrate workloads between the two platforms. There are plenty of options for running VMware software in the cloud, but because AWS is the largest provider of public cloud services, there are many customers who want to seamlessly move workloads between their on-premises IT environments and AWS via a common stack of VMware software.

    In many organizations the desire to shift applications workloads into AWS will bring many DevOps issues to a head, as VMware and AWS use has occurred in independently of each other. There could be many workloads running on each platform, but for all intents and purposes, IT organizations have managed both platforms in isolation. Now that VMware is becoming available on AWS, all the challenges associated with moving workloads between a public and private cloud come to the fore.

    Most of those DevOps issues will be focused first on moving some classes of workloads running on VMware into AWS. Application workloads that don’t need to run constantly will be among the primary candidates followed by a slew of data protection applications that will enable IT organizations to back up and archive massive amounts of data inexpensively. Next up will be applications that need to be distributed globally to increase availability.

    At the same time, a lot of VMware shops will be moving application development into the cloud. But just because an application is developed in the cloud doesn’t mean it will stay there. Many IT organizations have already discovered that when it comes to long-running applications, the total cost of deploying them on-premises is less than deploying them on a public cloud. The DevOps tasks ahead, of course, will start with figuring out which VMware-based applications make the best candidates from a financial and availability perspective to be deployed on AWS.

    — Mike Vizard

  • The Right DevOps Tools for Value Stream Velocity

    The Right DevOps Tools for Value Stream Velocity

    Think of DevOps tools as stepping stones to releasing new functionality into the hands of your customers faster and more frequently. Working together, tools automate processes and help companies become modern software factories. But isolated tools, like a lone rock in the middle of a river, are much less useful.

    Disparate tools may help an individual or a team do their job, but they impede the progress of the larger organization. To achieve desired efficiency across the application life cycle, companies may waste time and amass technical debt by integrating tools with scripts.

    The other answer is that organizations get restrictive. They may establish a “center of excellence” for DevOps that prescribes a toolchain that works together, but are those tools going to be the best fit for all individuals and teams, supporting an agile mindset?

    The ideal scenario is for a DevOps toolchain to become an integrated, dynamic continuous delivery powerhouse where “best fit” tools work in concert to enable better performing teams and better customer experiences.

    Start with the Right Tools for the Job

    DevOps teams are focusing on automating and streamlining all aspects of the application life cycle. It makes sense to choose the tool that’s best for the job. And teams should feel free to experiment and take advantage of new tools emerging every day that solve new problems or do a better job solving existing problems.

    Typical areas of DevOps tool investment include:

    • Planning and managing development work
    • Managing, tracking and documenting all code changes
    • Building, managing and deploying containers
    • Automating iterative builds/continuous integration
    • Developing and deploying to cloud platforms
    • Automating the deployment of applications into environments at all stages
    • Provisioning, configuring and monitoring environments
    • Managing all types of testing with the right requirements and data
    • Validating security before vulnerabilities become a problem
    • Promoting better agile team collaboration
    • Monitoring and managing an application in production

    These tool categories each serve a vital role in the application life cycle. However, there remains a significant lack of coordination and shared knowledge across the DevOps toolchain to generate the value stream velocity that can keep businesses competitive and customers engaged. For example, if a testing function is isolated, it becomes a black hole of activity and often a bottleneck to delivery.

    Build the Smarter DevOps Toolchain

    When teams can visualize, orchestrate and track the continuous delivery value stream from development through production, the pipeline becomes an intelligent entity of its own. How is the pipeline performing? How is it adding value to the organization? How do you improve the pipeline?

    For example, by integrating the planning and management solutions as part of the value stream, you can keep track of the user stories—including where they are deployed, what tests have been done and the results. Bi-directional integration enables automatic updates in the planning/management tool. Layer in collaboration tool integrations, and dev teams can get instant feedback and respond more quickly, shifting quality left.

    The result: real-time information and the ability to plan and prioritize more effectively. Release and development teams now have a shared view and understanding of release status and business implications.

    The testing function is another area where tools can work better together. Instead of sequential and separate functional, integrated, performance, API or security testing, you can build a testing workflow that runs in parallel, automatically promotes tasks, monitors progress and sends feedback of testing status. This can include orchestrating the generation of the right test cases, test data and virtual services for the app under test. With all the testing knowledge gathered in one place, it’s easy to see testing gaps and determine the risk of a release.

    Launch DevOps Mission Control

    A control center that manages the continuous delivery value stream can help teams not only manage the health and schedule of an active release more effectively, but it can also provide the visibility and analytics to help teams iteratively improve.

    There are a number of tools on the market that look beyond the silos of the application life cycle to connect and extract value from an end-to-end toolchain. As you assess tools that can enable a dynamic value stream, ensure they have capability in these five areas:

    1. Plug and play integrations: Does the tool support common DevOps plug-ins? How easy is it to integrate tools, whether they are open source, commercial or home grown?
    2. Full pipeline visibility: Does it enable orchestration and tracking of all releases at all stages—planning, development, test and production—including feedback loops and planning for the next release?
    3. Continuous quality: Does it support a shift-left testing model to ensure testing happens continuously and rigorously? Does it help you prioritize your testing and automate risk assessment for your releases?
    4. Business value alignment: Does it provide intelligence about the app content in the pipeline? Can you see what code is deployed where in the pipeline and its status? Can you map how the content of a release aligns to business priorities?
    5. Actionable analytics: Does it extract meaningful data points from pipeline operation to provide analytics that support real-time decisions, as well as inform improvements over time? Can you get insights into release and pipeline performance, testing coverage and speed, release readiness, team productivity, and so on?

    With intelligent integration of your DevOps toolchain you can increase your value stream velocity and deliver the best quality apps—every single time.

    — Patricia Johnson

  • Understanding the Data Scientist’s Role in Cross-Functional Teams

    Understanding the Data Scientist’s Role in Cross-Functional Teams

    In our previous article, “Data Science Industry Perspectives in the Cloud,” we discussed that evolution is key if you plan to grow your business. You can start with ready-made solutions, then, in time, you can switch to in-house solutions created with the help of a group of data scientists.

    This time around, we’ll talk about the cases and solutions when machine learning as a service (MLasaS) doesn’t work. Your company’s first move is not to go out and hire a data scientist to solve your urgent business needs; rather, your company should invest in a custom-made solution. Only when you have a workable solution can you dive in deeper and create a proper team that can create an in-house data science solution.

    Custom Made End-to-End Solution as a Start

    Data Science in the classical sense involves data, goals and models to solve a pressing issue. The best way to jump start a data science process is by putting together various ready-made services into a single workable product to show your customers clear-cut results in short order. This can be done without any complex or global research, and you can comfortably formulate specifications taking into account all the feedback from your customers to create a more high-level data science offering later.

    One of the biggest issues for any data science project is in formulating the specifications for it. The usual request is, “Create something for my company using data science. Analyze data for me.” This type of a job has lots of trial and error. Having a custom end-to-end solution from the start lets you add on extra services as needed, and enables greater insights and predictions into a specific business workflow.

    I believe companies should build their data science system not from the data science research but from the point of view of an end-to-end solution. And then, bit by bit, they can adjust their services in the process.

    Moving on to Data Science Research

    Companies start data science research by employing a data scientist who matches their needs. You can use the standard data scientist qualifications: This employee should have a firm footing in the application environment, mathematics and programming. Also, their business skills are key to success—I believe an understanding of the topical area is key here because we are solving business issues. And a data scientist who solves problems that are more academic will be more focused on winning a Kaggle competition than on addressing the business needs.

    It is important this person understands the product development cycle. This way, the models and analyses data will be built in such a way that they could be deployed in production. For example, if a person is using R (the language and framework for data scientists), we should note that it is more research-oriented and it is not production-ready. Correspondingly, the results of such research cannot be deployed in any end-to-end solution. Therefore, the data scientist should work with a programmer.

    Although in the above diagram we see that the Data Scientist should have hacking skills, in reality, this is not the case. In their vast majority, Data Scientists are not able to write quality code. And if you need not just the research but a solution then process-wise you need to have a Data Scientist working together with a Data Engineer.

    CAP Theory as an Analogy

    It is impossible to have three database properties at once: consistency, availability and partition-tolerance. This basic rule is known to all developers. This is true for a data scientist as well: People envision a data scientist overlapping all three  properties, but in real life, you cannot find such an ideal candidate. Usually, people tend to lean one or another way in their work and keeping a balance is not always a priority.

    In principle, data scientist should have business insight, understand the math behind it and work in tandem with a skilled developer. Of course, a proper data scientist should be able to write any semblance of code, but a data scientist should work with a data engineer to write and realize code, being responsible for both the quality and sustainability of this solution.

    The classic tragedy of a company that decides to initiate any data science research is when a data scientist says, “I’ve got 40,000 lines of Python code on my PC. Can you make it work in production?” And, of course, this is virtually impossible to do. When that happens, all the research is simply wasted.

    Cross-Functional Teams

    Any team that deals with data science has to be cross-functional. In other words, it has to cover a whole stack of the solutions it writes. In a normal infrastructure there should be a DevOps engineer, data scientist, data engineer and a product developer writing the web app and/or mobile app. This single team is responsible for the result. The team members should work together and solve tasks that are interconnected in their interactions.

    All of this means that the whole team is responsible for the business result. This is also true for the transitionary research done by a data scientist that is impossible to use in production on its own.

    Old-School vs. Vertical Teams

    To dig in deeper, let’s take a classic old-school layered company organization structure when you have a department of data scientists, operations, UI developers, a big data department, QA engineers and so on. In this case, we have every project penetrating most of these teams. And the classic problem is that tickets and tasks are being thrown around by one team to another, and the real business goals are being watered down along the way and not solved in the end. So, instead of this horizontal division, we have divided the teams vertically. This allows us to create teams that see a clear-cut goal they need to achieve. And at the same time, they can improve their cross-skills and boost their responsibility levels.

    As a result, such teams began to deliver, Scrum and Agile properly. It is not directly related to data science, but companies need to realize the difference between a data scientist, which works in academia, and a production data scientist. Companies should aim at employing the latter one within their teams, and not let a data scientist work alone remotely.

    About the Author / Stepan Pushkarev

    Stepan Pushkarev is the Head of DevOps Practice at Squadex.com and CTO of Hydrosphere.io. Co-founded and managed engineering teams for eCommerce, IoT and Ad-tech companies. He has been responsible for the full products stack: math models, infrastructure & operations, enterprise applications as well as hiring, establishing engineering culture and delivery process. Stepan combines strong technology, management skills and entrepreneurial spirit. Connect with him on LinkedIn.

     

  • Redefining NoOps to Better Inform IT Decision-Making

    Redefining NoOps to Better Inform IT Decision-Making

    It is safe to say that since the term NoOps escaped from its confines as an assembly language instruction (NOP or NOOP mnemonic for no operation) and in microelectronics (NoOps or null instructions inserted into pipeline, creating bubbles*) into enterprise IT, it has caused more than its fair share of consternation and controversy in the DevOps community. This is hardly surprising, when some of the more prominent claims for NoOps placed emphasis on ending operations entirely, threatening the jobs and livelihoods of thousands of knowledge workers. As with previous movements for automation, NoOps came under immediate attack—and, somewhat ironically, the criticism came from IT industry workers. For better or worse, IT has more than its fair share of hyperbole, religiosity and grandstanding, with the NoOps debate being no exception.

    As someone who started his technology career as a developer, moved into infrastructure and is now accountable for resources being allocated to IT, I find the debate does little to inform my decision-making. In this article I want to share my definition of NoOps with the broader community to clear path for greater adoption of NoOps principles among the DevOps community. I see both of these communities as being distinct and yet highly complementary, as long as our ultimate goal is to generate value for the organizations we choose to work for and, by extension, the customers which they serve.

    NoOps as an Ideological Principle

    Various definitions of DevOps have been refined and become more stable over time. I don’t believe it would be particularly controversial to recognize DevOps as a culture, a methodology for creating high-quality applications and delivering them into production in the most efficient manner. In my book, “NoDev NoOps NoIT,” I choose to define NoOps as a guiding principle that makes assertions to help IT decision-makers inform their decisions. It is more useful to separate our notions of DevOps and NoOps into methodological and ideological categories, because doing so strengthens our understanding of both terms and equips us to make better IT decisions.

    Before we examine the assertions made by the NoOps principle, I want clarify why I have rejected other definitions of NoOps and critiques of NoOps itself.

    NoOps Spells the End of Operations

    One of the most common and enduring uses of the word “NoOps” is to refer to eliminating IT operations entirely. The problem with such definitions is that they conveniently scope IT operations in a manner that allows them to be eliminated. IT operations roles are not static in the face of changing technology; arguably, they evolve more rapidly than application roles. The original activities of racking, stacking, cabling and administering IT infrastructures in a typical enterprise has been hugely diminished by outsourcing, commoditization and the advent of cloud infrastructure. However, new activities in software development life cycle, automated provisioning, monitoring, metering and security have created new roles for infrastructure/operations developers. NoOps is not about eliminating IT operations; rather, it is concerned with maximizing automation in IT operations.

    NoOps is the Evolution of DevOps

    While superficially it may well seem that NoOps takes DevOps to its logical conclusion, this line of thinking chooses to ignore the huge swath of DevOps methodology that does not overlap with the automation of operations or the fact that DevOps is itself an evolving methodology. I simply don’t find this definition to be particularly useful in day-to-day decision-making. At best, it’s a dimly lit observation.

    DevOps is NoOps—NoOps is Just a Confusing Term

    The idea of NoOps is an expression of the benefits of automation. How we choose to define the term will dictate its utility. If we choose to define it in a fashion that leads to confusion, then fault lies with the definition and not with the idea itself, which I believe is a powerful one that should not be ignored. Attempting to simply define away NoOps is as wrong-headed as stating that NoOps is the end of operations. Overloading DevOps methodology to encompass NoOps ideology stretches the definition of DevOps beyond its useful boundaries.

    NoOps Assertions: Eliminate All Manual Intervention in IT Operations

    Eliminating manual intervention in IT operations is a fundamental assertion of NoOps. This assertion is made from an understanding of the benefits of the automation and the observation that the very tools we use in IT—computers—were invented to automate conative and algorithmic processes.

    While we may never be able to fully automate every aspect of our operations—for example, some regulators demand four-eyes processes for approval and execution—we must rigorously challenge the unthinking orthodoxy that currently governs many IT processes.

    Automate, simplify and standardize: I subscribe to this mantra but one has to proceed in the right order; that is, simplify, standardize and then automate. Otherwise, you literally risk making an ASS out of your IT automation efforts.

    I can say with a high degree of certainty that any failure to automate a process could probably be traced back having made no attempt to simplify the process before automating it. No attempt was made to challenge the process, to question who is doing what and why before automating it. Indeed, no effort, was made to question the existence of the process in the first instance.

    Once a process has been simplified, standardizing it follows almost naturally. Stakeholders adopt your process because it is simple, it saves them time and generates value for them.

    The opposite is also true: A poorly thought-out, complex process will never win buy-in from users. Any attempt to put lipstick on such a pig will be fraught difficulty, often resulting in the vast majority of users and stakeholders finding exceptions to your attempts to increase adoption of the offending process. 

    NoOps Assertions: Eliminate All Dependencies Between Application Developers and Infrastructure/Operations Developers

    As the trend toward software-defined everything matures, we need to change our organizational structure to adapt with the new landscape. Within enterprise IT, the war between application and operation teams is over, even if there are still combatants that don’t realize that the application teams won. In the software-defined paradigm we are all developers. Application developers write business/end user logic and infrastructure/operations developers write run-book instructions using the tools and APIs made available to them by infrastructure-as-a-service (IaaS) and platform-as-a-service (PaaS) vendors. Both of these types of developers write code and we find more and more frequently it is the same individuals writing both application and infrastructure code; hence, the rise of the full stack developer and the full spectrum technologist.

    As a CTO I organize my technology division into three teams: Application Development, Infrastructure Development and User Experience Development. These teams are physically situated next to each other and work toward a single technology strategy defined by a NoOps ideology and DevOps methodology.

    In Summary

    When we are presented with ideas that are threatening and challenge fundamental elements of our existence, such as the way we work, the best response is to seek out opportunities presented by those ideas and adapt to them. For the DevOps community, the first step in engaging with NoOps is to accept the ideology is based on a very real and powerful foundation of automation. As a community we can choose to dismiss NoOps or we can choose to embrace the possibilities it offers and allow it to enrich DevOps methodology. I would posit that those among in the DevOps community who can adapt to a NoOps reality will not only survive but thrive.

    About the Author / Hussein Badakhchani

    Hussein Badakhchani is a Distinguished Technologist and author with over 20 years of professional experience in financial services technology. Having worked for institutions such as the Bankers Automated Clearing System (BACS), Deutsche Bank and VocaLink – MasterCard, Hussein has a proven track record of delivery in some of the most competitive global capital markets. As a trusted academic, practitioner and adviser, Hussein also provides critical analysis of technology to executive and investment decision makers in government, banking, financial, commercial and defense sectors. He is the author of “NoDev NoOps NoIT,” principles governing the ideology, methodology and praxeology of informed IT decision-making. Connect with him on LinkedIn.

    *”Microelectronics,” by Jerry C. Whitaker.