devopsy.kinsta.cloud

Tag: development

  • Split Launches Free Managed Feature-Flag Service

    Split Launches Free Managed Feature-Flag Service

    As part of an effort to attract more DevOps teams, Split is making available a free managed feature-flag service for up to five users within an organization.

    Based on the same commercial managed service Split provides today, company president Trevor Stuart said the free service provides those five users with unlimited feature-flagging and unlimited usage in the hopes that the number of DevOps teams employing feature-flagging will rise to the point where the organization will opt to pay to consume the paid managed service. The free managed service also includes user segmentation capabilities to provide control over how many end users might be exposed to any new feature.

    The paid managed service provides access to additional capabilities such as integrations, access and permissions, team collaboration and full-stack experimentation.

    Stuart noted many DevOps teams are today attempting to incorporate feature-flagging into their processes by implementing one of more than 40 different open source projects. However, many of them quickly discover that building and maintaining a feature-flag platform is a difficult undertaking. Many developers decide that building a feature-flagging platform is a challenge they want to take on because it provides a critical set of DevOps capabilities. But when those developers leave the organization, the DevOps team is suddenly left with a platform no one else in the organization knows how to support, he said.

    Feature flags, also known as feature toggles, provide a means for DevOps teams to test and add new capabilities to their applications without having to take the entire application offline. Feature-flagging also enables phased application rollouts that make it easier for DevOps teams to assess new functionality as it is added. Any problematic update can be turned off with a single click. The downside is they generally result in more branches of applications that need to be managed carefully within the context of a DevOps process. Because of that complexity, vendors such as Split have been making the case for employing platforms that enable DevOps teams to more easily incorporate feature-flagging within a DevOps process.

    Stuart said the ultimate goal is to provide a more controlled approach to experimentation. Instead of randomly testing new features on end users, feature-flagging makes it possible to more easily test features on specific end users, said Stuart.

    It may be a while longer before feature-flagging is adopted more pervasively. But as DevOps teams become increasingly more sophisticated, there’s little doubt feature-flagging won’t become more commonly employed. The simple truth is that uncontrolled application experiments on end users usually are not appreciated. Savvy DevOps teams are finding ways to test application functions in real-world scenarios with minimal disruption to the majority of end users.

    The only real issue now is determining how best to go about implementing feature-flagging at a time when most DevOps teams already are struggling to keep pace with their existing application development and deployment projects.

    — Mike Vizard

  • Using Continuous Backup to Keep Pace in an Increasingly DevOps World

    Using Continuous Backup to Keep Pace in an Increasingly DevOps World

    How continuous backup can help DevOps teams keep working with the right data

    The world of software engineering has changed; in fact it’s constantly changing. We’re moving on from the days of waterfall when big-budget releases with multiple features per version were the norm. Now, to compete, organizations have to implement agile development, with shorter release cycles and fewer improved or new features at a time. Modern software development has evolved to a practice of continuous development to the point that each new release feels like a simple patch rather than a dramatic overhaul.

    This has completely upended the testing process. Release cycles are a fraction of the time they used to be and the previous QA methods don’t stand a chance to keep pace. The latest model combines developers, operations, testing and security into close-knit teams. Development cycles are shortened and testing and deployment are automated from the source. All of this blending of teams and processes has been dubbed DevOps.

    This new paradigm for operations and development has been made possible by the flexibility ushered in by virtualization and the snapshots that can be made for a point-in-time failback. Yet, limitations remain that can keep technology from supporting the advanced needs of DevOps teams. More is still needed to keep environments up to date because once a snapshot is deleted, the data that it represented is gone.

    New tools have emerged including Docker, Kubernetes and Puppet to help on the operations side, but development teams can still be expensive if they aren’t used efficiently and effectively. Any amount of downtime or idle waiting for a database to be refreshed from production or for any sort of regression testing environments to be instantiated is burning money. DevOps processes can do this, but they aren’t effective if they don’t have a continuous stream of recovery points.

    The Quality of Code

    Development can only be as good as the copies of production data that a developer has to work with. The challenge is to get a current copy of production data in a reasonable amount of time. If it uses simulated test data designed to mimic production, results can be shoddy and defects can occur more frequently.

    A continuous backup solution with search and recovery across data, files or VMs from any point in time can make development and testing easy. Spinning up a clone of production environments is simple if you have multiple copies of data in different locations in a one-to-many configuration. Production data can then be replicated to DR facilities for protection, but also to development, test and QA environments to assure that developers and testers are working on the same, latest version of production data.

    Production replicas along with the simultaneous use of a proper API can allow the orchestration of test builds and deployments without manual intervention. This is essential because being able to perform tests on good, reliable data can significantly reduce the amount of code defects deployed into production.

    Confidence in Changes

    Because you can spin up real-time replicas of production workloads with ease, developers don’t have to guess what might happen when they push a new code update into production. Instead, the can reproduce an exact replica of production that has the latest data to an isolated area where it can’t affect production users. In this environment, they can observe the results without any risk.

    — Gijsbert Janssen van Doorn

  • How DevSecOps Can Help Avoid Catastrophic Breaches

    How DevSecOps Can Help Avoid Catastrophic Breaches

    Earlier this year, Tesla’s Cloud was hijacked and used to mine cryptocurrency, exploiting a vulnerability in the company’s Kubernetes cluster. A mountain of FedEx data was recently exposed, affecting 119,000 individuals. The Equifax breach garnered international attention after an estimated 145.5 million Americans were jeopardized. In other news, we’ve reported on how the Vine Docker registry fiasco was hacked, leaving an embarrassing PR trail in its wake.

    What is common across all these scenarios? They arguably could have been avoided with basic safeguards underpinned by a healthy dose of DecSecOps. In hindsight, let’s see what specific strategies developers can adopt to avoid such horrendous leaks.

    Basic DevSecOps Can Prevent Vulnerabilities

    DevSecOps is described as placing security at the forefront of every action. Within the world of cloud tooling, this means instilling protection at every point of the build life cycle. You would think this equates to advanced container and orchestration security, powerful access management and establishing hardened oversight for internal applications, right? Actually, many modern exploits simply involve a lack of basic password protection.

    Case Study No. 1: Tesla Cloud Cryptojacking

    In mid–2018, Tesla’s Amazon servers were hijacked by malware and used to mine cryptocurrency for rogue agents. Similar cryptojacking has transpired at Gemalto, Aviva and others. According to RedLock’s report, the hackers infiltrated the Kubernetes console, which was not password-protected. In one pod, they found access credentials to Tesla’s AWS environment. The breach exposed sensitive telemetry data, though was limited to “internally used engineering test cars only,” according to Tesla.

    How to Prevent

    Password protection for Kubernetes administrative consoles is a readily apparent lesson here. All cloud accounts, even if used internally, must be better-equipped and access credentials sequestered. Surely, instilling a “security by design” mentality can help limit the number of such consoles left accessible.

    Case Study No. 2: Vine Registry Breach

    In 2016 a white hacker by the alias avicoder was able to infiltrate Vine, the video-sharing social network. The hacker used tools to discover subdomains and behind one—https://docker.vineapp.com—found an open Docker private registry that housed the Vine source code. Such a vulnerability could have been used to collect user details or inject malware for malicious purposes. Thankfully, no users were compromised, and the Twitter Bug bounty program awarded avicoder a handsome sum for the discovery.

    How to Prevent

    In this particular incident, there was nothing at fault with Docker or Docker containers. The service (which was meant to be private) was simply left public and devoid of access controls. The lesson is that URLs on the world wide web are just that—publicly exposed to the world. It does not matter if no documentation links to them; such subdomains are easy to discover with crawlers and will be exploited if found.

    Case Study No. 3: Equifax Exploits

    In the Equifax breach, millions of customer records were stolen. The Equifax report notes that hackers exploited a “website application vulnerability.” Further reports detailed that the 2017 breach utilized flaws in Apache Struts, likely through a zero-day exploit, according to the Apache Struts Project Management Committee.

    How to Prevent

    Some note that the deserialization of untrusted data inherent in Apache Struts applications leaves some major vulnerabilities and potentially malicious code execution.

    Containers, due to their short lifespan, are not persistent, therefore making consistent hacking harder. They can also isolate functionalities so as to decrease attack vector. Regarding the Equifax breach, Aquasec speculates that container usage could have lessened the impact:

    “… it’s likely that such a breach would have been more difficult with containers, and that if successful, it would have been less persistent, not as widespread, and mitigated sooner.”

    Some view containers as a formidable force if armed with behavioral analysis and firewalls for network connections. Still, others note flaws in assuming containers are safer.

    Regardless of architecture, In DevSecOps, security for customer data isn’t drafted into the background; rather, it is imbued into the entire tooling process and holds equal footing within each build.

    Instilling a DevSecOps Culture

    Instead of a build-now, secure-later attitude, DevSecOps seeks to elevate security into every decision. With each build, security is melded with greater forethought regarding the repercussions for how data is treated.

    Admin consoles for Kubernetes clusters must be armed and password-protected, and the same goes for private GitHub repositories or Docker registries. Any traversal of data through cloud SaaS services must have authorization in place. A most obvious lesson is to password protect those “hidden” registries with discoverable URLs.

    Bounty programs for white hackers continue to lead to successful bug detection. At least for these institutions, there is less evidence of compromised user data and remediation is quickened.

    Much of this can be distilled by due diligence and through breeding a culture of security. While that’s a little fluffy for some, it apparently still requires repeating for institutions that continually de-privilege these basic safeguards.

    — Bill Doerrfeld

  • DevOps Chat: Microclimate and IBM Cloud Private with Toby Corbin

    DevOps Chat: Microclimate and IBM Cloud Private with Toby Corbin

     IBM Cloud Private has cemented itself in the DevOps space as a personal cloud environment, which is great for developers who need the agility and computing power of the cloud but don’t want to step into a public cloud setting. Tony Corbin, technical lead for the Microclimate Project, describes IBM Cloud Private as “a cloud specifically targeted at individual customer.”

    In this DevOps Chat, Corbin and I talk more about IBM Cloud Private and its role in DevOps. As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hey, everyone. It’s Alan Shimel, DevOps Chat, and you’re listening to another DevOps Chat. I happen to be joined today by a first-time guest on a podcast. So this’ll be an experience for him. It’s Mr. Toby Corbin of IBM. Toby, welcome to DevOps Chat.

    Toby Corbin: Thanks, Alan. How are you doing?

    Shimel: Doing great, and happy to have you on today’s show, Toby. Rather than introduce you to the audience in terms of your title and role and everything, I thought it would be a good icebreaker for you to kind of introduce yourself. So why don’t you kind of share with the audience, if you will, what your title is and what that means and kind of what you do with IBM?

    Corbin: Sure. So at the moment I’m Technical Lead for one of the squads in the Microclimate Project. Microclimate is our developing experience, which we’re targeting to help onboard customers and developers into the IBM Cloud Private. So my role and my squad is – we sort of deal with the backbone of Microclimate. So we deal with making sure everything runs smoothly. We add in things like analytics and the sort of analysis application. But we make sure the whole thing sort of runs. So we’re kind of the backbone. So the squad that I lead, we’re the backbone of Microclimate.

    Shimel: Excellent. So Toby, this sounds sort of like an Ops role, if you will, right? When we talk about the Microclimate – and you know what? Let’s back up. Some of our audience out here may not even be familiar with IBM’s Cloud Private offering. And I don’t claim to be an expert. I’m going to assume you know much more than I.

    Corbin: A little more.

    Shimel: Yeah, a little bit. I would hope so, so you’re not collecting your check under false pretenses, correct?

    Corbin: [Laughs]. Correct.

    Shimel: So Toby, let’s start with that. How would you describe Cloud Private to our audience?

    Corbin: So we’ve all heard of public cloud – things like Amazon have a cloud and Google have a cloud and IBM also has their public cloud. And these are quite large deployments. They’re hidden in the ether somewhere, no one knows where they are, and that’s where all the really, really clever analysis goes, and that computing power that IBM can bring. Cloud Private is sort of the onboarding into that. So Cloud Private is a cloud specifically targeted at individual customer. So they won’t be sharing that with anybody else. It will be an instance just for them and they’ll have the very similar applications and things on there that the public cloud would have. The intention being that when our customers are happy using this Cloud Private installation – so they don’t have to have their own machines. They can have the Cloud Private stuff working for them, doing all of their processing. When they’re happy having that working, we can then take them up to the public cloud, where they get even more of IBM’s processing power and they have an even better experience.

    So it’s kind of taking them from a private cloud that’s just for them up to the bigger public cloud, where although it’s still for them and they don’t share with other people, they have bigger computing power available to them.

    Shimel: Got it. So is it sort of a way station along an upgrade path? Or can someone stay on Private Cloud indefinitely?

    Corbin: Oh, you can absolutely stay on Private Cloud. You’re not forced to go up. But obviously as we go up through the clouds, the capabilities IBM can offer you get a lot greater. So that’s why often you’d find that the customers would want to move up to the next level, because there is an even greater amount of processing and a greater amount of options available to them and to their customer deployments.

    Shimel: So they grow into it, I guess. So Toby, I was under the impression, and maybe I’m wrong, that in Private Cloud there was also – I thought it was a Kubernetes-based offering. Is that –

    Corbin: That is correct, yes. So IBM’s Private Cloud is all based around Kubernetes. It’s IBM’s flavor. So it’s Kubernetes under the covers and IBM have made it sort of into their ICP, their private cloud offering. But it is Kubernetes-based, that’s correct.

    Shimel: So it’s containers with Kubernetes and this whole sort of – you know, the _____ _____, obviously, but now we’re starting to see the domination almost of Kubernetes kinds of architectures.

    Corbin: Yeah, and that’s how IBM’s helping our customers. So you think of the old way of doing things, you’d have a _____ application that was a huge monolith, a giant application that couldn’t really go anywhere. With IBM’s Private Cloud what we’re looking at is, how can we take these applications, deploy them on the cloud, free up all of the hardware – in a customer’s site they don’t have to worry about those things – and take these large-scale applications and deploy them into a cloud environment? And then it’s not just make them work in a cloud environment. How can we work with customers to actually make them change their applications and sort of rebuild them, if you like? Strip them down, put them back together again, as a microservice opportunity?

    And that’s where Microclimate comes in. That’s what we’re doing for the developers. It’s how you create those microservices.

    Shimel: Perfect. And of course, Toby – I think a lot of our audiences are familiar with the term “microservices,” and we see microservices being mentioned a lot around Kubernetes, around containers, around DevOps and cloud and so forth. And you anticipated my next question, which was, when we talk about a microclimate, is that sort of the climate in which we do microservices or perform microservices.

    Corbin: Yes. Well let’s say for this, yes. Effectively what microclimate is, it’s the environment that we want to give to our developers to go from having absolutely nothing to being able to develop their very first microservice, test it and then deploy it up into production through things like the Jenkins Pipeline and using Travis, and get it running in their cloud, all without actually having to need any prior knowledge. We sort of do this all for you. So in the past there was always ways of being able to create an application and there was always ways of pushing an application into a cloud, but it was several steps, different technologies. You had to rely on the customer having various knowledge themselves. So Microclimate is aimed to make this really seamless. You don’t have to leave your web browser; it’s entirely web-based. But we give you the ways to easily create an application, develop it in the cloud in a web-based editor, test, adding in these performance issues that we’ll talk about in a second, and then when you’re really happy, then get it run in production environments, all with the under-the-covers work being done for you by Microclimates.

    Shimel: That’s fantastic. So this whole Microclimate, if you will – excuse me, microenvironment and Microclimate, this is all managed via web browser? All web-based, huh?

    Corbin: Exactly. Because this gives you the opportunity that if you are fully running in the cloud, you don’t actually have to have anything on your laptop at all. Everything can be based in your ICP installation. You need nothing on your laptop to run it, which means then you don’t have to have Swift installed or Node installed. But you can still go and create and generate Swift and Node and _____ applications without having anything local to your machine. It frees up your environment to do everything in the cloud, so you’re straightaway using the power of the cloud to do your development.

    Shimel: hat’s fantastic. And Toby, it sounds to me, even more than that – you know, so I’m old, right? So I’ve been around for a while. And wen we first saw microservices and containers, to me this was like Heroku – better than Heroku. It was truly moving from infrastructure as a service to platform as a service. So that if you’re a developer and I want to write stuff let’s say in Swift or JS or whatever, I didn’t have to worry about, “Was that installed? What was the infrastructure like? What was going on? Was it installed in the OS level?” I could work directly on my application. Do these sort of Microclimate, microenvironments – is that what this really represents?

    Corbin: Absolutely yes. So that’s one of the beauties. Before coming on to work on Microclimate, I was doing Swift developments and Node developments, and I would have to have the latest versions of Swift on my machine and I had to keep up with them to make sure that I would have the latest bug fixes done or whatever it was to make my application run smoothly. With Microclimate, we take care of that whole environment for you. So when you say, “I want to create myself a Swift application,” you get a container running with the latest version of Swift. All of the security patches for that run are already applied. You don’t, as a developer, have to worry about, “Do I have the right installer? Do I have the right version of he compiler?” All taken care of. As a developer you can now just write code and we will do everything else in the background for you, so it really frees up the developer to do their job and not have to worry about keeping their machines up to date with all of the latest patches and run-suns and everything else to do that.

    Shimel: That’s pretty nifty. Good stuff right there. So then, Toby, as part of your – you know, you’re a team lead with portal, you had mentioned, and – but you know, but your whole team and what you’re doing – would you say most of the people you’re serving are developers? Or do you kind of correspond with their Ops team, who are still servicing the developers within their organizations?

    Corbin: So this is where Microclimate has nicely evolved. We started off s being the developer experience. So we were the developer experience _____. And we were aimed at, how do we nicely onboard the developer from having nothing on their laptop to being able to write their first microservice and deploy it to the cloud? But then we quickly realized that we need to have DevOps. So we brought on a whole DevOps pipeline to the backend of Microclimate. So Microclimate is really _____ _____ as an end-to-end story of two either distinct parts, if you like, but they actually work in one. You start with, “I’m on my laptop, even though I’m running in a cloud environment.” So I’m coding on my laptop and I’m writing my first microservice. And then when I’m happy, I’ll use Microclimate to actually send it to my preconfigured Jenkins Pipeline, and the backend of Microclimate that deals with DevOps will take that and will build you on your Jenkins Pipeline, it will push towards your production and you’ll actually go the full end-to-end now.

    So Microclimate is not just really about writing your first piece of code now. It’s taking any application and pushing it right through to deployment at the other end and have it run in production.

    Shimel: It’s end-to-end. That’s beautiful, man. That’s really nice. So share with our audience, if you can a little bit, what are the kinds of things that you and your team are actually doing as part of maintaining this environment?

    Corbin: So one of the things that we brought – so as I said before, my background in this, I actually came from doing tools monitoring for things like Java and Node. We now bring that to Microclimate. So we can think of things like a performance issue that you would typically see. You know, my CPU is going mad and it’s at 100 percent, or my disk is thrashing or running out of memory. And often it’s very difficult to know why that’s happening or even if it’s supposed to be happening. So what we do, what we bring to Microclimate is, we bring to the table our analysis tooling that we have outside Microclimate. So when you use Microclimate, we actually build in our monitoring stuff. So that means that when you write your very first line of code, we actually start monitoring it straightaway.

    So you don’t even have to think about, “Okay, do I have to monitor my application for performance problems?” or, “I’ve got my application written. We normally have a performance team that does this.” We give it all to the developer. Because if a developer can see a problem with a line of code they just put in – for example, they’ve just coded a new function and when they run that function their fan goes mad on their MacBook and it’s thrashing away and they think, “What have I just done?” If we can tell them what they’ve just done at the point they do it, this speeds up development in the future and your testing gets quicker, because you don’t get a performance team two months down the line saying, “Hey, we’ve just run your stuff and it performs like a dog,” and then it comes back to the developer to have to re-find-out what they’ve done. We show you straight away. As you type your code in and hit Run for your first test, we can show you what performance was, so you can see, am I regressing? I can see where my CPU is going mad or I can see where my hard disk is lighting up or I can see where I’m running out of memory.

    We build that in for you. So hitting the developer experience right from day one means that hopefully when you get to the end and your application deploys, there’ll be no surprises around, “Well it’s not running anywhere near as well as I thought it would.” So that’s what we bring to Microclimate.

    Shimel: Got it. Like you’re the developer’s best friend.

    Corbin: We’d like to think so, yes.

    Shimel: So Toby, as I mentioned before, I’ve been around the block a little bit. You know, one of the problems with what you’re describing is territorial issues, let’s call them, right? “Hey man, I’m the Ops guy here from my org, and I’ll help my developer, thank you. Don’t be giving away our secrets, telling the developer some of these performance things.” You know, performance kinds of insights. Do you run into having to deal with pre-existing performance teams as part of the Ops at these organizations?

    Corbin: Well not so much, because we actually find now is that the sort of dynamic of a team is changing. As we go into this sort of more agile world where each team is intended to be self-directing, we find by having smaller teams with sort of jack of all trades, we get away from this developing, developing, developing, testing, testing, testing situation that we _____ _____ five or ten years ago. And this means that much smaller teams can deliver functionality that’s fit for purpose a lot quicker, because you’re not waiting for a longer test run at the end or other sort of testings that happen that would typically end up in this loop of sort of, “I deliver my code, someone tests it, it comes back to me to make changes, someone tests it and it comes back.” This is a sort of way development has now evolved. So with the agile world, smaller teams can do things a lot faster, and that’s why giving them the tools they need means they can do that.

    But this doesn’t mean we put testings out of business, as it were. Because we aim our stuff at the developer, for like the piece of function they’ve just written or for what that application is doing as a whole. There will always be a need for sort of larger-scale teams that need to oversee the entire application. So whereas before, if we take the example of the giant _____ monolith that was one giant application, you might find in today’s environment that that’s actually seven, eight, nine, ten small microservices. So a developer would be working on one microservice and he’d get the performance stats back from that microservice in isolation. But what about the bigger picture? What about when all ten of those are working together? How do you get the overall view of your microservice environment that’s 10, 20, 30 things all talking to each other? So that’s where you still have DevOps teams and other teams working on that. And there’s software for that as well.

    So as an example, there’s Prometheus, that’s an open source monitoring solution that’s very good at taking data from different sources. And then what we do with the stuff that we add into the applications at the developer level so that the monitoring solutions we put into Microclimate for the developer, that will actually talk to Prometheus. Which means that if the developer is happy seeing their code running and their single microservice view of the world all looks good – “Yes, my database function is performing as I expect it too.” I’ll now put it into the cloud where there’s 20 other microservices running, all talking. And Prometheus, that sits over and above all of this, can now take the data from each of those 20, and the data they’ll be taking will be the same data that the developer has already been seeing when running it locally or in the small view. We just pass it up through. So this again gives the developer an easy way of seeing – or actually in fact _____ ____, the SysOps guys, what their applications are doing without any additional changes. Because the code has been in there from day one to provide this information.

    Now that’s one of the hardest things about monitoring. It’s, how do we get at the data? If you can get at the data, there’s plenty of ways to visualize it. All sorts of fantastic tools, even the simplest of just writing out to the screen. It’s always the case of, how do we get to the information? So we build the information in from day one, and that’s what makes Microclimate so cool, as it were, because it’s just there for you. And no matter what sort of environment you run in, we’ll be able to make the data available to see the performance at any level.

    Shimel: You know, Toby, that’s excellent. Hey, when we started – before we started recording I told you, “You’re going to be surprised when I tell you, ‘Hey, we’re going over time.’”

    Corbin: [Laughs].

    Shimel: So we’re really way over. That was a difficult interview. It was pulling teeth to get you to talk to us.

    Corbin: [Laughs]. I’m sorry.

    Shimel: No, not a problem. Toby Corbin, IBM’s Microclimate Developer Experience Tribe, thanks for being our guest on DevOps Chat today, and continued success with the Cloud Private team at IBM.

    Corbin: Thank you very much for having me on, Alan.

    Shimel: My pleasure. This is Alan Shimel. devopsy.kinsta.cloud. You’ve just listened to another DevOps Chat.

    — Alan Shimel

  • DevOps: Making Developer Experience Reality

    DevOps: Making Developer Experience Reality

    As DevOps professionals, among other things we are responsible for making developers’ lives a little bit easier. How many times have you heard, “Do you have a working vagrant box?” Or maybe you need to set aside a few days to onboard a new developer.

    Many times a development environment is created when a project is started, usually on a developer workstation. This typically means it will work on that laptop. But, as the project evolves and more engineers get involved, the need to share this environment becomes apparent. When the project grows in complexity, requiring more components and third-party services, this organically grown developer environment quickly becomes unmanageable.

    Get Ready for Production

    Another interesting aspect of the developer environment is the level of similarity with the actual production environment. Even evaluating this aspect requires that you have insight in what production looks like. If projects start on a developer laptop, the desired production setup is seldom considered. More times than I care to remember, I was called into a project when the “thing” that worked well on the development teams’ laptop had no sensible production deployment format.

    This teaches us that even if we start out small, we must think about how the project will function in a production environment and how we are going to get it there. In practice this means that any new project requires at least the following:

    • Some unit tests, they may even start out as stubs.
    • A minimal deployment target environment.
    • Continuous integration (CI), running the (stubbed) unit tests.
    • Automated delivery, preferably continuous deployment (CD).
    • At least one simple integration test, possibly just a curl to the deployment app to verify it is alive.

    This does add some complexity to your project startup, possibly beyond an individual developers’ abilities. It does, however, lay the foundation to, at the very least, be able to deploy the application somewhere. For a seasoned DevOps professional, adding a basic CI/CD only takes an hour or so and there are plenty of free cloud resources for small projects.

    When the project grows and the development environment gets more complex, things get a bit more tricky. The above steps will not really help you mitigate the possible frustration of keeping all the moving part inline. A pattern I see often is that developers try to spin up the entire production stack on their laptop. For a containerized environment, this might mean a Docker Compose-based setup or even minikube to run a local kubernetes cluster. These tools often are using shared volumes for quick iteration, which work well until a large number of files need to be kept in sync. Vagrant is a great way to quickly spin up an environment. This does offer some level of production parity, but certain parts are missing: The CI/CD system watching commits to your source code repository is not deploying to your local setup and traffic to and from it are not passing any additional infrastructure such as load balancers or firewalls, to name a few.

    Local-Remote

    When things break in production, “It it works on my laptop!” is a phrase we have all heard at some time or other. What if we could bring the laptop environment a bit closer to production? It’s not the developers’ fault if things break in production but work well on the local workstation. Providing an option for developers to develop iteratively using a remote infrastructure means we can get closer to production parity. Also, it means that onboarding new developers in the team could be simplified significantly because all the complexity is in the remote environment. This remote environment is provisioned in the same automated way as production, so we have a great degree of confidence about its state. For Kubernetes-based environments, many great tools exist including Skaffold and Draft. They both provide an easy way to build, push and deploy applications to a Kubernetes cluster. This makes it easy for the developer to take on the responsibility to ensure the application under development will run in the target environment.

    If, for some vague reason, you are not containerizing you applications but run code in a virtual machine instead, there are also options. Most modern editors provide automatic syncing with a remote server, either on-demand or when code is changed. Ksync is a tool for syncing your local code into a pod in a Kubernetes cluster on file change. Syncing code live removes the delay of rebuilding a container image and uploading it every time you want to see it in the deployment environment. As such, code sync tools are better suited for iterative development.

    Recently, I worked with a client setting up Telepresence  for a development team. This is also a tool for Kubernetes environments but it offers a twist—developers get to run their code locally with all traffic in and out proxied through a remote Kubernetes cluster. Like with sync tools, this means there is no more long delay before viewing changes, IDE debugging works as normal and all the log output is local as well. When code is committed and pushed, the CI/CD system should update the cluster with the latest version, which in turn can be used to iteratively develop on with Telepresence. It’s a relatively young project but offers great potential. An added advantage is that only a small portion of the entire application stack or landscape needs to run on the developers’ laptop, which means it can do with less memory and a cheaper CPU.

    Conclusion

    Containers and microservices are more widespread than ever, making it increasingly difficult for developers to have a production-like environment to develop in. The key to solving this problem is to not try to load everything into an evermore powerful laptop, but splitting the load and complexity between developer workstation and a remote environment.

    — Alex Knol

  • Collaboration: The Glue for Test Automation Maturity

    Collaboration: The Glue for Test Automation Maturity

    QA’s role in DevOps continues to evolve in positive ways. In a recent survey on test automation conducted by QASymphony and TechWell, nearly one-third of organizations reported that among organizations where testers and developers share testing responsibilities, about a third of test cases are automated. This survey and others show that on average, fewer than 20 percent of organizations have adopted test automation.

    What we can discern here is that when testers and developers work closely together, organizations can achieve greater levels of test automation maturity. This is important, because test automation ties in closely with the primary goal of DevOps, which is to develop high quality software, rapidly and reliably. There’s no way to reach velocity at scale without test automation maturity—and QA teams play a critical role in getting there.

    Let’s break this down a bit further. How does developer-tester collaboration contribute to test automation maturity, and what exactly do we mean by that?

    Once developers complete unit testing, their job is to communicate the findings of those first level tests and code verifications to the testing team. If they don’t, testers are somewhat in the dark when determining what other tests—whether functional, API or performance—to build from there. Developers have the first line of vision on problem areas in their builds. It’s not enough for testers to look at a dashboard to see that a code branch has passed a test. Test automation requires speed and accuracy, so the more detailed information testers receive upfront the more successful they will be in writing automated tests and in determining manual testing needs.

    Developers should also automate unit-level testing. DevOps requires more tests, more often; everyone should take advantage of the efficiencies that test automation brings.

    Making Collaboration Work

    How to exchange information is less important than just doing it. There are lots of options these days: Developers and testers can share information back and forth within DevOps code repositories and pipeline tools systems, such as GitHub and Jenkins, or by using collaboration tools such as Slack. Geographically-dispersed teams may prefer web conferencing systems to facilitate live discussions on the fly.

    Team members can sit side by side, doing what’s called paired development. It’s powerful for testers to watch the code as developers write it so they can ask questions during the process. That gives the tester more knowledge about the product and in turn, developers understand the tester’s objectives. The point here is that achieving a high level of test automation coverage isn’t easy; testers and developers are both learning new skills and processes. Through frequent exchange of information, they can make progress faster and always be on the same page.

    Clearly though, with such low average test automation adoption levels, we need to work on closing the collaboration gap.  Here’s how I see the barriers:

    • Developers are fiercely independent. While that’s not a surprise, it requires incentives and education to help developers understand the need to get out of their silos. Developers must adopt test automation and talk to the testers. If they don’t adapt, they won’t last—at least that’s what a CIO told me recently.
    • Testers and developers have been in opposition for too long. It’s time to get rid of this “us versus them” mentality. Developers sometimes think testers are just trying to break their code and criticize their work. Testers sometimes complain that developers could care less about writing buggy software. This is a cultural problem and leadership can help resolve it by promoting software quality as an organization-wide priority for which everyone shares responsibility.
    • Testers and developers don’t always speak the same language. The thinking behind how developer writes code is not the same as how the tester writes automated scripts. Developers don’t consider all the scenarios a tester might when they write code. As a result, they may not write clear object identifiers in the code which help testers write scripts.

    Overcoming these barriers comes back to collaboration. Developers and testers need to find ways to understand each other’s world view. Begin by obtaining a joint understanding of completion. If developers and testers can agree on the release criteria, meaning that testers are comfortable with the level of automation in place to support future releases, it avoids a huge backlog of tests to automate later. Backlogs lead to delays and frustrations all around. Using a common development language for coding and test automation is also helpful; otherwise, it’s more difficult for both sides to speak the same language when working together. In the same vein, testers should have access to all the tools that developers use: source code repositories, continuous integration, project management and so on.

    We live in a digital world and have endless instant communication options, but even on a software development team, there’s nothing that replaces the value of a face-to-face conversation. It’s going to take more than one tactic to make collaboration work in your organization. Keeping the lines of communication open should be a central goal to increase test automation coverage and streamline the DevOps release process.

    — Ryan Yackel

  • 5 Tips for Creating Blockchain IOS and Android Apps

    5 Tips for Creating Blockchain IOS and Android Apps

    Blockchain development and blockchain technology are playing a major role in the creation of various iOS and Android apps. Blockchain technology has seized the attention of many and when it is utilized properly, mobile app developers are able to achieve results that few thought possible.

    There are no shortage of benefits provided by blockchain development. Data security levels are increased and unnecessary intermediates are removed from the process. Financial organizations have been taking an increased level of interest in blockchain technology in recent years and this creates added opportunities for app developers.

    So, how can app developers properly capitalize on the opportunities that are created by blockchain technology and blockchain development? This guide is here to help. Here are some tips and pointers that are designed to help developers create iOS and Android apps that take advantage of blockchain technology and blockchain development.

    Focus on the Interface

    Blockchain development can be used to create all sorts of valuable iOS and Android apps, but for these apps to reach their full potential, developers must remain focused on the interface. Not considering the user’s experience during the development process could lessen the level of impact of the app.

    Is the front-end programming language operational? Is the blockchain application that is necessary for operating this language included? Has the developer considered the importance of an external database? Is the right application control being included during the development process?

    These are all questions that need to be answered so the app developer can create a quality design for the user interface. Identifying the proper analytics is also crucial. Otherwise, it becomes more difficult for developers to identify the right system. Building an admin console is a key step of the process as well.

    Take a Closer Look at the Importance of Architecture

    If a blockchain app is being created to be used on an iPhone or Android device, developers must take a closer look at the design architecture so that they can avoid common mistakes. Unfortunately, many aspects aren’t always considered, and they often involve additional processes that can add extra time to the process.

    For example, those who are planning on creating a hybrid app will need to obtain certain permissions first. The more developers know about the necessary parameters that have been established, the better their chances of creating an app that will be useful to those who rely on blockchain technology daily.

    The processors, size, memory and operational systems must be configured properly, and these configurations all fall under the architectural umbrella. Great design is crucial, and app developers who aren’t willing to consider the most vital components will come up short when it comes to their design architecture.

    Proper Platform Utilization

    An app that relies on blockchain development and technology needs the proper platform to be successful. This platform should be unique yet accessible. When issues come to the forefront (and they are going to, despite your best efforts), developers’ ability to solve these problems will be determined by the chosen platform.

    For issues to be solved, a consensus must be reached, which is easier when the correct platform has been selected. Now that more platforms are openly sourced, this makes life much easier—developers have the chance to select the platform that best aligns with all of their needs and desires.

    Whether a developer decides to choose Ethereum, Quorum or a different platform entirely, it behooves them to take the time to study the advantages and disadvantages of each, especially when blockchain technology is being utilized.

    Recognizing the Importance of Consensus Mechanisms

    Is the application being decentralized? This is one of the main questions that must be answered before this process can be completed. An iOS or Android app that is reliant on blockchain needs a consensus mechanism to work in the proper manner once it has launched.

    Without decentralization and a consensus mechanism, many of the typical problems and issues that arise are not as easy to solve. The system used to connect and provide a link needs a proper consensus. Without a consensus mechanism, the system’s ability to carry out all of the necessary tasks is severely compromised.

    Apps that are related to blockchain must be developed with the right approach. Are the necessary considerations being made? Is the best method being chosen for delivering proofs? Are the basic requirements being met when these proofs are delivered? A developer looking to take advantage of the current climate needs to be able to answer all of these questions.

    Identification of Objectives

    Last but certainly not least, developers must identify their objectives. What is the goal they are trying to accomplish by creating this app? What role is blockchain technology going to play? Are all of the necessary blockchain development principles being adhered to? Is the issue that is going to be solved being made perfectly clear?

    Answering these questions in a timely manner allows a developer to decide on the correct course of action. The same rules that apply during the creation of any other app still come into play when blockchain apps are being developed for iOS and Android users. The problem that is being solved by the creation of the app must be clearly defined.

    Are the issues being caused by data loss? What resources are being provided that were not previously available? To get the most out of the development process, the issues and the associated goals must be examined closely.

    — Harnil Oza

  • Rigidity or Agility During Change Processes: You Do Have a Choice

    Rigidity or Agility During Change Processes: You Do Have a Choice

    As most who work in service management, on an IT team or as a member of the service desk understand that change management can be a rather rigid, shall we say, still process. Do you want to make your change implementations more agile? There are different ways to go about this. In this piece, I plan to discuss how to implement a single change in an agile way, and how to make all your department’s future changes more agile. Let’s first begin with how to make the implementation of a single change more agile.

    Describe Your Goal and Preconditions

    Ideally, when necessary changes are required, you shouldn’t need to complete a full change request form when submitting the request. The description should only contain basic information regarding the change. Most importantly, what must be included are your goal and the required preconditions.

    Here’s an example: You receive multiple calls about a coffee machine breaking down, or an email server running slowly. As a service desk representative, you’d likely prefer not to receive and handle a bunch of calls about the same problem, but you do want your customers to be able to know whether or not a problem has already been reported and is being dealt with. Your team’s goal, then, is to show your customers that a ticket already has been submitted for the problem and that you are working on it, but you also want to make sure your customers don’t get to see the private information of other customers—that would be one of your preconditions.

    Apart from your goal and preconditions, there’s not much more you need to know when you get started with a change.

    Check Regularly if Your Change is Still Meeting the Pre-Set Goal and Preconditions

    Implementing changes according to the waterfall method can mean there’s a chance that, when done, the solution doesn’t meet the customer’s needs. The waterfall method likely means you want to execute your plan without alterations along the way. Under this model, there’s no room to adjust to new situations even if you need to make changes. You may find new information about what your customer really wants, or there could be new technologies on the market providing a better solution to your perceived problems.

    So, while you’re designing and implementing a change, regularly check to see if what you’re doing right now still contribute to the original goal, and whether you’re still meeting the pre-conditions. As you know, agile processes lets you make adjustments along the way. But perhaps you’re already past the point where you could have made a change. If that’s the case, in an agile workflow, you can change course and simply work the change.

    Involving Others in Your Change

    Once you’ve got an idea of how you want to implement a change, make sure to include your team’s experts so they are able to take a detailed look at your plans and determine the impact of the change on other applications and hardware. Additional points to consider include analyzing any change that may lead to security risks.

    When following the agile philosophy, it’s better to let your team come up with its own solution, but if the team figures everything out for themselves, you want outside members to perform the quality check. As the saying goes, freedom is not free. For IT teams, the freedom to come up with a solution means they also must be responsible for determining whether the solution will work as planned.

    Run As Few Changes As Possible at the Same Time

    IT departments almost never work on more than one change at a time, even those that are small. Unfortunately, there can be too many changes to the handle. To keep the number of changes in check, limit the number of changes that need to be undertaken at any given time.

    Doing so makes it easier to predict the duration of changes. Working on fewer changes at once means you’re working with fewer variables, so you can better predict when you’ll finish a change.

    Also, the fewer tasks in operation, the more likelihood of the service being higher. If you’re able to focus more on the change you’re currently working on, you don’t make as many mistakes. Finally, with fewer tasks you’re able to focus on your work and quickly generate results. Visible progress is one of the most important factors that determine happiness at work and getting things done in a timely manner is more motivating than constantly toiling on a number of different things with nothing ever getting finished.

    Managing Changes vs. Incidents

    When there’s a constant flow of incidents coming in, your service desk employees must make the decision about your team’s priorities and what they are going to be. For example, do you want to guarantee a maximum time for required changes? To meet outcomes and objectives that are set by this team, you must dedicate a set amount of time that each team has to spend on said changes. This means you must be able to accept that some of your reported incidents won’t be addressed immediately.

    Alternatively, if it’s more important to solve incidents quickly, you need to accept the time required to implement these changes, and an understanding that your team won’t be able to handle new changes as quickly as you might hope to. Change requires time, even if time to solve incidents is your biggest priority.

    — Nancy Van Elsacker Louisnord

  • Using Laravel and AWS: What You Need to Know

    Using Laravel and AWS: What You Need to Know

    Most likely, you have heard about and Laravel, the open source PHP framework, and you might be planning to go to the next level to modernize your PHP Laravel application. AWS offers a number of features to help organizations scale to support millions of users, not to mention the agility and cost benefits associated with a cloud infrastructure. These features and capabilities are the compelling promoters to transform your application into a high-performing and resilient PHP Laravel application.

    In this article, I will describe how to architect your software-as-a-service (SaaS) Laravel application with high-level scalability standards on AWS to help you understand what is required and discuss the considerations and action items to gain business value and resiliency. We chose the AWS cloud since is the most mature infrastructure-as-a-service (IaaS) provider and makes things easier to innovate; however, these practices can be applied to any cloud platform.

    I also will explain why AWS is a vital option to scale and grow your SaaS Laravel application and present the current/future enterprise architecture. In part two of this series, I will discuss the importance of the 12-factor app methodology and multiple lessons to include in your strategy when architecting on AWS DevOps.

    The Value of Migrating Your Laravel App to AWS

    In an article I wrote for my company blog on running the digital transformation on AWS for your enterprise or web application, I explained the business benefits that brings the AWS cloud adoption, including a reduced IT labor force, pre-configured services to achieve faster implementations and minimal downtime, due to the repeatable and flexible infrastructure. The cloud is an essential environment to scale SaaS Laravel applications—in there, you can decouple your architecture into multiple service pieces, including cloud storage (Amazon S3 and CDN) and automatic scaling EC2 instances (AWS Auto Scaling), and integrate various SaaS/platform-as-a-service (PaaS) AWS components for your modern, cloud-native application. Also, if you are planning to move to the AWS cloud, you could use this AWS Migration Checklist to prepare for your journey.

    “Web Applications and enterprises need to transform, they need to transform to a continuous transformation, and the cloud is the key.”

    – Stephen Orban

    I really enjoyed this phrase from AWS’ Stephen Orban, because you’re not only transforming your application; once on board the AWS cloud, you are in an infinite journey of continuous innovation: pay-as-you-go, microservices, serverless and so on.

    Laravel Architecture on Docker and Microservices

    Throughout the AWS cloud journey, there have been multiple disruptive evolutions, and one of them is the microservices, although, I won’t cover the microservices (Docker) approach. As a first step to transforming your application toward a cloud-native application, start with a monolithic approach. As your application matures and user base increases, I suggest jumping into a Laravel microservice approach, i.e., dockerize your Laravel PHP product using Docker, AWS ECS, Kubernetes or AWS Fargate. I’ve found many digital agencies that are adopting the AWS Cloud and DevOps practices with Docker/Serverless strategies and are getting overwhelmed with these sophisticated technologies, which has led to project failure because they were not ready to this stage.

    Here’s how to avoid that: Grow gradually.

    However, if your Dev team has enough experience with Docker and microservices, I wouldn’t hesitate to architect with microservices. This will be covered in the next article in this series, “Microservices with Laravel, Docker and AWS ECS.”

    Who is This Article For?

    Whether you are kicking off an MVP app on Godaddy/Hostgator, deploying Laravel on AWS with basic principles or ready to jump from hundreds of users into million users, this article is for you and will help you in your Laravel architecture on AWS.

    Laravel for Large-Scale SaaS Architecture on AWS

    So, let’s start by making clear what do you most need to scale a Laravel application on AWS.

    The diagram below illustrates how a next-generation Laravel application should be architected with the full range of AWS DevOps components. Each piece mentioned in this AWS stack is highly elastic, scalable and resilient.

    AWS Laravel Architecture Diagram

    AWS Auto Scaling and Load Balancing

    Initially, the dynamic traffic comes from AWS Route53, which will route all requests to the Application Load Balancing, which the AWS ALB will load balance across a fleet of EC2 instances ( 2…N Instances). This fleet should “scale” horizontally according to the traffic demands or the AWS CloudWatch metrics.

    Amazon S3 and CloudFront CDN

    All static content including images, videos and HTML will be hosted on Amazon S3, the cloud storage with infinite storage and elasticity. In front of Amazon S3, we will compound AWS CloudFront to cache the entire static content and reduce bandwidth costs. Integrating this pair of components is crucial.

    AWS RDS Aurora (Managed Database Services)

    The database data should be placed on AWS Aurora or RDS and AWS ElastiCache (Redis) for session management and cache user data. These AWS components are elastic and scalable, which helps to design the AWS Well-Architected framework with the highest fault-tolerance and resiliency.

    Amazon VPC and Networking

    In the AWS Networking level, we propose AWS VPC with a public/private enterprise network. Following PCI and HIPAA practices, there should be a VPN encapsulating the Laravel application within the private AWS enterprise network. Also included should be an AWS VPN connection and OpenVPN service to access the private network to your multiple Laravel environments.

    Route53

    From the DNS perspective, Route53 is needed to manage the application domain, DNS scalability, routing latency and more features to reduce failures.

    AWS Lambda

    This serverless component will be used for async jobs and backend scripts to minimize load in your AWS Auto Scaling cluster.

    We will explore these AWS components substantially in detail, later in the third article of this series.

    Web applications that can fit within this architecture:

    • API Laravel applications
    • Mobile backend applications
    • Laravel enterprise applications: CRMs, ERPs and back-office systems
    • SaaS Laravel applications
    • Front-end applications. i.e., Angular, React, Vue.js or any HTML site
    • ANY web platform: Python, Node.js, Ruby, PHP, Zend, CakePHP, Yii and Symfony

    A word of warning: Don’t use Cpanel for any web application. It is not recommended to scale as it can slow down your application performance.

    — Alfonso Valdés

  • 4 Ways to Tackle Common Test Automation Pitfalls

    4 Ways to Tackle Common Test Automation Pitfalls

    The demand for higher-quality products and new features delivered at breakneck speeds has the average engineer struggling just to stay afloat. With demands rising but resources not always following suit, engineering teams are embracing automation as a key testing option.

    According to a report by advisory firm KPMG, test automation increased by 85 percent from 2015 to 2017 across all industries. Automation is one way to address this accelerated development schedule, but it can also introduce other problems. To ensure that doesn’t happen, there are four common pitfalls that engineering teams need to look out for.

    Problem No. 1: Right Job, Wrong Tools

    It’s tough to build a shed with a spatula and a sewing needle. Likewise, you can’t build a successful test automation suite if the tools used don’t fit your company’s testing strategy.

    The solution: Pick the right framework for your suite.

    The first question you need to ask yourself is: Should you build or buy your test automation framework? The answer likely depends on how mature your testing structure is. Smaller teams new to the technology typically are best-served starting with an off-the-shelf enterprise solution. This approach makes it easier to get up and running quickly, while having the support to handle software bugs along the way.

    Larger, more mature testing teams may have more to gain from an open source automation framework. This cost-effective approach provides flexibility and customization, but often requires advanced technical expertise and the luxury of time to build such a solution.

    Problem No. 2: Living Fast, Dying Young

    Many automation projects never make it off the ground because teams move too fast and automate too much, too soon. Inserting automation into the delivery pipeline prematurely will cause workflow and build issues more often than not.

    The solution: Build test automation into your pipeline incrementally.

    It may be tempting to run full speed as soon as you are set up, but the truth is you need to take your time for developers to build confidence in the testing process. Once that confidence is established and you’re receiving valuable feedback, you will have the full team’s support as you continue to scale your automation strategy.

    Problem No. 3: Automation Without Documentation

    Automation can make testing easy. Sometimes too easy, to the point where engineers will simply kick back a failed test without looking into the reason it failed in the first place. Oftentimes, these failures are the result of an outdated testing script or an issue with the test environment, and not with the code at all.

    The solution: Triage automation test results just like manual bugs.

    Instituting an intermediary triage stage will save your engineering team time in the long run. There’s no need to treat your processes for automation results any different than you would bugs found manually. Having access to materials such as activity logs and screenshots will only help to strengthen and speed the triage process.

    Problem No. 4: Setting and Forgetting

    Automation makes testing quicker and easier. That’s why you invested in it in the first place, right? Unfortunately, there is more to such solutions than just plugging them in and letting them run.

    The solution: Take maintenance into account when building your framework.

    Many teams are surprised by the maintenance required to keep test automation suites running smoothly and effectively. They shouldn’t be. Solutions need to be updated continually to address user interface changes, new operating systems and browsers, and automation library changes. The maintenance of third-party dependencies also needs to be considered before automation is used for production test runs.

    Maintaining your automation suite is a continuous process. Make sure you have enough testing resources—either in-house, outsourced, crowdtesting or a combination of all three—to ensure your efforts don’t go to waste.

    Measure Twice, Implement Once

    Make no mistake, automation is more than a quick fix. If you are going to spend the time and money necessary to build a successful strategy, you should make sure to set yourself up with a solid foundation. A framework that is thrown together within days but falls apart in hours will only serve to make automation seem like an unpalatable solution within your testing department. Having the right partners in place and the right solution from the start will help automation become a reality in your company, and can bring the promises of speed and flexibility to life.

    — Drew Horn