devopsy.kinsta.cloud

Category: Doin’ DevOps

  • Best of 2021 – 6 Signs You’re Doing DevOps Correctly

    Best of 2021 – 6 Signs You’re Doing DevOps Correctly

    As we close out 2021, we at DevOps.com wanted to highlight the most popular articles of the year. Following is the twenty-first in our series of the Best of 2021.

    A lot of organizations claim to be DevOps-focused because they use specific tools and loosely follow certain practices. But to really be a DevOps-centric organization, there are specific DevOps principles and processes that ensure your organization is as optimized and automated as possible. To ensure that you’re not just using the DevOps title without following the underlying principles as an organization, here are six signs you’re actually doing it right.

    Deployment Is Fully Automated

    Having deployment completely automated from start to finish is one of the more apparent signs that you’re following the DevOps principles correctly. Automated deployment is a huge time saver, especially when things go wrong and you need to revert to a previous build version quickly. Manual deployments can lead to problems often caused by human error. Having a strong focus on automating processes is a core belief DevOps-centric organizations should hold dearly. Automation increases an organization’s efficiency and reduces errors.

    You Have Frequent and Rapid Release Cycles

    As soon as software is ready to go, it should be deployed and delivered to the customers. We live in a world where competitors are building better software daily. It’s to an organization’s benefit that new features and bug fixes are continuously deployed to stay ahead of the competition. DevOps is all about continuous delivery, so having developers produce new code and quickly release it into production is one of DevOps’ core features.

    You’re Using The Right Tools and Platforms

    The tools you use to deploy features and fixes should make building applications easy and testing straightforward. You should generally be able to build, test and deploy to production with no hiccups along the way. If you’re spending a lot of time fixing any issues that occur, then you might not have the right tools or processes in place. Fortunately, today there are extensive continuous integration software options that can adapt to any current existing infrastructure. Although tools are an essential piece of DevOps, it’s important to remember that while tools make DevOps possible, it’s the practices that truly make the changes happen.

    You Have a Continuous Feedback Loop

    Nothing is perfect in its first iteration, and errors are bound to occur along the way. You must have a continuous feedback loop system to detect when and where things go wrong. Having software tools that monitor processes and quickly inform you when errors occur will make fixing issues that pop up easier and faster.

    If you have a system set up to identify issues, fix them and improve upon them, you’re doing DevOps correctly. If you run into problems and simply fix them without taking the time to analyze why they occurred or how to prevent them from appearing again, you might want to consider making some changes.

    Development and Operations Groups Work Together

    Communication between development, testing and IT operations groups is vital for DevOps success. Cooperation between these different groups ensures that the pipeline will run smoothly. Any problems that do occur will be faster to fix when developers and IT operations personnel can easily coordinate with one another.

    Goals and Objectives Are Clearly Established

    While communication between different teams is essential, having clearly defined goals and objectives that everyone is striving toward is just as important. Ultimately, all the teams have the same goal: making the organization more efficient; however, the individual practices they follow to make that happen might differ. Team members should be aware of precise requirements that need to be met for a goal to be reached, instead of making assumptions or leaving it to chance.

  • Best of 2021 – Hiring When You’re Not FAANG

    Best of 2021 – Hiring When You’re Not FAANG

    As we close out 2021, we at DevOps.com wanted to highlight the most popular articles of the year. Following is the twentieth in our series of the Best of 2021.

    Most of us are not FAANG, and we shouldn’t act like we are. Instead of using that, I’m going to use the term we used to use to refer to them: “Web Monsters”, which isn’t 100% accurate, but includes more than just the five FAANG members (Facebook, Amazon, Apple, Netflix and Google) for this article. At some point, we are going to need a new term. Web Monsters doesn’t clearly include Apple or Citi, but FAANG clearly excludes Citi and eBay. But this blog is about something else, so please accept my apologies for that lack of clarity. Assume that ‘Web Monster’ means “Big, popular high-tech company that pays well and looks good on a résumé.”

    There are several angles to “Not Web Monsters” that we can address, but I’m mostly going to focus on DevOps hiring, with a nod to irrelevant sales/marketing techniques. Hiring is an easy target because Web Monsters offer resume-building, pay and benefits that are super attractive in the industry, so they can (right now, at least) reject 99% of applicants and still have a good selection of candidates to choose from. Most organizations cannot compete in even one of those areas, though a few can when it comes to salary and/or benefits.

    We in IT have long had a bad habit of asking our applicants to be more qualified than the average employee. It’s pretty ridiculous at times; things like asking for a candidate to have more years of experience with a technology than the technology itself has actually existed. What happens when you ask for the impossible? Honest, qualified applicants self-select out, and only those who don’t realize the technology hasn’t been around that long and liars end up applying. That’s just bad hiring policy, even if you are a Web Monster. But at smaller organizations where each hire has a larger impact on the organization, it can be devastating.

    It seems worse in DevOps. In fact, all of the hiring items listed here seem worse in DevOps because DevOps is actually a nightmare mish-mash of skill sets. No two people define the job the same because “the job” covers everything from cradle to grave for an application. There is no part of app dev that “DevOps” couldn’t touch. That’s problem number one. Define the responsibilities of the role clearly, don’t ask for everything that it could be. As a rule, an Ansible specialist is not going to be a DevSecOps expert, and you shouldn’t expect them to be. I think, at a minimum, we need to split hiring along my traditional emphasis lines… DevOps and DevOps. If you are writing code and using Checkmarx while GitLab is building the results and you configured it to roll out to test … you’re not a DevOps engineer, even though there’s a bit of ops in there. Your strengths need to be in development and app configuration. Your job description needs to focus on that.

    Could there be roles that need it all? Yes, particularly in the SRE or chaos engineering space; I could see that. But they are certainly not entry-level roles, or even close. Knowledge in the breadth of technologies required to touch the entire DevOps toolchain is developed over years and with a healthy curiosity. And, of course, the person you are hiring should have a similar skill set to those doing the job today. Which is a good sanity check against the “Throw in the kitchen sink” syndrome.

    Next up is take-home tests and shared coding during interviews. We know that you have a vested interest in landing the best people, but if you are doing coding exercises with people who are primarily DevOps, you are wasting everyone’s time. Most DevOps jobs are done with PowerScript and/or bash with YAML files. Other languages used occasionally have no more place in a core job interview than other occasional bits. “It’s about how you think” is a common refrain. I would argue that an assignment that takes an applicant hours to complete (or in several cases I know of, days) is not an efficient way to evaluate the way a prospective employee thinks. And, in at least one case, I’m pretty certain the interview solved an actual problem that employees didn’t—or couldn’t—solve. Ask the candidate questions about how they’d solve problem X. Ask them convoluted questions. They don’t need to get to code level to show you how they think.

    And finally, my nod to bad marketing: Unless you actually are doing the volume of one of the Web Monsters, the fact that Facebook is a customer of vendor X is totally irrelevant. In fact, like the government, most vendors can mark one or more of the Web Monsters as their customers, just because Web Monsters are so big they try a lot of different tools across many different teams. So having them as a customer should be irrelevant, but if it isn’t considered irrelevant, should be expected.

    The team you have today is rocking it. Hire more people like them. You know what works and what you need. Get back to customizing hiring processes to deliver more of what you need and stop over-asking. There are more great people out there looking for roles; don’t scare them off by asking for irrelevant experience. Stick to the core. What are they going to do daily? Require that, then probe for other useful items at interview time. And get more good peeps in the seats!

  • Best of 2021 – Nine Pillars of DevOps Best Practices

    Best of 2021 – Nine Pillars of DevOps Best Practices

    As we close out 2021, we at DevOps.com wanted to highlight the most popular articles of the year. Following is the fourteenth in our series of the Best of 2021.

    In a prior blog I explained my 27-factor DevOps assessment model which includes nine pillars of DevOps and three dimensions of people, process and technology.

    The nine pillars represent categories of practices. In this blog, I include examples of practices for each of the nine pillars. In a future blog I will explain how to use these practices to conduct a gap assessment for your DevOps practices.

    Leadership Practices for DevOps

    • Leaders demonstrate a long-term vision for organizational direction and team direction.
    • Leaders intellectually stimulate the team by encouraging them to ask new questions and question basic assumptions about the work.
    • Leaders provide inspirational communication that inspires pride in being part of the team, says positive things about the team, inspires passion and motivation and encourages people to see that change brings opportunities.
    • Leaders demonstrate support by considering others’ personal feelings before acting, being thoughtful of others’ personal needs and caring about individuals’ interests.
    • Leaders promote personal recognition by commending teams for better-than-average work, acknowledging improvements in the quality of work and personally complimenting individuals’ outstanding work.

    Collaborative Culture Practices for DevOps

    • The culture encourages cross-functional collaboration and shared responsibilities and avoids silos between Dev, Ops and QA.
    • The culture encourages learning from failures and cooperation between departments.
    • Communication flows fluidly across the end-to-end cross-functional team using collaboration tools where appropriate (for example Slack, HipChat, Yammer).
    • The DevOps system is created by an expert team, and reviewed by a coalition of stakeholders including Dev, Ops and QA.
    • Changes to end-to-end DevOps workflows are led by an expert team, and reviewed by a coalition of stakeholders including Dev, Ops and QA.
    • DevOps system changes follow a phased process to ensure the changes do not disturb the current DevOps operation. Examples of implementation phases include: proof of concept (POC) phase in a test environment, limited production and deployment to all live environments.
    • Key performance indicators (KPIs) are set and monitored by the entire team to validate the performance of the end-to-end DevOps pipeline, always. KPIs include the time for a new change to be deployed, the frequency of deliveries and the number of times changes fail to pass the tests for any stage in the DevOps pipeline.

    Design-for-DevOps Practices for DevOps

    • Products are architected to support modular independent packaging, testing and releases. In other words, the product itself is partitioned into modules with minimal dependencies between modules. In this way, the modules can be built, tested and released without requiring the entire product to be built, tested and released all at once.
    • Applications are architected as modular, immutable microservices ready for deployment in cloud infrastructures in accordance with the tenets of 12-factor apps, rather than monolithic, mutable architectures.
    • Software source code changes are pre-checked with static analysis tools, prior to commit to the integration branch. Static analysis tools are used to ensure the modified source code does not introduce critical software faults such as memory leaks, uninitialized variables, and array-boundary problems.
    • Software code changes are pre-checked using peer code reviews prior to commit to the integration/trunk branch.
    • Software code changes are pre-checked with dynamic analysis tests prior to committing to the integration/trunk branch to ensure the software performance has not degraded.
    • Software changes are integrated in a private environment, together with the most recent integration branch version, and tested using functional testing prior to committing the software changes to the integration/trunk branch.
    • Software features are tagged with software switches (i.e., feature tags or toggles) during check-in to enable selective feature-level testing, promotion and reverts.
    • Automated test cases are checked in to the integration branch at the same time code changes are checked in, together with evidence that the tests passed in a pre-flight test environment.
    • Developers commit their code changes regularly, at least once per day.

    Continuous Integration Practices for DevOps

    • A software version management (SVM) system is used to manage all source code changes. (Git, Perforce, Mercurial, etc.)
    • A software version management (SVM) system is used to manage all versions of code images changes used by the build process. (Git, Perforce, Mercurial, etc.)
    • A software version management (SVM) system is used to manage all versions of tools and infrastructure configurations and tests that are used in the build process. (Git, Perforce, Mercurial, etc.)
    • All production software changes are maintained in a single trunk or integration branch of the code.
    • The software version(s) for supporting each customer release are maintained in a separate release branch to support software updated for each release.
    • Every software commit automatically triggers a build process for all components of the module that has code changed by the commit. The system is engineered such that resources are always sufficient to execute a build.
    • Once triggered, the software build process is fully automated and produces build artifacts, provided the build time checks are successful.
    • The automated build process checks include unit tests.
    • Resources for builds are available on-demand and never block a build.
    • CI builds are fast enough to complete incremental builds in less than an hour.
    • The build process and resources for builds scale up and down automatically according to the complexity of the change. If a full build is required, the CI system automatically scales horizontally to ensure the builds are completed as quickly as possible.

    Continuous Testing Practices for DevOps

    • Development changes are pre-flight tested in a clone of the production environment prior to being integrated to the trunk branch. (Note: “production environment” means “variations of customer configurations of a product.”)
    • New unit and functional regression tests that are necessary to test a software change are created together with the code and integrated into the trunk branch at the same time the code is. The new tests are then used to test the code after integration.
    • A test script standard is used to guide test script creation, to ensure the scripts are performing the intended test purpose and are maintainable.
    • Tests are selected automatically according to the specific software changes. CT is orchestrated dynamically, whereby the execution of portions of the CT test suites may be accelerated, or skipped entirely, depending on how complex or risky the software changes are.
    • Test resources are scaled automatically according to the resource requirements of specific tests selected and the available time for testing.
    • Release regression tests are automated. At least 85% of the tests are fully automated and the remaining are auto-assisted if portions must be performed manually.
    • Release performance tests are automated to verify that no unacceptable degradations are released.
    • Blue/green testing methods are used to verify deployments in a staging environment before activating the environment to live. A/B testing methods are used together with feature toggles to try different versions of code with customers in separate live environments. Canary testing methods are used to try new code versions on selected live environments.
    • The entire testing life cycle, which may include pre-flight, integration, regression, performance and release acceptance tests are automatically orchestrated across the DevOps pipeline. The test suites for each phase include a predefined set of tests that may be selected automatically according to predefined criteria.

    Elastic Infrastructure Practices for DevOps

    • The data and executable files needed for building and testing builds are automatically archived frequently and can be reinstated on demand. Archives include all release and integration repositories. If an older version of a build needs to be updated, then the environment for building and testing that version can be retrieved and reinstated on demand and can be accomplished in a short time (for example, minutes to hours.)
    • Build and test processes are flexible enough to automatically handle a wide variety of exceptions gracefully. If the build or test process for a component is unable to complete, then the process for that failed component is reported and automatically scheduled for analysis, but build and test processes for other components continue. The reasons for the component failure are automatically analyzed and rescheduled if the reason for the failure can be corrected by the system; if not, then it is reported and suspended.
    • System configuration management and system inventory is stored and maintained in a configuration management database (CMDB).
    • Infrastructure changes are managed and automated using configuration management tools that assure idempotency.
    • Automated tools are used to support immutable infrastructure deployments.
    • Equal performance for all. The user performance experience of the build and test processes by different teams are consistent for all users, independent of location or other factors. There are SLAs and monitoring tools that ensure the user performance experience is consistent for all users.
    • Fault recovery mechanisms are provided. Build and test system fault monitoring, fault detection, system and data monitoring and recovery mechanisms exist. They are automated and are consistently verified through simulated failure conditions.
    • Infrastructure failure modes are frequently tested.
    • Disaster recovery procedures are automated.

    Continuous Monitoring Practices for DevOps

    • Logging and proactive alert systems make it easy to detect and correct DevOps system failures. Logs and proactive system alerts are in place for most DevOps component failures, and are organized in a manner to quickly identify the highest-priority problems.
    • Snapshot and trend results of each metric from each DevOps pipeline stage (for example, builds, artifacts, tests) are automatically calculated in process and visible to everyone in the Dev, QA and Ops Teams.
    • Key performance indicators (KPIs) for the DevOps infrastructure components are automatically gathered, calculated and made visible to anyone on the team that subscribes to them. Example metrics are availability (uptime) of computing resources for CI, CT and CD processes, time to complete builds, time to complete tests, number of commits that fail and number of changes that need to be reverted due to serious failures.
    • Metrics and thresholds for DevOps infrastructure components are automatically gathered, calculated and made visible to anyone on the team that subscribes to them. Example metrics are availability (uptime) of computing resources for CI, CT and CD processes, time to complete builds, time to complete tests, number of commits that fail and number of changes that need to be reverted due to serious failures.
    • Process analytics are used to monitor and improve the integration, test and release process. Descriptive build and test analytics drive process improvements.
    • Predictive analytics are used to dynamically adjust DevOps pipeline configurations. For analysis of test results, data may indicate a need to concentrate more testing in areas that have a higher failure trend.

    Continuous Security Practices for DevOps

    • Developers are empowered and trained to take personal responsibility for security.
    • Security assurance automation and security monitoring practices are embraced by the organization.
    • All information security platforms that are in use expose full functionality via APIs for automation capability.
    • Proven version control practices and tools are used for all application software, scripts, templates and blueprints that are used in DevOps environments.
    • Immutable infrastructure mindsets are adopted to ensure production systems are locked down.
    • Security controls are automated so as not to impede DevOps agility.
    • Security tools are integrated into the CI/CD pipeline.
    • Source code for key intellectual property on build or test machines are only accessible by trusted users with verified credentials. Build and test scripts do not contain credentials for access to any system that has intellectual property. Intellectual Property is divided such that not all of it exists on the same archive and each archive has different credentials.

    Continuous Delivery Practices for DevOps

    • Delivery and deployment stages are separate. The delivery stage precedes the deployment pipeline.
    • All deliverables that pass the delivery metrics are packaged and prepared for deployment using containers.
    • Deliverable packages include sufficient configuration and test data to validate each deployment. Configuration management tools are used to manage configuration information.
    • Deliverables from the delivery pipeline are automatically pushed to the deployment pipeline, once acceptable delivery measures are achieved.
    • Deployment decisions are determined according to predetermined metrics. The entire deployment process may take hours, but usually less than a day.
    • Deployments to production environments are staged such that failed deployments can be detected early and impact to customers isolated quickly.
    • Deployments are arranged with automated recovery and self-healing capabilities in case a deployment fails.

    What This Means

    DevOps is a powerful tool that enables many benefits for organizations that use it. Achieving performance efficiently with DevOps depends on following best practices. By following the nine pillars of practices enumerated in this blog, organizations can achieve the performance potential that DevOps has to offer.

  • Best of 2021 – How to Combine DevOps and Agile

    Best of 2021 – How to Combine DevOps and Agile

    As we close out 2021, we at DevOps.com wanted to highlight the most popular articles of the year. Following is the thirteenth in our series of the Best of 2021.

    In recent years, application development and deployment have become an increasingly critical part of business operations. Because of this, various entities have sought to optimize their product development process.

    This has led to a rise in the popularity of DevOps, which is designed for that purpose. In simple words, the DevOps application during the software development process reduces the number of steps necessary to bring software to market. These faster releases and streamlined processes mean swift user feedback.

    DevOps implementation focuses also on the software scalability, how well it could be deployed and also its monitoring and maintenance after the future releases However, there is also a downside to traditional DevOps benefits. The system does not include the kind of continuous testing and improvement that Agile offers. 

    This has led to Agile practices being overwhelmingly focused on what you might define as the development aspects of software delivery. Though, there is less focus on operational aspects.

    Hence, both practices are quite critical to be implemented in the SDLC of any product. 

    Separating Agile and DevOps approaches to software development leads to the building of the product but its deployment, work automation, as well as infrastructure management fail as it’s “somebody else’s problem” when the Agile team is looking at it. Moreover, “operability” disappears into the background.

    The solution is combining Agile sprints with the integrated teamwork offered by DevOps. By doing so, you can optimize incrementally the development lifecycle and maintenance of your product both. It helps to correct an imbalance but has little influence over the practices that happen during the continuous development phase.

    What is Accomplished by DevOps and Agile Integration?

    The integration of DevOps and Agile helps in the following areas:

    • Streamlines the release process and improves product offerings.
    • Allows for better collaboration.
    • More value and fewer risks in each release. 
    • Fewer bugs and faster fixes.
    • Increased visibility.
    • Higher user satisfaction rates as the products are more qualitative.

    What to Pay Attention to When Integrating DevOps and Agile

    To simplify the process of implementing DevOps into Agile product development we have gathered the most common pitfalls that might happen during this process and how to avoid them making your process working smoothly.

    Improve Teamwork Flow

    Using the DevOps framework and Agile approach together makes it crucial that team members have a broader understanding of all development aspects. You get DevOps business value and the practicality of Agile together. 

    Team members such as the Product Owner, Scrum Master and the PM, along with operations, infrastructure and sysadmin roles need to consider not only the software development process but also delivery and maintenance. Your teams should be equipped with the knowledge of release, service and change management, environment provisioning, automation and tools, and application deployment. Build-and-run product-centric teams solve the problem of the Agile development-to-operations hand-off.

    Defining the Lifecycle

    Successfully attempting a DevOps implementation plan with an Agile framework means defining the product lifecycle. This increases consistency, reduces cost by minimizing waste and speeds up a time to market. Teams now inherit more of the operational concern within the entire lifecycle. Therefore, it’s ideal if you start to implement some of the DevOps principles right at the beginning of your development process. 

    DevOps Adoption in Sprints

    Agile workflow assumes the software development process divided into sprints, therefore it’s strategic to integrate DevOps management while handling sprints.

    Follow these instructions as you start working the DevOps approach into your Sprints.

    • Invite ops/infrastructure/support personnel to planning sessions.
    • Discuss product functionality and operability features.
    • Include them in the upcoming sprint.
    • Involve the DevOps team in sprint backlog planning and daily meeting, as well as sprint review and scrum and plan alignment.

    The engagement and collaboration of your development team and operations team also keeps the ops team abreast of functionality release timelines. The ops team can then help the dev team in planning the release schedule with greater accuracy and can assist the dev team in shipping the product faster.

    Including QA in Each Phase

    Making the QA part of the entire development lifecycle is another essential factor when integrating DevOps and Agile. Testing assumes a vital position when combining the two. Besides functional testing applied in Agile, DevOps approach requires performance and load testing of the software. Hence, constant testing is equally as important as continuous development.

    Implement Service Backlog Under DevOps

    When implementing DevOps and Agile together, you need to rebuild your service backlogging process. As under DevOps framework, it should include: 

    • Scalability of the software.
    • Deployment capability.
    • Service monitoring.
    • Logging.
    • Setting alerting. 
    • Software testing. 
    • Security and compliance aspects.
    • Operational performance.

    Leveraging the Right Tools

    Using the right tools is the key to successful Agile and DevOps adoption into your software development. Keep in mind that applies to your software development process configuration management tools to create and replicate infrastructure using the Infrastructure as a Code (IaaC) concept needed for DevOps. This helps developers to deploy the application on different types of platforms without any reworking efforts.

    Automation

    Automation of workflow is another part of Agile combining with DevOps. Try to automate all the code scanning processes and avoid any potential vulnerabilities. Build artifacts in a repository, or automate the release out the door. Automate these elements entirely so your end-to-end goalposts are the one possible places where someone has to manually check for issues.

    Documentation 

    While in the Agile approach, teams don’t document their meeting minutes or other communications. Instead, they prefer lo-fi methods such as pen and paper. DevOps, on the other hand, requires entire design documentation and specs to understand a software release.

    Measurement and Analysis

    Well, after building DevOps into Agile project management to keep track of its progress, you need to care about establishing the metrics to measure its effectiveness. This allows for the successful enablement of multiple releases to production faster. As under the Scrum Alliance Organization recommendations, some of these could be:

    • Percentage of release date adherences.
    • Percentage increase in release numbers.
    • Time taken for release to production.
    • Defects attributable to platform/support requirements.
    • Percentage of NFRs met.

    Though you can define other metrics during DevOps implementation to measure that have higher business value.

    Why DevOps and Agile both matter should be obvious now. Though these practices serve to streamline and simplify the product creation and deployment processes, combining Agile and DevOps requires a shift in the organizations. But putting efforts in combining Agile and DevOps management in the right way, you will see how it can improve your development process and deliver results in reliable, scalable and maintainable applications.

    — Slava Vaniukov

  • Rails 7 FTW | WFH is SOP | 586M Passwords Dumped

    Rails 7 FTW | WFH is SOP | 586M Passwords Dumped

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Ruby on Rails 7.0 is go, working from home is still de rigueur, and HIBP gets far, far bigger.

    DHH is Excited

    First up this week: Ruby on Rails version 7.0 is now in production. It promises less bloat, better security and even more developer efficiency.

    Analysis: Last chance for greatness

    Talk to devs about Ruby on Rails and you’ll hear a wide range of opinion—all the way from “obsolete memory hog” to “productivity secret weapon.” But its important promise seems to be to encourage devs to use better design patterns.

    Paul Krill: Ruby on Rails 7 rejiggers JavaScript support

    Ruby on Rails 7, the latest version of the web application framework, is now available as a production release. … It gives developers a “no-Node” approach to front-end development while still providing access to NPM packages and modern JavaScript.
    …
    Most Rails applications will not require Node.js, given the new defaults. … Other improvements:
    …
    Encrypted attributes have been added to Active Record, enabling applications to offer at-work encryption in addition to traditional at-rest and in-transit. … With asynchronous query loading, two unrelated queries can be run concurrently. … The Spring application preloader is no longer on by default, as faster computers have made it unnecessary.


    Let’s hear from Rails creator David Heinemeier Hansson—AKA DHH:

    Seven is … the culmination of years of progress on five different fronts at once. … The part that really excites me … is how much closer it brings us to the ideal of The One Person Framework. A toolkit so powerful that it allows a single individual to create modern applications upon which [to] build a competitive business.
    …
    It’s been well over twenty years for me now, and a big release like this still makes me giddy. … What a glorious time to be working in web development.


    What a whatnow? Here’s bb_matt:

    I don’t pretend … it’s a “glorious time to be working in web development,” except for one important aspect: … Experienced programmers coming over from software development and starting to make amends [with] strong typing, robust CI/CD, a focus on TDD, a focus on domain modeling.
    …
    99% is still total **** though—but then again, so is most software.

    Home Working Ain’t Gone Away

    A month ago, my long-suffering editor permitted me to write, “WTH? We Wanna WFH.” And survey after survey says tech workers work better at home. After the last 21 months, there’s simply no denying it.

    Analysis: Genie out of bottle—no going back

    If you’re under the misapprehension that DevOps is somehow “special” and your workers will be glad to return—think again. Find a way to make it work, or work will stop—and your business will die.

    Steven J. Vaughan-Nichols: How to lure employees back to the office? You can’t. Not now. Not ever.

    Months have gone by, and the great resignation keeps rolling along. … People don’t want to catch COVID-19 [but] bosses still think they can force skilled workers to return to offices. … That’s not going to happen.
    …
    The problem? Many executives and owners haven’t gotten the clue yet. [But] any doubts you may have about people not doing a good job unless you’re looking over their shoulders should have vanished by now. … The monthly academic WFH research.com survey has found almost six out of 10 workers reported being more productive working from home. … In the Dice State of Remote Work report … 53% of technologists listed greater productivity as one of the main benefits.
    …
    It’s simple: It works better for them and for your company. … Anyone who’s been paying attention … must agree. You can either go along with the flow, or you can fight it and first lose your staffers and then your company


    So what do DevOps managers think? One of them is kozikow:

    100% remote is not great either. … When there was a long period of 100% remote during Covid, there were some issues. … We developed a policy that Mondays are required, Wednesday are encouraged, rest is work from home. … It was shared decision of the team.
    …
    I think that most people who are strongly against hybrid work just have to live in a miserable place like SF. … It’s unfortunate that so many software companies are in the Bay Area.


    But you couldn’t pay some people enough to go back—people such as this Anonymous Coward:

    No way in hell they’d ever be willing to pay me … to come back into an office. … No having to go out in ****ed up weather, no fighting traffic and … dangerous drivers, plus I get all that commuting time back.
    …
    And I don’t have to put up with Linda cooking fish in the microwave.

    Have I Been Pwned? There’s a 38% Higher Chance

    This week saw a huge bump in the size of Troy Hunt’s already-massive database of stolen credentials. In case you’ve not seen it, HaveIBeenPwned.com is an amazing free service that lets apps securely check a user’s password isn’t already used elsewhere on a compromised site.

    Analysis: Build the API into your auth flow ASAP

    The UK’s equivalent to the FBI has donated a huge trove of credentials—after deduping, it’s added more than 225 million new passwords to the database. Frankly, if you still allow your users to rely on any of the 847 million passwords in HIBP, it would be negligent.

    Simon Sharwood: National Crime Agency finds 225 million previously unexposed passwords

    Troy Hunt, of Have I Been Pwned (HIBP) fame, yesterday announced … the NCA shared 585,570,857 [creds] with HIBP, and Hunt said 225,665,425 … he hasn’t seen before in the 613 million credentials HIBP already stored. [HIBP] lets anyone conduct a secure search of stolen passwords to check if their credentials have been exposed.
    …
    The NCA’s statement to Hunt did not reveal the source of the password trove [but said they] “were able to identify a huge amount of … passwords in a compromised cloud storage facility. … It became clear that these credentials were an accumulation of breached datasets known and unknown.”
    …
    [The] release brings the total Pwned Passwords count to 847,223,402, a 38 percent increase.


    How’s this relevant to DevOps? Troy Hunt explains:

    In the last month, there were 1,260,000,000 occasions where a service somewhere checked a password against … HIBP’s Pwned Password API. … There are all sorts of amazing Pwned Passwords use cases out there. For example … checking their customers’ passwords on every registration, login or password change to see if it’s previously been breached.
    …
    The UK’s National Crime Agency has done some wonderful work over the years to combat cybercrime. … Today’s release is about turning on the firehose of new passwords and making them immediately available to everyone for free. Having this open to the community, owned by the community and supported by the FBI and NCA is an enormously pleasing result.


    Time for a metaphorical thought experiment. LenKagetsu obliges:

    You were told by a reliable source that an unknown person … posted a high-res image of … your house key … on the internet. Would you change the locks?

    The Moral of the Story: Never switch horses in midstream


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

    Image: David Emrich (via Unsplash)

  • AWS Outage Outrage | Rusty Linux | ARM Latest

    AWS Outage Outrage | Rusty Linux | ARM Latest

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Amazon Web Services falls on its face, Linux’s move to Rust takes the next step, and the FTC stabs another fatal wound in the horrible Arm/Nvidia deal.

    AWS us-east-1 Breaks the Internet

    First up this week: Amazon Web Services’ outage on Tuesday sent scads of DevOps teams scrambling. But what was the cause?

    Analysis: DNS DDoS?

    While we still don’t officially know, there is some chatter to speculate on. Could it be that AWS’s DNS infrastructure is vulnerable to DNS reflection attacks?

    Hello. It’s Dave Lee: From angry Adele fans to broken robot vacuums

    It started early on Tuesday morning. Robot vacuums ceased sucking, Wi-Fi cameras stopped watching and eager Tinder daters were left unable to “swipe right.”
    …
    Within Amazon itself … the unthinkable occurred: grounded delivery drivers were unable to load packages and deliver to customers’ doorsteps, just as the peak Christmas season begins to step up. … Drivers at multiple facilities across the country were sent home … some dreading whatever workload may await.
    …
    An outage at Amazon Web Services … had rippled through the online economy, crippling services used by millions of people. … An “impairment of several network devices” in … US-EAST-1 was the “root cause” of the disruption, Amazon said.


    I was wondering what DevOps’ers learned? Enter Max Rozen:

    AWS took approximately an hour to update their status page to indicate anything was happening. … While our infrastructure is hosted across 20 AWS regions, a single point of failure was hosted in us-east-1.
    …
    What we learned: … I learned the saying, “Friends don’t let friends deploy in us-east-1.” As a result, we’ll be moving some things over to us-east-2 to start with. Secondly, we’ll be adding redundancy and the ability to failover to another AWS region.


    Getting inside the business, here are Eugene Kim and Katherine Long, to go over everything:

    Even inside AWS … information on the outage remained sketchy [and] rumors spread among staff. One AWS employee speculated that the outage was caused by an “orchestrated DNS attack,” while another employee downplayed those concerns.
    …
    According to a screenshot of an internal AWS communique … ”firewalls are being overwhelmed.” … A separate internal note said … the network issue “specifically impacted” Amazon’s internal DNS servers.

    Linux Officially Gets Rust Code

    The Rust language has taken the next step toward being part of Linux kernel development. Some “drivers and other non-core kernel programs” written in Rust can now be PR’ed. But there’s still no plan to go further.

    Analysis: Still needs lubrication

    Rust is much better than C for writing memory-safe code, and coding provable implementations. However, bootstrapping the toolchain is still stupidly hard.

    Steven J. Vaughan-Nichols: Linux is getting more Rust in it by the day

    It wasn’t that long ago that the very idea that another language besides C would be used in the Linux kernel would have been laughed at. [But] this has been coming for several years now. … Why? Because it’s much safer than C, especially at handling memory errors.
    …
    The next “patch series [adds] support for Rust as a second language to the Linux kernel.” … While he’s encouraging a slow but steady approach to introducing Rust into Linux, [Linus Torvalds] has also said that using Rust interfaces for drivers and other non-core kernel programs makes sense. [But] no one’s going to rewrite the kernel’s 25 million lines of C in Rust.
    …
    The Rust code proposed for the kernel now relies on the stable Rust compiler rather than the beta compilers. … As Linux kernel and lead Rust on Linux, developer Miguel Ojeda, put it, “By upgrading the compiler, we have been able to take off the list a few unstable features we were using.”


    How is it “safer than C”? ssokolow shares this fascinating insight:

    Rust tries to encourage that through good API design which allows it to be proven at compile time that the invariants will be upheld, thus avoiding the need to choose between safety and performance. My favourite example … is “the typestate pattern” which lets you teach the Rust compiler to prove correct traversal of any state machine.
    …
    A more mundane example would be the iterator APIs: No need for bounds checking if the iterator looks up the length once at the beginning and the APIs prevent you from wandering out of each entry it hands you. … Rust programs tend to run faster than their C and C++ counterparts because the developers feel comfortable introducing and maintaining more aggressive optimizations.


    “And then a miracle occurs,” quips @dave_universetf:

    I’m intrigued [but] I can’t help but feel sorry for the people doing bootstrapping work. C is hard enough to bootstrap, but Rust is a whole new level of pain.
    …
    Rust is literally not source bootstrappable. [The compiler] that works right now is Rust 1.39. So then you have to compile 18 more Rust compilers to get up to the current version.

    Nvidia’s ARM Woes Continue

    As I’ve said before, this deal really shouldn’t go through. Ownership of the ARM ISA and the core designs would be a huge strategic stick to beat Nvidia’s competition with. ARM chips are an increasing fixture in the datacenter—especially those that value “performance per Watt.”

    Analysis: Stick a fork in it

    Nvidia’s attempt to buy Arm from SoftBank is suffering death by a thousand cuts. And the latest wound comes from the FTC’s Higonokami knife.

    Paul Thurrott: FTC Sues to Block Sale of Arm to Nvidia

    The U.S. Federal Trade Commission … sued to block Nvidia’s $40 billion acquisition of … Arm, citing competitive concerns. … Nvidia announced its intention to purchase Arm from Softbank for about $40 billion, over one year ago, in September 2020. But the deal fell under immediate criticism from the major industry players that rely on Arm’s … designs, and the UK government announced … it would “intervene” for national security reasons.
    …
    Now, it may not happen at all. “We will continue to work to demonstrate that this transaction will benefit the industry and promote competition,” Nvidia said.


    Explain like I’m 5? Here’s swillden:

    ARM has a non-trivial and increasing share in the data center. … ARM is much, much bigger than phones … Arm is an important competitor to Intel and AMD. … Arm licensing IP to Qualcom, nVidia, Mediatek, Unisoc, Samsung, Apple, Google, etc., has been great for many areas of technology.
    …
    Arm acting as a common … designer for a bunch of different chip vendors has been extremely good for … progress in CPU/SoC design. … It would be really unfortunate if a lot of the industry were to shift back to x86, eliminating all of the competition on hardware features that we get from the many ARM chipmakers.


    But what of SoftBank? Ian King runs the numbers:

    Wall Street analysts are all but calling the deal dead. … According to Citibank analysts, the chances of the Nvidia purchase closing are now just 5%.
    …
    The company faces the prospect of losing a jackpot of roughly $74 billion. … Other suitors are unlikely to match such an offer. … So SoftBank’s best alternative may well be an initial public offering for the Arm business—but that is likely to value the company at … $19.8 billion—only about 27% the value of the … Nvidia deal.

    The Moral of the Story: It’s no secret that the both of us are running out of time


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk. (It’s so typical of me to talk about myself—I’m sorry.)

    Image: Ləman Şirinzadə (via Unsplash)

  • Nvidia/ARM Wavering | Google Outage Outrage | Backblaze IPO on Fire

    Nvidia/ARM Wavering | Google Outage Outrage | Backblaze IPO on Fire

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    Nvidia Can’t be Allowed to Buy Arm

    First up this week: Nvidia’s faltering attempt to buy Arm: Regulators around the world are alarmed by the idea. And now the UK—the legal home of the company—has taken the next step, which could nix the deal.

    Analysis: RISCy home for ARM

    This matters because ownership of the ARM ISA and the core designs would be a huge strategic stick to beat Nvidia’s competition with. Arm doesn’t make chips—it licenses technology to Nvidia and many of its competitors—AMD, Apple, Broadcom, Samsung and Qualcomm, to name but five. So this deal really shouldn’t go through, regardless of Nvidia’s fluffy PR reassurances.

    ARM chips aren’t just found in 99% of smartphones, but are an increasing fixture in the datacenter—especially where “performance per Watt” is important. They’re even found in exascale iron, including the Riken Fugaku, the world’s fastest supercomputer.

    Helen Thomas: Nvidia faces tough Arm wrestle

    It is one thing for a regulator to give your deal a thorough going-over. It is another for it to imply publicly that your assurances [aren’t] credible. … Nvidia got both from the UK’s Competition and Markets Authority. … The CMA’s full phase 1 report doesn’t make for easy reading.
    …
    The transaction, which was announced in September 2020, is bogged down in regulatory reviews around the world: … Arm’s “neutral” business model has always made it a difficult acquisition. … Arm licenses its designs to other chipmakers, providing scope to restrict rivals’ access.
    …
    Arm was a political talking point in 2016, when its sale to SoftBank … was waved through with embarrassing ease. There is now a much more muscular political take on UK national security concerns with a particular focus on tech.


    But if it’s owned by a Japanese company, why is the UK getting involved? Glad you asked—Aighearach answers:

    What did they buy? Shares of a UK company. … Then it is a UK company! … Buying shares is buying ownership, but it doesn’t move the company to another jurisdiction.
    …
    This blathering about “sold to a Japanese firm” is getting old.


    Nvidia claims, “The transaction will … boost competition and innovation.” Falmari finds that hilarious:

    Did Nvidia’s spokesperson previously work as a stand-up comedian? Because as jokes go, “boost competition” is a whopper. [Do you think] that’s what a company like Nvidia wants—to boost the competition?

    Google Cloud Outage has Ripple Effect

    Google’s load balancers all went offline for two hours on Tuesday. Aside from the knock-on Google Cloud failures, DevOps tools such as BigCommerce, Datadog, Deno, Discord, Egnyte, Evernote, Linear, Netlify and Toggl were reported to be affected. But perhaps the worst impact was felt by the poor, unfortunate users of Pokémon GO.

    Analysis: Paralysis

    In theory, putting all your eggs in one basket is okay—so long as that basket can reliably stop your eggs smashing on the pavement. But this isn’t the first time we’ve seen Google Cloud issues such as this; I can’t imagine it’s the last.

    Steven J. Vaughan-Nichols: Google glitch triggers major internet outage

    The result? Many of us have seen 404 errors when trying to get to some of our favorite web pages. … There appears to be more major sites having problems than not.
    …
    The specific problem seems to be with the GCP load balancers. These, working with Google’s Cloud Delivery Network (CDN) [are] designed to stop website failures and Distributed Denial of Service (DDoS) attacks by putting your website behind a single anycast IP.


    Are you feeling a touch of déjà vu? jeffbee, too:

    This is a sort of repetitive outage for Google. They’ve wiped out the GSLB configs before.

    A year ago there was also that big outage where they blanked out the whole Gmail delivery config and started rejecting all mail. … Config safety is not their strong suit.


    The #3 IaaS provider won’t get to #2 if it keeps this up. Here’s u/totopapa:

    I’m trying to convince my company to switch from AWS to GCP but an outage like this one is definitely a showstopper.

    Backblaze IPO Burns Brightly

    Backblaze is now NAS:BLZE. In its first week, the stock jumped 60%.

    Backblaze is well known for its tiny-margin consumer backup product and its open-source “storage pod” hardware—but is less well known for its DevOps-focused “B2” cloud storage service.

    Analysis: Can it Avoid Death-by-Packrats?

    Backblaze is trying to grow this newer DevOps business, without losing its classic consumer chops. But many of its former competitors have essentially dumped their “unlimited” offerings—it’s the old tragedy-of-the-commons problem.

    Jordan Novet: Backblaze [goes] public on small valuation and minimal funding

    In an era of blockbuster market debuts preceded by astronomical private financing rounds, Backblaze … a cloud service for backing up data … is a throwback to a time when sub-billion-dollar companies routinely went public to raise capital and build their profile … to win the trust of customers.
    …
    Backblaze shares climbed 24% in their debut on Thursday and another 12% on Friday, lifting the price to $22.31 and giving the company a market cap of around $650 million. … By the standards of its peers, Backblaze is a micro-cap.
    …
    One of Backblaze’s competitors is Amazon, the world’s fourth most valuable U.S. company. … Backblaze currently stores two exabytes of data — that’s 2 million terabytes — and has a half-million paying customers.


    Donovan Jones: Backblaze Targets $100 Million IPO

    The firm provides cloud-based storage services to small businesses and individuals worldwide. BLZE has grown revenue and gross profit impressively through the 2020 pandemic period and the IPO appears fairly valued, so is worth a close look.
    …
    The firm’s financials show strong revenue and gross profit growth. … The firm’s dollar-based net revenue retention rate for 2020 was 114%—a solid result. … The market opportunity for providing cloud-based storage services to small businesses and individuals is large and expected to grow at a material rate. … As for valuation, compared to partial competitor Dropbox, management is seeking the same Enterprise Value/Revenue multiple. [However] the combined revenue growth rate and EBITDA percentage rate … was 3% as of June 30, 2021, so the firm needs improvement in this regard.


    But there’s a long way to go, opines John Kozubik (aka rsync):

    Of the ~500,000 customers, 70,000 are B2 customers and 430,000 are “backup” customers—which are flat-rate “unlimited” users. … Roughly 85% of the user-base are paying $7/mo for “unlimited backup.”
    …
    These flat-rate users are very fickle and price-sensitive. … Roughly 85% of the user-base are paying $7/mo for “unlimited backup.” … It’s a fools errand to pursue this market and I assume the pivot to B2 was to build something more sustainable with a much higher price and usage ceiling.

    The Moral of the Story: Success is a journey, not a destination


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

    Image: Kristin Wilson (via Unsplash)

  • But What About When You Fail?

    But What About When You Fail?

    One thing Agile and DevOps definitely brought IT was a more accepting view of the whole “mistakes happen” mantra. Crashing systems is no longer a guarantee of a free ticket out the door. Indeed, off the top of my head, I can think of at least two cases where a CIO themselves checked in sloppy code and it trashed the system—CIOs at orgs big enough that they probably should not have been coding in the first place.

    And that brings us to today’s blog. The standard response to “But what if there is a bad update?” is “We’ll just roll out a new version!” That works in some instances. In a lot of instances—most that involve GitOps as part of DevOps—it doesn’t. Implosions can be huge and take out chunks of infrastructure. So you need a better plan than just assuming you can fix it with a forward update. Do you have rollback capability? Are you making use of it, assuring all is set to roll back if needed? Testing to make sure rollback works with the other changes made across the system in this update?

    That is part of the problem. Fixing with a new update is the best option, simply because in the age of massively distributed, microservices-based solutions, rolling back comes with a ton of baggage—enough that it may not be viable for you. Okay. So you can’t quickly roll forward. You can’t quickly roll back. Quick! What do you do?!

    And I don’t have the answer to that question, because I’m not in your organization working on your systems. At some point, all IT is personal. And this is one of those points. You need to know what your best options are if an update spirals everything, and you need to have a plan to implement it. But what your best options are is not going to be the same as the next org. This is that point.

    One option is to keep the ability to build the entire system. That’s a large setup, but every minute systems are down is hurting the company. And that’s what you need to plan for. Think of it as disaster planning for DevOps. A man-made disaster that destroys systems and infrastructure.

    So, like any other disaster planning scenario, walk through the chain that makes the system work, identify weaknesses and list ways to address them. Then test those to make certain they do what you hope they will. Then, set up an automated system to keep this whole plan up-to-date.

    In short, we’re in a new automated landscape, and we need a new automated tool to pull our rears out of the fire when the inevitable happens. Be it from dev error or malicious attacker, it is a safe bet that, sooner or later, you will have a massive systems outage that your DevOps toolchain can’t adequately address. Know what you are going to do. Or at least have thought about it, so you’re not just starting to think it through while coworkers and customers can’t access systems.

    And keep rocking it. This is just another layer of protection for all the hard work you’re doing. Take the extra step. Like insurance, if you ever need it, you will absolutely be glad you did.

  • Zero-Trust Network Access Platforms Slash DevOps Bottlenecks

    Zero-Trust Network Access Platforms Slash DevOps Bottlenecks

    In a prior article, I indicated why a new remote zero-trust platform is needed to support DevOps teams.

    Traditional remote access configurations are not well-suited for DevOps given a shifting user population comprised of employees, contractors and other third parties often using their own devices; the large number of tools and systems to which access must be provided and the hybrid nature of the infrastructure itself (on-premises, hybrid and multi-cloud).

    DevOps Pain Points

    Three pain points are associated with traditional approaches when used with DevOps.

    1. DevOps user experience is poor in traditional remote access configurations. Each DevOps service must be accessed as separate secured sessions, which is confusing and tedious.
    2. System management requires IP whitelisting rules in VPN, static SSH keys in Bastion, firewall rules controlled by different teams and application-specific authentication. The coordination and execution of so many touchpoints for common actions—including the onboarding of new team members, changing roles or adding a new service—invites mistakes that cause productivity issues and introduces unforeseen security holes.
    3. Security is complex and error-prone. DevOps users must be granted restricted access to the individual services they need to be productive. Administering restricted access for individuals and groups is time-consuming. Security policies typically require monitoring detailed audit logs of user access.

    How a Zero-Trust Remote Access Platform Works with DevOps

    In my whitepaper, Zero-Trust-Remote Access-For-DevOps, I explain how a zero-trust remote access platform can resolve the requirements for improved DevOps user experience, consistent and manageable administration and simplified, less error-prone security.

    Improved DevOps User Experience

    By replacing the need for the user to tediously set up separate sessions for each DevOps service they need to use, a client-side DevOps service catalog enables all DevOps services to be accessed with a single click. This service catalog approach is more convenient for DevOps users and provides a form of DevOps governance. It does this by encouraging DevOps teams to select choices that are available in the catalog instead of creating new choices that result in expensive DevOps tool sprawl.

    Simplified DevOps Management

    Management of remote DevOps users is simplified by consolidating access for all DevOps users and services under a single unified service manager and connector. The need to manage each user and service with different teams, each with their own processes, is unified and streamlined when the management requirements for DevOps users and services are consolidated under a common zero-trust remote access platform.

    In addition, this approach makes it much easier to implement more adaptive policies across the breadth of users and services.

    With a zero-trust remote access platform, onboarding new DevOps users and groups is accomplished within a single session—not separate sessions for each onboarding requirement.

    Simplified and Less Error-Prone Security

    Consolidating access to DevOps services under a common zero-trust remote access platform with unified services and services connectors enables simplified and less error-prone security controls, improved security monitoring and policy management. Audit logs of access are reported consistently for all services. Monitoring and changing access rules can be supported for individuals or groups when adjusting a policy. Trust-based access controls can be implemented consistently. Granular policies ensure access is predicated on appropriate device trust score thresholds being met. Rather than a single check prior to granting access, these production services policies continuously assess the security of the users and devices accessing the services.

    Transforming to a Zero-Trust Remote Access platform for DevOps

    Transformations of all types, and especially those for DevOps, are made more certain by following a
    seven-step transformation process, described in my book Engineering DevOps. This process serves as a framework for implementing the zero-trust remote access platform.

    The first two steps, vision and team alignment, involve getting the leaders, key influencers and implementers of an organization to recognize and establish actionable goals for the implementation of the zero-trust remote access platform.

    The third step, assessment, requires taking inventory of the current state of DevOps users, DevOps services and policies that need to be supported by the new platform.

    The fourth step, road map, requires determining the plan for implementation. Typically, it makes sense to start with one application. Start with a set of DevOps users and DevOps services needed for a single application or pipeline which can serve as a model to drive adoption to other applications as the solution is proved and accepted.

    The fifth step, realize, is the step in which the zero-trust remote access platform is installed and set up for the initial model application or pipeline. Banyan Security explains how an example tool vendor helps to simplify this step.

    Capabilities to look in for in zero-trust remote access tools include:

    • Deploys in minutes, with no integration requirements on any existing DevOps services or infrastructure.
    • No need to set up VPNs, open inbound firewall ports or manage DNS.
    • Use existing DevOps user devices without diminishing security.
    • DevOps service catalog provides users visibility and one-click access to DevOps services.
    • Support for bundled services, favorites, and autorun capabilities.

    In step six, operationalize, the procedures for commissioning and using the zero-trust remote access platform are proven, benefits are measured and playbooks are hardened for the organization.

    In step seven, expansion, the zero-trust remote access platform deployment is expanded to other DevOps applications pipelines, DevOps users and DevOps services across the enterprise.

    What This Means

    Traditional remote access configurations are not well suited for DevOps administrators or users given the dynamic nature of services, team members, and environments. A zero-trust remote access platform addresses the need for an improved DevOps admin and user experience, simplified access management to DevOps services and simpler, less error-prone security practices and monitoring.

    I want to express my appreciation for the time spent with the Banyan Security team, who has pioneered many of the concepts outlined in this blog and contributed valuable insight.

  • Meet the New(ish) Pipeline

    Meet the New(ish) Pipeline

    My company keeps a sample project running pretty much all of the time. Our current project (multi-target web/Android app using Java and Spring built with Gradle and Ant that we’ll call “Finder” here) is getting a bit long in the tooth, so we have been casting about for a new one over the last several months. From non-work sources, we came upon the idea of multi-targeted Godot with C# as a viable side project. Moving to a completely different set of tools is kind of the point; it makes us stretch, and that is the idea of our ongoing projects. The stated goals for these projects are:

    1. Stretch knowledge about different languages, DevOps tools and platforms.
    2. Keep doing what you all are doing. Too many pundits have “grown out of” hands-on work, and it shows.
    3. Try out new ideas/architectures.
    4. Avoid “kitchen sink.” We can’t have a current test app that includes all available technologies, so keep it streamlined. Approach this as a production app that we use for testing, not as a test app.

    That’s why we’re looking to replace our Java/Spring project. We’ve got that pipeline down, we’ve tweaked the app until we are happy with the core and it hasn’t changed in a good while. But change is a part of DevOps, so we’re off.

    The Obvious

    Let’s take care of the obvious elements in our toolchain right away:

    1. Git. We’ll use GitHub, but it was a given that some form of Git would be holding our code.
    2. Visual Studio Code. We’re doing C#, so this is going to be the most useful option for code editing. The built-in Godot editor is good and will be used, but VS-Code offers C# functionality add-ons.
    3. Antsle is the deployment tool of choice, though this has changed over the years. We’ll use it to set up install sets for clients.
    4. Jenkins or Hudson. We’re going to use this only for test management.

    The Less Obvious

    There are several constraints we want to (or have to) meet. First, we want to avoid using Jenkins or Ant to control the entire build process, mostly because we’ve done both in previous projects, and most DevOps teams are no longer using these methods for new projects. Second, this is a sample we keep, not an investment, so we want to use open source technology if at all possible, to keep expenses down. Third, we’re using a game engine to do app development. This has some impacts–like most game engines, it is in control at runtime and we need to bundle the engine in the final application.

    1. The Godot Engine. Godot is written as a game platform, but its graphics/cross-platform abilities make it useful for graphics-heavy apps like ours. But we have to include the engine and its config files in the pipeline.
    2. Code Scanning. Static analysis was a late addition to Finder, and it’s in Java. This project is in C# and Godot Script. We have not yet found a tool that supports Godot Script, but C# we can get; it just adds restrictions. So we will use a static analysis tool, but we’re still deciding which.
    3. Build Automation. We’ve used Jenkins, Gradle, Maven and Ant to handle this step at different times in the past, but something newer would be a good idea. We are still evaluating this step. We will likely end up back on one of those tools because they are still used today. Godot is a new element to us, so we are also evaluating what is required to hook into it and make functioning apps in a repeatable process.

    Fueled Innovation

    We’re having fun—first couple sprints have us running a working prototype, so we’re happy with that. There’s still a lot to implement in the app and in the pipeline, so we’ll no doubt have more to say here soon. We’re talking about putting a server component in—not because the original concept needs it, but because our outgoing app—Finder—had server and client sides, and our need for things like Postman go away without this architecture. And yet, API security, testing, etc. is an important part of the DevOps architecture. As you can see, we’re still iterating.

    As to Finder, we normally take down the pipeline for a previous playground project when the new one goes up, but Finder costs us little. There are five containers (three permanent for the app hosting; two for build/test) and some processing power, so we’ll probably keep it since it is using a different set of pipeline tools (and languages—it is mostly Java/JavaScript).

    Meanwhile, you all keep rocking it. We’re building an(other) app that no one but us will ever see—but you are keeping the enterprise humming. Kudos to you!