devopsy.kinsta.cloud

Category: Features

  • Hardware in the Age of Automation

    Hardware in the Age of Automation

    There’s a joke making its way around the internet that goes like this:

    Q: How do DevOps engineers change a lightbulb?
    A: They don’t. It’s a hardware problem.

    The trend to abstract hardware away from the day-to-day work of most IT personnel continues. We in DevOps have given it a name: infrastructure as code. Hardware has been pushed so far back into the recesses of IT that I’ll wager good money to say that today most folks working in information technology have never seen the inside of a data center, and fewer have done something as labor-intensive as run cable wire over a rack of servers. For many, hardware is but a concept that’s made real every month by a bill from Amazon, Azure or Google Cloud.

    But hardware is real and it does matter, a lot. If you don’t think so, look at your cellphone. That little piece of hardware had changed just about everything about how the world works today. The cellphone has made possible not only Tinder, Twitter, Instagram, Uber, Lyft and a multitude of startups that will never see an IPO or private equity buyout, but it also has brought earth-shaking events such as the Arab Spring, Brexit and SpaceX landings into the palms of viewers worldwide.

    Hardware counts and it’s going to count more in the coming years. Those working on the forefront of innovation understand that special hardware is, and will continue to be, a critical factor in the development of increasingly complex software. We’re already seeing the ability to choose specific hardware configurations available on cloud providers such as AWS. For example, the g3.16xlarge EC2 instance type is backed by both CPU and GPU chips. This is only the tip of the iceberg.

    According to Jacob Smith, senior vice president at Packet:

    “People ask what’s next. Well, this stuff [GPUs] is an example of what’s next. It’s not just really expensive stuff, but really specialized stuff. Whenever you start to do something that’s really big and really important, which are a lot of the things we’re doing on the internet and in the world, whether it’s bioinformatics or cars driving themselves, those require really big workloads. There’s a lot of data, a lot of processing and you just can’t do it with generic hardware. The reason that hardware is the next innovation layer is because of all the software you guys have been writing to make software portable and deployable around the world in minutes, now the software wants to touch on the new kinds of hardware. And I think that embracing this is the next wave.”

    Packet, and companies like it, are looking forward to a time when computing platforms will move beyond the generic to the specialized. We’re already seeing the trend play out: The value of the stock of Nvidia, a major GPU manufacturer, has gone up 10x in the last three years. (See Figure 1.)

    Nvidia stock has grown 10x in value in over the last three years. (The Wall Street Journal)

    Also, the projected demand for specialized chips is attracting a lot of venture capital (VC) money that’s funding startups such as Cerebras, Wave Computing and Graphcore. Even Google is getting into the specialized hardware business with Tensor Processing Units (TPU). Hardware is indeed becoming very cool again.

    So, what does all of this have to do with automation?

    Automation is a processor-intensive undertaking. And, as AI weaves itself more into the fabric of day-to-day data processing and machine learning, we’re going to see more demand for specialized hardware to handle the activity. In a way, that’s not new; we’ve had specialized hardware for a while now. TV remote controls have been around well before Bluetooth came on the scene. What is new is the degree of intelligence at play in these new devices. All my old-style TV remote control had to do was change the channel when I clicked the button. Today, my remote control recognizes my voice and helps me find a show of interest. It’s a big leap from an operation that responds to a button-click to one that uses voice recognition and inferential lookups.

    As Packet’s Smith pointed out above, as the complexity of software grows, so, too, will the need for special hardware to drive it. The new trend will be to wrap hardware around software. Thus, I have no trouble imagining a new type of hardware that is designed to handle a very particular process—gene editing, for example. Imagine ingesting a pill that is really a nanodevice that contains the intelligence to travel your body and physically alter your genetic makeup. It’s not that far-fetched an idea. Making the idea a reality isn’t so much about the software, it’s the hardware. We don’t have the nanotechnology—yet. Hence, the opportunity.

    Right now, most automation activity is run on generic chips housed in remote data centers around the world. A lot of the work we’re doing is still commodity: Spinning up Kubernetes clusters, facial recognition under Tensorflow, discovering security vulnerabilities using Macie, for example. Yet, as artificial intelligence (AI) becomes more commonplace, the use cases for which AI is appropriate will grow. This growth will require new types of specialized hardware. Engineers will imagine the software and then design the hardware to make it run. We’ll come to a point, which is not that far away, where innovation will be as Smith describes. It will indeed be about wrapping hardware around software. As a result, we’ll go from the “Internet of Things” to the “Internet of Really, Really Smart Things, from the Microscopic to the Gargantuan.” The implications will be profound.

    — Bob Reselman

  • The Software Testing Metrics That Make Or Break DevOps Success

    The Software Testing Metrics That Make Or Break DevOps Success

    Organizations today are racing to develop software at a breakneck pace through efficient DevOps processes. But it’s important for them to avoid losing sight of the forest for the trees. If too much of that speed comes with quality sacrifices that degrade the user experience, the rapid software iterations hurt the business more than they help it.

    “Organizations adopt DevOps to accelerate the delivery of innovative software that creates positive user experiences,” said Clark Golestani, former CIO at pharmaceuticals giant Merck and current managing director of C Sensei Group. “If you start accelerating delivery without also rethinking your approach to quality, you will end up rapidly delivering updates that drive your customers to competitors.”

    This is why continuous and automated testing is so essential to DevOps success. But instituting effective and streamlined testing is easier said than done. Organizations must continually tweak what, how and when they test their software to achieve the optimal test coverage while introducing as little friction as possible into the software delivery pipeline. It’s a delicate balance, and the only way to achieve it is to keep meticulous track of metrics that evaluate how well the testing process is improving quality throughout the development life cycle.

    “As the saying goes, ‘You can’t manage what you don’t measure,’” said Andrew Magarie, senior market impact consultant at Forrester Consulting. “So firms need to be able to define and track the right metrics that can help enable quality at speed while minimizing risk.”

    Magarie and his team at Forrester recently conducted research into 94 different quality metrics frequently measured by DevOps organizations across four development stages: build, functional validation, integration and end-to-end regression testing. The study showed that the most effective test metrics that bubbled up across all of stages tended to clump into at least one of three categories.

    Risk-Focused Metrics

    According to Magarie’s report, the most mature and effective organizations fundamentally chose testing metrics that best helped them understand business risk posed by the software throughout the development process.

    “For this study, we use the following definition of business risk: Any application shortcoming that impairs the end user’s (or customer’s) expected experience and ultimately erodes confidence in the business,” he explained.

    For example, during the build stage mature organizations consider measuring the number of automated tests prioritized by risk as one of the most important assessments.

    “Tracking ‘unit’ tests prioritized by risk is the key. As the code base evolves, developers and testers need immediate feedback about change impact. This feedback is significantly more actionable if prioritized by level of risk,” wrote Magarie.

    As he explained, business risk will be different for every application, and the complexity of an organization’s architecture and transaction dependencies adds more variables to the equation. However, organizations need to seek ways to measure ‘relative risk’ across the software development life cycle (SDLC) if they truly want to take a disciplined approach to incrementally improving quality in their software.

    Contextual Metrics

    Forrester found that simple “counting” metrics don’t offer QA nearly as much value as contextual metrics. So simply tracking the number of tests will not provide teams with as much actionable data as the measurement of the ratio of requirements covered by tests. Approximately 69 percent of mature organizations say running functional tests and checking against covered requirements is a top measurement and 75 percent say the same about API tests run against functional requirements.

    Holistic UX Metrics

    The most mature organizations recognize that while application-specific or team-specific QA metrics are important, the ultimate goal is ensure the quality of the user experience (UX) across their entire interaction with an application. This means measuring the thoroughness of end-to-end regression testing is crucial. Approximately 70 percent of leading DevOps organizations say they believe the percent of automated end-to-end test cases as a top metric.

    — Ericka Chickowski

  • DevOps Experience: Learn a Lot without Leaving Your Spot

    DevOps Experience: Learn a Lot without Leaving Your Spot

    The great thing about conferences is they can be hotbeds of learning: keynote presentations, seminars, one-on-one sessions, networking … the list goes on. The drawback, however, is the travel: delayed flights, the dreaded middle seat on the airplane, lost luggage, jet lag … the list goes on. That’s why virtual conferences are now my first choice: All the of the learning, none of the hassle.

    MediaOps—the company behind devopsy.kinsta.cloud, Container Journal, Security Boulevard, DevOpsTV and more—and CA Technologies have teamed to present a free virtual summit chock full of information and education for the DevOps professional presented in a virtual reality environment. DevOps Experience is a true virtual conference, replete with theaters to view the keynotes and presentations, a conference lobby to chat and socialize and a virtual trade show floor with virtual exhibitor stands equipped with video, downloads and even schwag for attendees. And did I mention it’s free?

    DevOps Experience will feature keynotes from a lineup of DevOps luminaries including:

    • Gene Kim, CTO, researcher and co-author of “The Phoenix Project” and “The DevOps Handbook,” and founder of IT Revolution
    • Nicole Forsgren, founder and CEO at DevOps Research and Assessment (DORA)
    • Alan Shimel, editor in chief and founder of devopsy.kinsta.cloud and Security Boulevard, and
    • Aruna Ravichandran, vice president of Global Marketing DevOps at CA Technologies and author of “DevOps for Digital Leaders” and “The Kitty Hawk Venture”

    In addition, DevOps Experience will include a number of Spotlight Sessions with topics designed to provide even greater value to attendees. Sessions include:

    • The Accelerate State of DevOps Report
    • Delivering DevOps Practices for Mainframe
    • Women in DevOps Panel
    • DevSecOps, Why the Time is Now

    DevOps Experience is scheduled to be held Sept. 20 at a computer or mobile device near you. Visit the website for more information or to register for this daylong event. Registration—which is free, by the way—also includes access to the event on-demand.

    — Charlene O’Hanlon

  • Why Database Development is in a League of Its Own

    Why Database Development is in a League of Its Own

    Despite how the database is the lifeblood of many organizations, its management remains siloed and is more often than not left to a separate team of resident experts outside of the sphere of DevOps. Database developers also may work separately from the database administrators, thus further slowing down deployments.

    The end result, of course, is that database management generally fails to benefit from the same level of agility DevOps offers application development. Bottlenecks thus occur when slower and less-agile database development holds up DevOps-managed application releases and deployments.

    Determining the role DevOps needs to play for successful database development hinges on a number of things, including making sure everyone is on the same page when it comes to database changes.

    Database changes can mean different things to different people. For example, a database change can mean applying a patch or fix, or it can mean deploying database changes. It’s thus important to add context to avoid confusion. “When you talk about deploying database changes, it involves changing the structure and the logic of a database to keep it in sync with the applications it supports,” said Ben Geller, vice president of marketing for Datical.

    “Database release automation” also calls attention to how DevOps needs to address both the application-code pipeline and the database-code pipeline, Geller said. “When you’re doing application release automation, you’re automating the build, deployment and release of application code. When you’re doing database release automation, you’re automating the build, deploy and release of database code.” Geller said. “You need both of those things to happen in sync to in order to ship software.”

    An example of a database change that can be processed with Datical’s rule engine. Source: Datical

    The ability to rely on a DevOps teams to successfully deploy and manage database schema and logic changes is also contingent on adopting the right tools.

    This is not to say that making the shift to using DevOps to give database development the attention it requires is easy. Getting there often can require major changes to an organization’s DevOps’ processes, including making cultural and job-definition adjustments. Realizing what exactly is involved conceptually as well as adopting the necessary tools, for example, represent the starting points.

    Meanwhile, the resulting advantages and improvements to an organization’s ability to deploy database schema and logic changes in ways described above should pay off in both the near and long term. This likely will lead to improvements in agility measured by deployment speeds and other key performance metrics.

    — B. Cameron Gain

  • From Thought to Leadership

    From Thought to Leadership

    Thought leadership is a term that those of us in the industry hear a lot. A thought leader is a person who has ideas/thoughts that push the edge and brings great ideas to the rest of us. Most vendors want to be seen as thought leaders—and some do, through the agency of rocking individual employees.

    We all benefit from true thought leaders. They dream up new ways to do things (such as DevOps and agile), or find new uses for things (such as adapting agile practices to operations) that, in the long run, we can all benefit from. Good thought leaders (in any space) either look outward and see how what we’re doing today can be used to greater benefit tomorrow, or look inward and make us think about ideas and where we are in accepting them.

    This Isn’t About Talking Heads and Vendors

    The thing is, if you are establishing or attempting to expand DevOps operations in your organization, you are a thought leader. And that’s what I’d like to discuss.

    Those of you who say, “No, not a thought leader; I’m showing how long-established DevOps practices can improve the organization,” are wrong. The key to thought leadership is that you are there, and your audience isn’t. If those long-established DevOps practices were acknowledged in the organization then there wouldn’t be a need for DevOps leaders to establish or expand. Everyone would agree and it would just happen. After sufficient meetings, of course. (The definition of “sufficient” varies by organization and falls somewhere between zero and infinity.)

    In short, a thought leader must simply see what others do not (yet) see. If the audience is internal and resistant, then a DevOps leader is indeed a thought leader for that audience.

    So, What is Required?

    It is true that a big chunk of DevOps is communications. I’m an “automate first, streamline communications after” kind of DevOps person, but acknowledge that the opposite route is also valid. But increased communications are going to be required for long-term DevOps success. Part of that communications is with other managers both in and outside of IT. There is no better time to start than the present, by acting like a thought leader.

    Target the next application/portfolio you think would greatly benefit from DevOps. Sit and refine where you think improvements and/or savings would occur, get the buy-in of line-of-business leaders responsible for the application or portfolio, and go on a PR blitz.

    Thoughts are Great …

    But leadership is better. Be the leader, advocate for what you believe to be right. There are those who will tell you DevOps is inevitable, and it’s not a big deal once there is at least one project in the building doing DevOps. I could not disagree more. Change—any change—doesn’t happen without a champion. If you are in charge of DevOps, yes, take the DevOps Institute Leadership Certification (Disclosure: I have written paid promotional material for this certification. But it’s good material, so I’m mentioning it here anyway), but also look outward. Bring the organization to the table; be visible; be driving.

    But don’t be annoying. Know when to back off, or maybe target a different application first. And know how the existing system reporting and deploying will be mirrored in a full-on DevOps environment, so you can answer important questions such as, “How will we get through test and deploy if we re-target the application?”

    Do I think DevOps is inevitable? No. But I do think it will have a lasting impact on the way we develop and deliver software, even if a particular shop doesn’t claim to be a DevOps shop, deployment automation, test automation, enhanced defect and runtime monitoring, increased communications. These are coming because they’re good ideas. It’s best to have a plan, be a thought leader and get it rolled out so the organization can reap the benefits.

    — Don Macvittie

  • DevOps Security: 3 Privileged Access Management Best Practices

    DevOps Security: 3 Privileged Access Management Best Practices

    The tremendous upside of DevOps practices and tools are enough to keep organizations pressing forward at all costs. But when sloppy use of DevOps toolchains cause breaches, more than half the time it comes down to poor protection of privileged accounts. According to a recent study by security vendor Beyond Trust, 52 percent of IT practitioners say that overprivileged users are at the root of DevOps and other next-generation technology-caused breaches.

    If organizations are going to reap the biggest benefits from DevOps without putting their IT infrastructure and data at risk, they’re going to need to think more strategically about how they handle privileged access management (PAM), said Morey Haber, CTO at Beyond Trust. devopsy.kinsta.cloud recently caught up with Haber to break down three of the most important PAM best practices for DevOps teams to build into the continuous delivery pipeline.

    Discover and Inventory All Privileged Accounts and Assets

    You can’t manage accounts and assets you don’t know about, so this step is foundational to the whole process of PAM. Unfortunately, with so many scripts and so much automation layered all over the DevOps toolchain, it can be tremendously difficult.

    “If it’s embedded in other runtimes or it’s hard-coded into compiled executables, that discovery is the hardest,” Haber said. But it must be done. Organizations need to get clear visibility into exactly what tools are executing the automation and what the privileges are assigned to them. “Is it a power-shelf script? Is it embedded in Jenkins or somewhere else? What is actually running? Who’s running it and when?”

    Additionally, organizations need to understand where the automation is stored and, consequently where that embedded credential information is being stored, so it can be examined for tamperability and safety of the credentials.

    “Now, this isn’t something that’s easy to discover in any of the three contexts, but it has to be,” Haber said. “So, one you have to understand the entire process and where you’re actually embedding privileged credentials for the edit, storage, operation and scripting itself.”

    Manage Shared Secrets and Hard-Coded Passwords

    Hard-coded passwords are one of the biggest no-nos in credential management. Unfortunately, even when application security teams are meticulous about rooting hard-coded passwords out of their finished applications, they often leave them within the IT infrastructure that helps support the development of that software for the sake of expedience. Same goes for account sharing, which is a frequent mistake organizations make to just get the automation working and keep it working with stability.

    The problem is, this trashes any semblance of traceability or auditability of activity within the affected environment.

    “Whether that is using some type of proxy technology, a VDI environment—anything, you don’t want people logging in directly with shared accounts because you can’t trace that,” Haber said, noting there needs to be a control plane in the mix that can keep track of both individual users and script or automation accounts that are impacting the environments to track each process or activity against unique credentials. This is essential not just for compliance, but for forensics and the overall integrity of a DevOps team’s software “factory.”

    Enforce the Rule of Least Privilege

    Ultimately, strong PAM depends upon only giving individual users or specific automation accounts the exact amount of privileges they need to get their job done.

    “We see credentials being given to DevOps teams all the time that are way, way overprivileged,” Haber explains “It should come down to the least privileged model—that you’re giving enough privileges for the DevOps process to work, but not overprovisioning so that if any process or account is compromised, it doesn’t jeopardize the rest of the environment.”

    — Ericka Chickowski

  • DevOps: Build a Continuous Improvement Culture

    DevOps: Build a Continuous Improvement Culture

    DevOps was built around the concept of continuous everything. We even use it in many of the names—continuous delivery (CD), continuous testing, continuous integration (CI)—and it is a worthy goal to want to improve the places that form bottlenecks in your software delivery systems and processes.

    But what about the bottlenecks in the DevOps systems and processes?

    Like any other system, DevOps involves tools. We choose those tools, invest in learning those tools and use them to do great things. But then we have investment in the tool. If our knowledge and systems for deployment are all based around CD tool X or ARA tool Y, we are just as unlikely to want to move from it as we were unlikely to want to move away from the previous toolset.

    Even if you discount the number of shops that have done pilot or small-team DevOps but kept the core of their business waterfalling, there are still a large number of organizations that stepped into DevOps and stopped improving. Granted, it is a huge job to move a significantly sized application portfolio into the DevOps mold and toolchain, but it is contrary to the process/goals to stop just because you did.

    The market is nowhere near finished evolving: New tools are coming available and massive improvements to existing tools are being released. Keep taking advantage of those facts.

    The more projects/portfolios you have moved into DevOps, the more weight there is to resist change. That is the simple reality, just as the larger a rock, the harder it is to move (assuming the same type of rock, of course). The answer: Watch for those improvements that will make moving the rock worth the force required to do so.

    Working containers into the overall DevOps process, for example, can benefit every step in the toolchain. But if they’re not already in the process, it is a lot of work to get them folded in. I chose this example because for most shops, a little research will show it is well worth the effort. From preconfigured Dev/Test/Prod servers to run-and-done compile instances, containers can make the process more manageable and more agile. The benefits outweigh the costs for most shops.

    So keep an eye out. Don’t implement and spend all of your time on the tools you’ve chosen. Stay aware of what is out there, and what is coming. Understand how it would impact the DevOps process and try to pull in those items that fit the environment.

    And, as always, keep rocking it. The end goal is to keep the systems up, running and usable. And most of you are doing a great job.

    (This, and all blogs, brought to you as much by the geeks behind the scenes as the author and editor.)

    — Don Macvittie

  • Thinking About the Future in the Age of Automation

    Thinking About the Future in the Age of Automation

    Let’s do a thought experiment.

    Let’s imagine a world in which almost all the work required to sustain human life on the planet is performed by intelligent machines. Think about it: All the crops get planted, raised and harvested by robots. Same for livestock. Machines make all the clothing and build all the houses. People and things get transported by driverless vehicles on the ground, over the oceans and in the sky.

    Food gets delivered by an AI-controlled fulfillment service that brings groceries right to the front door. Refrigerators and storage cabinets are smart enough to request replenishment when supplies run low. Packaging and containers are IoT devices that integrate easily with refrigerators and storage cabinets. Everything is communicating everywhere all the time.

    Power sources are installed and maintained by robotic intelligence, some of which are supervised by a small cadre of humans. Power is delivered by AI, too. Most power is supplied by solar and nuclear energy. Given the high energy yield-to-weight ratio that fossil fuel and biofuel provide, internal combustion engines still power most aircraft.

    Humans login to online schools where the curriculum is designed by AI. Lessons are delivered by AI, too, and the teaching AI is equipped with nice, supportive voices. Each lesson is custom designed to the needs and ability of the learner. Progress is monitored minutely and reported back to interested, authorized parties at regular intervals—mom, dad and the university. Some school systems provide community centers where children and teenagers congregate to learn and practice basic social skills.

    Everybody’s DNA is registered in the cloud. Medical services are provided online. People are connected directly to a diagnostic cloud via a variety of fitness devices and smart apparel that constantly monitor the patient’s body, including but not limited to heart rate, blood pressure, lung capacity, daily calorie burn and current body fat index. The devices are smart enough to get blood samples painlessly on demand and then do perfunctory analysis in real time. More complex procedures are administered at a medical center staffed by robots expert in a given procedure, from administering a colonoscopy to resetting a broken bone or performing a liver transplant. A few highly educated, highly trained humans staff the medical centers to handle edge cases beyond the capability of current AI/robotic technology.

    Economic productivity is almost completely separated from human labor. Intelligent robotics continue to be the most efficient way to fuel the economy. Human purchasing power is determined by a combination of market-driven forces and social support systems. Those few who have the motivation and skills required to participate in the workforce are compensated for their activities. Such work requires a high degree of education, intelligence and creativity to do the inquiry, analysis and synthesis needed to provide economic value. Their compensation is quite luxurious. Those who are not in the workforce are provided with the purchasing power necessary to acquire the goods and services that are needed for a comfortable life. People who have inherited wealth live a life of luxury, too, for as long as they can manipulate their purchasing power to maintain or increase their wealth. Some wealth is still so immense that it’s inexhaustible.

    Crime exists in varying degrees, going up and down at predictable levels. All human activity is observable in the public space. Most humans allow parts of their private space to be monitored in exchange for some type of compensation.

    Policing data is analyzed continuously to improve crime prediction models. Humans perform law enforcement activities with the help of robotic assistants. The robots confront armed assailants. The humans supervise. Transgressions such as moving violations and drunk driving are rare, as most transportation is driverless. Violent crime still occurs—always has, always will. Persons exhibiting extreme criminal behavior are isolated in prisons that are run by robots who are supervised by well-compensated humans. Administering justice is still a human activity, as is politics.

    Warfare continues, but with fewer humans acting as belligerents. Warring states as well as non-state actors assault each other using physical robots that serve as an extension of the combatant. Cyberweaponry is powered by bots, but most weapons are controlled by a human operator. One human can control hundreds of weapons. Some weapons are self-directing, using artificial intelligence that can interpret predefined assault plans and act accordingly.

    Governing policy is created by artificial intelligence under the supervision of human oversight. AI creates legislation based on information gathered from data streams. One stream is data provided by analyzing the behavior and condition of the population. The other stream is polling data. Humans increase their purchasing power by participating in frequent polls administered by government agencies. The authenticity of polling data is verified directly against the DNA of the contributor.

    The daily activities of most people are to consume content from the internet and play interactive games either with other humans, AI endpoints or a combination of both. Also, most people use shared assets for amusement; for example, riding a motorcycle or skiing at a resort. Some of the shared assets are owned by individuals. Most are owned by multinational corporations.

    Movies are still popular. Some movie scripts are created by humans with AI assistants. Most movies are produced by pure AI, according to deep analysis of the past viewing behavior of a wide variety of audiences. Movies that are custom-designed according to the profile of the particular viewer and produced on demand continue to grow in popularity. Most of the actors in the movies are human emulations designed by AI.

    Sports such as baseball, football, soccer and basketball struggle for fans as spectatorship grows in the online gaming sector. Online gambling is still popular, as is the lottery. However, participants cannot use standard purchasing power to gamble. Rather, a special recreation token, considered an extreme luxury item, is used. Lottery tickets are available for free—one per person for a monthly drawing. The winner is compensated luxuriously.

    The Stuff of Science Fiction? Not Anymore

    If you’ve gotten to this point—and I hope you have—you might be thinking that the fully automated world I describe above is pure fantasy, a pipe dream, science fiction. It might be, but let’s go through the items.

    First, in terms of agriculture, in 1790 90 percent of the U.S. population was made up of farmers. Today, the number is around 2 percent, yet production far surpasses 1790 levels. How? Automation.

    As far as robots making clothing, check out the Jaeger Rapier Weaving Machine: It makes cloth in 45-foot widths. Also, check out the video from the Wall Street Journal that describes using robots to replace garment workers in Bangladesh.

    In terms of transportation, it’s not a question of if driverless vehicles will dominate the roadways of the world. Rather, it’s when. Also, IEEE is reporting that crewless container ships are on the horizon. Norway’s Yarla Birkeland is already here.

    Flying is already highly automated. Other than takeoff and landing, human crews serve more as a safety backup in case something goes wrong in flight. In the military, attack drones are still manned by remote human crews. But, again, the humans are there more for fail safety than operational purposes. Flying may never become a solely automated undertaking, but keep in mind that it takes only two humans to operate an Airbus a380, which can seat 850 passengers.

    Amazon and Walmart are already in the food delivery business. Right now, it’s a safe bet to assume that a human is doing the delivery. But, given the dramatic improvement in delivery robots, how long will it be until that whole process is run by a machine? Also, LG makes a smart refrigerator that provides many of the food provisioning services described above.

    Oh yeah, did I mention that China is presently collecting DNA samples, fingerprints, iris scans and blood types of all residents in Xinjiang between the age of 12 and 65? We’re talking millions of people. Granted, China is not a poster child for respecting the privacy of the individual. But, let’s say that here in the United States there was an offer to give anybody an ancestry report for free in exchange for a DNA sample—or, if “free” proved to have little motivational value, the offer was for $100. People sell their blood for a lot less.

    Robotic surgery is real. Right now the robot is an assistant to the human. But, as we’ve seen with the progression of other technologies, allowing surgical robots to have increasing degrees of autonomy is only a matter of time.

    In case you’re wondering about the real future of warfare, according to an article on the BBC’s website, “As early as 2005 the New York Times reported the Pentagon’s plans to replace soldiers with autonomous robots.” Interested in a high-powered robotic machine gun that can hit a target 1.5 miles away in the dead of night? Check out the Super aEgis II, manufactured by DoDAAM.

    I could go on and on, but I won’t, tl;dr and all that. Still, given the current rate of technical achievement, when you look at any of the scenes I described above, it’s not difficult to suppose that everything I imagined is very possible. From where I sit, we’re just not that far away from the endgame: AI and robots replacing most human labor in the economic sphere.

    So, what’s the missing link? What’s it going to take to make sure that, as human labor is replaced and never to return, purchasing power is still maintained? The first step, of course, is to have some reasonable articulation of what that path to a positive outcome can look like. Fortunately, there’s a good deal of thinking going on about the topic. I’ll share a reading/viewing list at the end of this article.

    But. more importantly, it seems as if no one in government is really talking about the inevitability of the dominance of AI and robotics or the consequences thereof. The powers that be are frightfully ignorant about what’s to come. I have yet to hear the words, “We need to start anticipating the consequences of a highly automated society,” come from the mouth of anyone in government. This needs to change.

    It’s a serious issue. Today, the impact has been felt mostly factory workers, here in the United States and worldwide. When it really hits, when all professions are subject to replacement—which some estimate to be within the next five years—the shock will be overwhelming. Then, unless government “gets it,” the best recourse is to own a lot of stock in pharmaceutical and cannabis companies, because most people will need to be anesthetized to their environment to tolerate the social and economic breakdown on the horizon. Or, we can start planning to work toward a future in which ubiquitous automation provides not only a luxurious life for a few but a comfortable life for the rest of the population.

    As for me, I wish we were planning.

    Read and Listen to  More:

    Book: “Rise of the Robots,” by Martin Ford

    Book: “The War on Normal People,” by Andrew Yang

    Video: “Humans Need Not Apply”

    Video: “Obsolete by 2030”

    Podcast: “Andrew Yang and Sam Harris Discuss Universal Basic Income“

    — Bob Reselman

  • Portability Shmortability: Most Applications are Going Nowhere

    Portability Shmortability: Most Applications are Going Nowhere

    I’m a huge fan of the idea of taking our super-duper applications and deploying them where it makes most sense. I always have been. Those of us in application development spent forever bemoaning the fact that our applications weren’t all that portable between operating systems. In fact, looking back through the history of computers, you find there has always been one roadblock or another to stop portability.

    First there was byte order and instruction set, and we’ve largely overcome those. Internationalization wreaked havoc on us all for a while, and we overcame that. We’re still pretty mightily struggling with OS-dependent things such as windowing systems, but browser-based apps have largely overcome those also. Java takes a lot of heat for a lot of reasons, but it did improve portability of apps written in it. Maybe not “Write once, run anywhere,” as was often claimed—things such as file locations were still OS-specific—but still a lot closer than compilers that were deeply tied to the operating system they ran on. Virtualization and containers have enhanced portability in another dimension, letting the developer take the OS with the app.

    We have overcome a ton of issues, and advancing technology has introduced a ton more. Now we have to either choose mobile targets or find a tool we like that can generate code for wildly different mobile systems from a single source. DevOps really isn’t helping with this process—though it will, eventually. By automating the build process, the target will be switchable at the targeting end, meaning our app—given a tool that can multi-target—can be built for whatever mobile platform the cool kids are playing with today, and the ones that are considered “cost of entry” can be generated in the same run.

    So far, so good, right? We find problems, we overcome problems, DevOps helps us overcome them faster with a wider selection of outputs …

    And then we hit the data.

    Vendors in a variety of spaces would love to paint you a picture where it doesn’t matter. It does. The mass of your data, wherever it resides, is a pull like gravity to new application development. And it should be.

    Some will try to explain to you that in the age of APIs, it doesn’t matter where the data actually resides: You can get it.

    But it really does matter, and this is not Utopia. Even if you ignore the performance implications of data being latency-infested miles away from applications that need to use it non-stop, those APIs are only a viable option if the data can be secured and protected. By definition, an API is an advertisement that whatever data/service it fronts can be found right here. For applications that traditionally are data center apps, it might be possible to expose your customer file or other critical data via API and then lock down access to the API by IP …

    But then we quit ignoring the latency problem. If every bit of data an app needs has to flow from San Francisco (or Tokyo, or Bejing, or London, or wherever) to Seattle before the calling app can do what it needs to and then has to be sent off again to Albuquerque (or Rostov, or Madrid, or wherever) the delays will, by definition, be more than they would be if the app sat with the data and had to transfer over a 1GB (or 10GB) connection between servers before sending it to users.

    This means most data-heavy apps, no matter the level of agile or DevOps, aren’t likely to move. Vendors with a stake in the process will trot out the one or two customers they convinced to make such a move to tell you how great it is, but performance math and security sense tell you it is only great until things get busy or that exposed API is used by bad actors to vacuum user data.

    Some applications have smaller datasets and can move the data with the app. Other applications can be architected not to care where the data is. A few don’t need a database at all. Move these applications to whatever platform makes sense. Make the CD for them retargetable.  Most serious business applications rely too much on core data to fall into these categories. And thus, they aren’t moving, unless someone comes up with a viable way to move a ton of data between platforms that is safe and fast and effectively translates where necessary.

    So, that’s what we need to see next in the march of portability. We currently have containers that can be run pretty much anywhere and languages that will work on pretty much any platform. We need data tools that can alleviate the current set of non-portability/latency constraints and get data where it is needed, when it is needed, securely.

    Unfortunately, I don’t see that happening soon. So addressing the issue, “Where do we target our new application?” will have to start with the question, “Where is the bulk of the data it needs?” At least for now.

    Most of you knew that, because you’re living it. I just thought you might have a use for a third-party article about it. Because someday, you will have this conversation with someone who isn’t living it. Enjoy.

    — Don Macvittie

  • GitLab Moves to Automate DevOps

    GitLab Moves to Automate DevOps

    GitLab is looking to eliminate the pain of setting up and maintaining DevOps processes by making available an option that automates DevOps processes end to end using a prescriptive approach defined by GitLab.

    The latest version of GitLab includes an Auto DevOps option that organizations can employ to automate the building, testing, code-quality scanning, security scanning, license scanning, packaging, performance testing, deploying and monitoring their applications.

    GitLab CEO Sid Sijbrandij said the company is now able to make this service available because GitLab 11.0 can now be deployed as a set of more integrated applications running on top of a cluster based on Kubernetes container orchestration software. That support for Kubernetes also makes it easier to deploy GitLab 11.0 in a public cloud or on-premises environment as needed. To facilitate that process, GitLab now includes templates that simplify the spinning up of Kubernetes clusters as needed, Sijbrandij said.

    To further its support for Kubernetes, GitLab also decided to start deploying the software it makes available as a cloud service on Google Cloud Platform. Previously, GitLab had been committed to Microsoft Azure. Microsoft recently announced plans to acquire GitHub.

    GitLab is all in favor of automating DevOps processes, but doesn’t think organizations should be forced to move everything into a public cloud to achieve that goal, explained Sijbrandij. Organizations can still elect to construct their own DevOps processes using GitLab as they best see fit.

    But as interest in DevOps continues to rise, more organizations now would prefer to rely on a set of best practices that are defined to make them much easier to consume, he said.

    GitLab 11.0 also adds security scanning capabilities for application built using .Net and Scala programming language in addition to existing support for C, C++, Java, Python, Ruby on Rails, Go and PHP.

    In general, Sijbrandij noted that organizations that embrace DevOps usually find themselves trying to manage various islands of automation with mixed success. Auto DevOps is intended to provide a more comprehensive approach that eliminates the friction that often exists between those various islands of automation. Sijbrandij acknowledged that a prescriptive approach may not appeal to every IT organization, but as traditional enterprise IT organizations start to embrace DevOps, many of them prefer a more platform-centric approach that’s easier to both set up and maintain, simply because they lack internal DevOps expertise.

    Sijbrandij also noted that as DevOps gains more traction in the enterprise, many of the decisions about what platforms to employ will be driven from the top down. Today, most acquisitions of DevOps tools and practices are driven from the bottom up, which accounts for why there are already so many silos of automation, he said.

    Over time many organizations will conclude that the opinionated approach enabled by Auto DevOps will free developers to spend more time on writing code instead of on managing the infrastructure employed to create it, Sijbrandij said. Most traditional enterprise IT organizations today are at least aware they should be employing DevOps practices, they just know exactly what such an effort fully entails.

    Thanks to advances in automation, many IT leaders may never know or care what specific DevOps processes are in place. After all, what most of them are primarily interested in how fast the next set of application code is going to be delivered.

    — Mike Vizard