devopsy.kinsta.cloud

Tag: cloud services

  • Managing Complexity in Dev/Test Infrastructure Using Machine Learning

    Managing Complexity in Dev/Test Infrastructure Using Machine Learning

    Businesses are increasingly developing software solutions that can drive innovation and digital transformation. As the number of developers increases in the organization, the ratio of developers to IT also increases over time. The net result: more dynamic and complex dev/test infrastructure requirements and slower support from IT.

    Developers and testers often cannot get their hands on infrastructure environments in a timely manner to accelerate delivery of software applications to their customers. They have to deal with lack of timely support from IT and long waits for hardware resources and for provisioning and configuring of the environments.

    How does engineering IT manage this complexity and deliver more agility to their engineering stakeholders?

    Provide Self-Service Creation of Dev/Test Sandbox Clouds

    Engineering IT teams can increase the velocity of development and testing if they enable self-service ability for dev teams to quickly provision their own lab infrastructure and application stacks at a click of a button.

    Self-service, API-driven infrastructure is a fundamental requirement for enabling developers to write code (which can be done using their favorite programming language) and make RESTful API calls to the underlying programmable infrastructure. This allows them to dynamically and automatically manage initial deployments and configurations as well as manage ongoing, automated dynamic provisioning of infrastructure, autoscaling, monitoring and alerting.

    All this automation removes the confusion and error-prone manual steps for the entire application delivery process, including development, testing, staging and production deployments. This, in turn, accelerates software delivery and increases quality.

    Additionally, if self-service is available as a SaaS-based delivery model, it is easy to add more features and workflows very quickly without having IT doing a major upgrade.

    Breaking these silos improves collaboration between teams, accelerates velocity of software development, improves infrastructure utilization, increases overall operational efficiency and reduces costs.

    Eliminating Silos in the Dev/Test Infrastructure

    Silos are common in companies where various specialist teams (storage, networking, security) form fiefdoms around their respective functional areas. Silos impede velocity because they lead to complexity of operations, lack of consistency in the environment, and lack of automation.

    Automating across silos turns into to an exercise of custom scripts and lot of “glue and duct tape,” which makes maintenance and change management complex, slow and error-prone.

    Breaking these silos improves collaboration between teams, accelerates velocity of software development, improves infrastructure utilization, increases overall operational efficiency and reduces costs.

    A hyperconverged cloud design with a software-centric, scale-out architecture tightly integrates compute, storage, networking and virtualization resources and other technologies from scratch in a commodity hardware box supported by a single vendor. Also, companies can keep costs under control by leveraging scale-out cloud designs that make it easy to start small, grow based on demand, and stay close to the right size and customer demands.

    Additionally, development teams should have the ability to quickly deploy/clone/share complex multi-tiered application stacks between various teams and between development and testing, breaking the silos within the development organization.

    Automate Operations, Monitoring and Patching

    Engineering IT teams need to have complete visibility and control of their entire stack from the infrastructure up to applications. They need intelligent software to monitor the hardware and software stack, manage large-scale clusters and automatically handle routine but time-consuming and complex operations such as failure handling, patching, security updates and software upgrades.

    Running a service with traditional sysadmin teams who execute the above activities manually becomes expensive—especially if they operate in server, storage, networking and security silos—as the dev/test environments become more dynamic and the demand for more environments and projects grows.

    However, it is possible to use an intelligent private cloud platform that leverages hyperconverged scale-out designs, machine learning software and a SaaS-based operational console to reduce complexity and increase the agility of dev/test teams.

    Cloud-based monitoring and advanced analytics dramatically reduce the need for experts in different parts of the infrastructure, scale linearly as the size of the operation increases and cut operational complexity by 90 percent.

    Manage Resource Management Using Machine Learning

    Applying machine learning to infrastructure management means intelligent software could learn about operational patterns, anticipate capacity needs, raise alerts about security anomalies, self-monitor and self-heal your environment in the face of failures, intelligently apply security patches and automatically upgrade hardware and software systems without any downtime. The next generation of infrastructure management will be driven by advances in machine learning and artificial intelligence, where the infrastructure is able to basically “drive itself” with minimal user intervention.

    This will help engineering IT teams optimize resource usage and capacity based on current and future dev/test demand and also be able to better handle the availability and performance they can deliver to their engineering teams. A lot of efficient resource management comes down to capacity planning, utilization monitoring, right-sizing of workloads, demand forecasting and detecting zombie VMs and unused resources.

    Demand forecasting and capacity planning can be viewed as ensuring there is sufficient capacity and redundancy to serve projected future demand with the required availability. Capacity planning should consider organic growth, which stems from natural service adoption and usage by dev/test teams. Having intelligent predictive analytics and machine learning can help greatly with accurate forecasting, alerting and providing lead time for acquiring additional capacity.

    Better insights into how the infrastructure is performing can also help in fine-tuning performance of end user workloads. For example, an intelligent system that is monitoring a workload for storage performance might recommend using SSDs instead of spindles to increase the IOPS and improve workload responsiveness.

    An intelligent private cloud platform that uses hyperconverged scale-out designs, machine learning software, and a SaaS-based operational console can reduce complexity and increase the agility of dev/test teams.

    About the Author / Kamesh Pemmaraju

    Kamesh Pemmaraju leads product management, product marketing, and technology partnerships at ZeroStack. Kamesh is a seasoned product management/marketing leader with a strong engineering and consulting background. In his most recent product roles at Mirantis and Dell, he focused on delivering private and hybrid cloud solutions to enterprises. He holds a Master of Science in Computer Science from Indian Institute of Science. Connect with him on LinkedIn and Twitter.

  • Q&A: Michael Beckley, CTO/CCO, Appian

    Q&A: Michael Beckley, CTO/CCO, Appian

    Recently I had the chance to speak with Michael Beckley, chief technical officer and chief customer officer at Appian, about his company’s move to go public and the DevOps space in general. For those of you who don’t know Michael, he is a founder of Appian and leads the company’s technology vision and oversees customer initiatives worldwide. He also serves on the Industrial Advisory Board for the University of Virginia’s Computer Science Department and is a member of the University of Maryland Advisory Board in Computing. He has an interesting take on the IT space and DevOps in particular. Read on for our discussion.

    Alan Shimel: Michael, what do you think has been the “special sauce” that has led to Appian’s success to date?

    Michael Beckley: With Appian, companies build unique business applications up to 20x faster and mobile-enable and modernize existing systems. For example, a company like Ryder was able to build a web and native Android tablet application that cut the time it took to rent a truck in half.

    But the “special sauce”? That’s Appian’s culture of innovation. Our engineers think long-term and invest in breakthrough tech like our patented SAIL UI technology for zero code native mobile apps. Thanks to SAIL, the web apps you configure in Appian automatically deploy to iOS and Android as native, full-featured apps or specialized micro apps we call “Sites.” As long as we get the culture right, we expect to continue to deliver enduring value for our clients.

    Shimel: With becoming a publicly traded entity, some companies have found “managing to the street” does not match their own long-term business plans and their customers needs, how does Appian avoid this?

    Beckley: Companies, like people, are an aggregation of their habits. Appian’s habit has always been to hold to our long-term vision of enabling every company to be a software company. We are not your typical Silicon Valley tech company. Our roots are in DC, we largely bootstrapped this business, and learned to be good stewards of capital while helping to support the core missions of federal agencies and some of the world’s largest brands. The proceeds following this IPO will be invested back into our business to benefit our customers.

    Shimel: What do you see post-IPO being the biggest areas for growth as it relates to new products and services?

    Beckley: We  plan to invest in innovating and enhancing our platform. While our platform is industry-agnostic, we have recently made, and plan to continue to make, investments to enhance the expertise of our sales and marketing organization within our key industry verticals of financial services, health care and government. We have made, and will continue to make, investments in research and development to strengthen our platform and expand the number of features available to our customers. As we continue to increase the functionality of our platform and further reduce the amount of developer skill that is required to build robust applications on our platform, we believe that we have the potential to expand the use of our platform.

    Shimel: Digital transformation is a term that is being used a lot these days. How far along the adoption curve are we on digital transformation?

    Beckley: While many speculate that digital transformation is just a tech industry buzz word, in reality, digital transformation is a living, breathing effort. According to a recent survey, digital transformation is a C-level priority and is considered a top focus for almost 50 percent of all companies this year.  From Appian’s perspective, we believe that the utilization of low-code software to drive digital transformation is still relatively new and our customers will be able to succeed in their digital transformation efforts by incorporating low-code development.

    Shimel: Where does the DevOps market intersect with the Appian business model?

    Beckley: Organizations are turning to DevOps to accelerate development and bring an application’s capabilities to a user quicker. The Appian low-code development platform fits into the DevOps model and supports the seven major DevOps categories including code, build, test, package, release, configure and monitor.

    Overall, our business model supports the DevOps market; we’re living in a world where citizen developers are playing such an important role in the enterprise and our platform empowers the citizen developer while alleviating developer resources to focus on the more challenging, business-critical developer work.

     Shimel: What about containerization and microservices? How do these technologies affect Appian?

    Beckley: Appian already uses containers and microservices. Appian’s enterprise customers often require a mix of cloud, hybrid cloud, and on-premise installations. We view containers and microservices as enabling technologies to support the diverse workloads and deployment models required by our enterprise clients.

    Shimel: What about security? What challenges does cybersec pose to Appian’s development of their platform?

    Beckley: Appian works with some of the nation’s most important and highly regulated industries, therefore we have built a comprehensive security and compliance structure that meets several industry standards. We frequently undergo third-party audits to ensure that we are always operating effectively to protect our customer data.

    — Alan Shimel

  • New Relic Extends Monitoring Reach

    New Relic Extends Monitoring Reach

    New Relic this week at its Future Stack 2017 conference extended the reach of its monitoring services via the release of a software development kit (SDK) that makes it possible for IT teams to monitor custom applications.

    In addition, New Relic unveiled Health Map, which makes it much easier for IT administrators to correlate data collected from New Relic application and infrastructure monitoring services. Todd Etchieson, vice president of product management for New Relic, says developing Health Map was a natural next step after the expansion of its services into the realm of infrastructure monitoring. That capability should significantly reduce the finger-pointing that often occurs between developers and IT operations teams anytime there’s an application performance issue, says Etchieson.

    New Relic is also adding billing and cost management features that enable customers to monitor costs in real time as well as plan for future Amazon Web Services (AWS) resource requirements. Rather than having to invoke a separate tool, users can leverage New Relic’s extensive history of relying on AWS to host its tools.

    Application performance management (APM) tools from New Relic first gained prominence among developers who preferred to monitor their applications using a service. Because New Relic is deployed on AWS it’s especially well-suited to monitoring applications deployed on AWS. The company this week also announced that its monitoring tools now come with 20 pre-built integrations to AWS services including Amazon Kinesis Firehose, Amazon Elasticsearch Service, Amazon Route 53, Amazon EC2 Container Service and Amazon EC2 Container Registry.

    Now Etchieson says New Relic is expanding its reach in the enterprise as organizations embrace a more integrated approach to DevOps. In fact, he notes half of its customers are already consuming multiple services. Driving that adoption is a need to roll out more applications, many of which are based on microservices enabled by containers. This fact is forcing organizations to confront their DevOps issues because of the dynamic and ephemeral nature of containers. New Relic added support for container monitoring back in 2015.

    Etchieson says New Relic doesn’t necessarily care where IT organizations choose to deploy their applications. Because New Relic made its APM software available as a service, use of APM expanded considerably. The company currently has 1,700 enterprise customers, he says.

    Not all those customers are very far down the DevOps path, but more than a few organizations have begun their DevOps starting with an APM tool. In fact, in many organizations DevOps is a bottom-up transformation that begins with developers seeking more information about how their applications are performing.

    As DevOps continues to evolve, the divide between application and infrastructure performance monitoring tools will continue to narrow. There may be different roles inside an IT organization that require different views to address specific issues. But at its core DevOps is all about providing cross-functional IT organizations with the context needed to resolve issues faster. In almost every case, that will require correlating data across both the application and infrastructure stack.

    — Mike Vizard

  • Boots and Sandals: Know the Market

    Boots and Sandals: Know the Market

    Boots and sandals are basically the same thing—something to protect your feet while you walk about. The thing is that they are intended for slightly different purposes. You wouldn’t wear sandals outside in the middle of a Wisconsin winter*, and you wouldn’t wear boots to your day off, hanging at the beach. The thing is that boots and sandals have been around forever, and it is relatively simple to gauge the merits of each in a given setting. The same is not true of DevOps tools.

    Cowboy Boot Sandals.

    * Every few years there is a news report about someone found only in sandals in sub-zero weather, they’re not the norm, and there’s a reason they’re in the news.

    Rate of Change

    The rate at which DevOps markets are maturing is a pace that few enterprises can keep up with. When both development processes and operations processes are changing, new tools are being introduced and the two groups are increasingly working in tandem to support the business, yet another market space is not what they want to study and yet another tool in the tool chain is not something that they want to just “pop in.”

    But the DevOps space is young, and there is a lot coming that can help achieve the goals. I have had reason to look into application release automation (ARA), cloud security auditing and global file systems (GFS) in the last few months, and all three bring something to the table in terms of helping IT adapt and automate. The problem is that in each case, the market is a bit muddied. ARA must explain why it does more than Jenkins alone, cloud security auditing must explain why it is better than calling the cloud vendor APIs to do the same task, and GFS must explain why it is more than a cloud storage gateway and how that helps the enterprise.

    And they can. But when considering them, you need to do the research and figure out the market and how that market applies to your organization. Which is a bit unfortunate. devopsy.kinsta.cloud and other sites can help by talking about the tools over time, and the vendors’ own promotional pages can help you understand what gap they’re filling, but it’s legwork (though, in many cases, the legwork is worth it). ARA marries app provisioning tools with CI tools, giving a unified workflow, for example. And it makes the deployment process more agile. If you are running many projects that deploy on a regular (not hourly or even weekly, but regular) basis that you may one day want to deploy to a different environment, ARA can help, along with the wealth of other functionality it provides, bridging the two sides of the DevOps coin is probably its largest driver. The same type of thing is possible with the other two toolsets I mentioned, but trying to keep this short enough to read, so will leave it to those interested in GFS or Cloud Security Auditing to research them.

    The Key is Knowing Why and Which

    In the end, if you’re building out DevOps, you want to automate the process as much as possible. That means as the markets mature, you will want to explore what’s new. These tools can help many organizations provide a smoother or more secure process, but knowing the reason the market exists is the first step, and only then can you decide if you want to evaluate products. They do not all help all customers, so seeing which are right for you—there is no need for either GFS or Cloud Security Auditing if you are a single office with no cloud presence, for example—will help you narrow which toolsets might improve your DevOps efforts.

    And whatever you do, don’t buy the “sandal cowboy boot” of IT. It might look good, but won’t be very useful in either of the environments it’s intended for. 😉

    — Don Macvittie

  • Protecting Data in Public Cloud and Hybrid Cloud

    Protecting Data in Public Cloud and Hybrid Cloud

    Backup, business continuity (BC) and disaster recovery (DR) have all been a critical part of IT for more than 30 years—ever since we began relying on technology to run our businesses. Traditional solutions were designed for the world of on-premises infrastructure and structured applications and relational databases. But the world is changing. In the last blog, I talked about the era of digital transformation and the impetus it has on rethinking and reinventing the fundamental backup and recovery architecture for workloads moving to the cloud and applications being born in the cloud.

    The Motivations Leading to Reinvent Backup and Recovery

    Application and data platforms are undergoing the biggest transformation since the dawn of computing. There are several forces at work:

    • New applications. The third-generation applications are geo-distributed, scale across multiple systems, are always on and typically deployed in a cloud-first model.
    • Existing applications are moving to the cloud. They aren’t going away, but companies are moving some or all of them to the cloud. They still need backup and recovery.
    • RPO and RTO windows are shrinking: Enterprises want “always-on” and gone are the days when you could do nightly backups.
    • Smaller companies will go all-in on public cloud. SMBs don’t want to be in the business of IT. They’ve been driving the rapid growth of cloud applications and platforms.
    • Enterprises will build hybrid clouds. Enterprises will deploy applications and data across on-premises and public cloud environments. Scale, compliance and other factors mean they’ll need to keep some systems on-premises.
    • Everyone will use multiple clouds. No one will bet their business on one cloud or one provider. Even now, enterprises are splitting workloads across clouds or cloud and on-premises. Development and test may use one cloud, while the same application might be deployed in a private cloud or a different public cloud.

    Cloud’s Impact on Backup, Recovery and Continuity

    Cloud gives organizations much more agility, operational savings and a pay-as-you go model. Public cloud providers can also build much more resilient infrastructure. Amazon guarantees 99.95 percent availability for EC2 and 99.99 percent for S3; S3 is designed for 11 nines of data durability, with multiple availability zones. Because the cloud is so reliable—and cheap—it’s quickly becoming a backup target for on-premises data. But that shouldn’t trick us into believing backup and recovery are “built-in” when we run applications in the cloud. Even Amazon itself recommends backup services for all AWS-native applications and cloud databases.

    While service availability and data resiliency addresses infrastructure business continuity and disaster recovery, it doesn’t provide point-in-time recovery or application-level intelligence for backup and recovery. As good as the cloud platforms are, they don’t protect against logical errors. And research shows 8 out of 10 errors are logical errors—data corruption, user errors.

    But Won’t Existing Backup Products Work in the Cloud?

    As we noted above, traditional backup and recovery products don’t meet the needs of cloud applications—even for existing applications that have moved to the cloud, and not just because they were built in a different era. Cloud and distributed architectures present several other challenges:

    • Cloud breaks the media server based architecture of traditional solutions. Cloud applications and data don’t reside on a specific array or disk, so you can’t easily backup what you can’t see. Backups also don’t capture configuration data in the cloud, such as AWS Cloud Formation templates.
    • Cloud doesn’t speak the same language. Legacy solutions talk to tape, disk or virtual disks. Backup and recovery in the cloud means integrating with the right protocols, such as the S3 API or Google Cloud Storage.
    • Backup appliances can’t be moved to the cloud. Existing backup appliances that work extremely well on-premises can’t be picked up and moved to the cloud.
    • Traditional backup agents won’t scale. If you could get a backup agent running in the cloud, it wouldn’t scale well across dozens or possibly hundreds of nodes.
    • VM is not the right layer of abstraction: Look for technologies that provide a scalable application-centric view of data management and data protection, introspect application data and use global semantic de-duplication to achieve storage efficiencies, instead of relying on traditional de-duplication techniques. The benefit of this approach is fine-grained and highly space-efficient data protection that can span clouds over network links.
    • Cloud Gateway or Migration Services: Unidirectional only.

    The problem of backup and recovery of cloud applications is novel there are three critical things a cloud backup and recovery architecture should have:

    • Elastic compute only. The architecture should scale efficiently on elastic compute instances. There shouldn’t be any CapEx costs for servers or appliances.
    • No media servers. Backing up large scale out databases requires direct a parallel streaming architecture for data movement between the database and secondary storage. Legacy backup architectures rely on media servers that would quickly become choke points. Direct parallel streaming also allows the data to remain available in native formats.
    • Semantic deduplication. Scale-out application databases typically have a 3x replication factor. If you backup individual nodes or even manage to snapshot the whole database, two-thirds of the backup data is redundant. Over time, backups will be 75 percent to 80 percent inefficient without semantic deduplication that works across a distributed architecture.

    — Brian Yin

  • Azure vs. AWS for the Enterprise

    Azure vs. AWS for the Enterprise

    In a previous post we discussed Azure & OpenStack, the less-explored hybrid cloud, and we thought we’d segue on the hybrid cloud theme, and follow up with an additional post focusing on a question we’re often asked …

    Azure or AWS?

    It’s one of the most common questions enterprise customers today face as they look to go to the cloud. According to this year’s Gartner’s Magic Quadrant report, the two cloud powerhouses are the clear market leaders, and as they each add new services to their arsenal and expand their service offerings, customers must decide which one is best-suited for their business needs. This, of course, is in the context of public cloud, whereas the question of OpenStack vs. VMware usually arises in the context of private cloud. And with reports popping up all the time that demonstrate growing numbers around every hybrid cloud mix-and-match scenario, many enterprises are debating on what to bet their cloud strategy.

    To begin this comparison, it’s important to understand how each has evolved. Azure was built as a holistic platform and has been geared toward the enterprise from the beginning. It began as a platform as a service (PaaS), making it easier for developers to build applications without worrying about the servers on which they’re running. In addition, part of the Azure approach always has been to think about how to integrate each service to the platform as a whole and support enterprises’ IT flexibility with keeping specific workloads on-premises while complementing their IT requirements with services running on the cloud.

    AWS, on the other hand, started scaling from the infrastructure from the beginning. Its approach has been to continually be ahead of the innovation curve and offer services that IT operations can easily comprehend and use to support their compute and storage requirements on demand.

    Market Share Comparison: The Path to Growth

    It’s not hard to see why both AWS and Azure have grown into their roles as the market leaders. Today more than 70 Amazon services can be utilized, including the basic compute, storage and network building blocks, as well as “up the stack” services such as database as a service and even frameworks to easily scale mobile applications, databases and more. AWS has a massive public cloud with the largest data centers in the world, and excels at allowing their customers to choose the building blocks they need to create a variety of service offerings.

    That said, Azure is growing as well, and quickly gaining ground on AWS. Microsoft has moved aggressively to match AWS by enhancing and building up its array of offerings. The company has been selling to the enterprise market for decades and, as a result, has garnered strong relationships with many large global organizations. These efforts and relationships seem to be paying off, as a recent survey of 100 CIOs by Morgan Stanley predicted that Azure would be the largest infrastructure-as-a-service (IaaS) vendor by 2019.

    Enterprise-Grade Product Capabilities

    AWS and Azure both provide comprehensive enterprise cloud services, and they both do it well. The core features of AWS include a large suite of infrastructure and application services that enable running nearly everything in the cloud, from traditional enterprise applications to large web-scale workloads. AWS is the clear winner these days in terms of the number of services it offers; the services are well-connected and provide full cloud capabilities for most enterprise needs.

    Azure is also fast closing the gaps with the enterprise cloud market, including with large-size virtual machines, StorSimple Virtual Array, itsNAS offering and the SQL Data Warehouse. In addition, with its recent announcement of Azure Stack, and via solutions such as Cloudify, Microsoft strengthens hybrid architecture and workload mobility between on-premises and public cloud workloads.

    Other factors that are driving enterprises to adopt Azure include the Azure Security Center (ASC), which provides a centralized view of security policies and configurations from multiple Azure subscriptions; the ability for customers to save time and effort through unified operations tools such as SQL Manager, Visual Studio and other such services; and seamless VM migration to migrate servers between disparate environments.

    Growth Strategies

    Microsoft also plans to invest heavily in cloud growth going forward, with a big push and investment in new features and infrastructure. Microsoft’s Intelligent Cloud unit continues to be the focal point of the company’s growth strategy, as the company claims that nearly 60 percent of Fortune 500 companies use at least three of Microsoft’s cloud offerings.

    Both AWS and Azure, in fact, are investing heavy capital expenditures on worldwide data centers and servers. AWS, for example, has just announced a new Asia Pacific region in Mumbai, India, with two availability zones. And with the introduction of new regions in the U.K., Microsoft has announced 34 Azure regions around the world, with 28 generally available today—more than any other major cloud provider. With certifications such as MCSE and many more, Microsoft has more certifications than any other cloud provider in the industry.

    Both AWS and Azure are poised to continue growing, and both companies have big futures in the cloud market. It’s clear, however, that in the cloud game today, Azure is narrowing the functional gap. Azure gives enterprises the ability to use existing Microsoft software licenses under the Enterprise Agreement (EA), giving enterprises better terms and packages. Microsoft enterprise users are potentially ‘power users,’ and prefer a DIY approach versus pre-packaged and opinionated options. This leaves the market open to more collaboration, such as through the use of Docker Swarm. With this type of approach, users have control over the way they run their apps on various platforms, with open-source orchestration to allow consistent management and control of their applications and services.

    — Sharone Zitzman

  • Health Care’s Shift to the Cloud via the Hybrid Cloud, Part 2

    Health Care’s Shift to the Cloud via the Hybrid Cloud, Part 2

    In part one of this series, we looked at how the healthcare industry has changed, requiring integrations and interoperability that were not previously needed.

    Having established that the cloud is the way to go and that the hybrid cloud is the stepping stone hospital CIOs are comfortable with, we must ask how DevOps can help.

    DevOps in Health Care

    The legacy HL7 connection technology healthcare facilities use is a great example of how DevOps brings interoperability to health care and the hybrid cloud. “We have a cloud-hosted HL7 interface that can connect to any number of different hospitals and to our system as well. That’s something that would be very difficult to do without a cloud-based HL7 interface,” said Mayank Thanawala, CTO of HealthLoop, a provider of a cloud-based platform to automate follow-up medical care.

    Another challenge hybrid cloud can address concerns the registries that track patient episodes and the bundled payment initiatives that health insurance companies use to pay the hospital systems, Thanawala noted. “The challenge is that hospitals have to get the patient data into the registries in order to receive payment.”

    A hybrid cloud makes it easier to submit data to the registries because it provides an opportunity to connect to the registries using a system that already connects with the EMR system also and contains other data useful for registries and payments.

    Thanawala illustrated registry solutions in the hybrid cloud. The Comprehensive Care for Joint Replacement Program is a large registry hosted by Medicare CMS. Hospitals need data such as diagnosis histories from medical records and patient-reported survey information to submit to this registry. “So, in an example of a hybrid cloud, HealthLoop is a cloud system that uses a web interface to ask the patients the questions and collect the data. Then we use this hybrid cloud-based HL7 interface to query the on-premises EHR (electronic health records) system for all the rest of the data, fill in the blanks, create this file for this registry, and basically solve the hospital’s problem,” he said.

    How DevOps Speeds the to the Hybrid Cloud

    DevOps and the move to containers have helped accelerate health care’s move to the hybrid cloud. “When you deploy a Ruby on Rails API in, say, a Docker container, because of how the container is configured it is going to take care of logging for all of the applications and transactions and putting those in a store that will keep the transactions for the seven-year period that HIPAA requires,” Thanawala said. This automation saves the hospital CIO the step of having the logging set up.

    In the same way, if encryption of data both in flight and at rest is set up automatically in the container default configurations, that’s another matter the hospital CIO doesn’t have to worry about, he said.

    If applications are set up as microservices in containers so that they already perform all these functions when containers are created or scaled up, the hospital and HIPAA requirements surrounding these activities automatically are met. “This lets startups like HealthLoop focus on building app functionality so we don’t have to worry about these other more involved matters,” Thanawala noted.

    Even with health care’s complex move to the hybrid cloud, the modularity, completeness and singularity of applications in container technology as microservices make it possible to configure the containers, use them as building blocks and to build out efficient, scalable solutions in the cloud, he said. “It allows companies such as Catalyze, a medical systems integration and compliance organization, which we use to offer its products in a PaaS (platform-as-a-service) model using compliant containers so that we can simply deploy the containers and not have to worry about building compliance.”

    Saving Time, Effort, Worry

    HealthLoop’s story is a great example of how the automation and modularity of DevOps components such as containers and microservices can make or break new business opportunities, such as taking health care to the cloud and solving interoperation challenges.

  • Health Care’s Shift to the Cloud via the Hybrid Cloud, Part 1

    Health Care’s Shift to the Cloud via the Hybrid Cloud, Part 1

    The changing healthcare scene is dragging hospital systems into the cloud to enable the interoperability necessary for the industry to send electronic medial records (EMRs) and other data to outside systems such as registries that enable bundled payments.

    Because health insurance companies and Medicare only pay for patient episodes rather than for each subsequent treatment regardless of whether they’re medically necessary, hospitals must follow each treatment and take responsibility for them all. This gives hospitals incentive to get treatments right the first time.

    devopsy.kinsta.cloud paints the landscape behind the healthcare industry’s DevOps-enabled move to the hybrid cloud together with illustrations of how DevOps speeds that transition.

    The Old World Order in Health Care

    The Old World order for health care dictated that each hospital maintain its own isolated, protected, siloed and closed systems and patient health records, which in the not-too-distant past were maintained entirely in paper documents and charts. “That closed system was a file cabinet,” said Mayank Thanawala, CTO of HealthLoop, a provider of a cloud-based platform to automate follow-up medical care.

    Patients likely would have outpatient visits with their doctors at the local clinic adjacent to the hospital. Interoperability between the hospital systems and the clinic systems amounted to hospital staff faxing over the patient record or chart or sending it over via Sneakernet.

    A transition still occurring to move all that into EMRs/EHRs (electronic health records), although in a similar model as paper, with closed systems tied to the hospital and local clinics. “These are mostly not cloud systems but are rather on premises with largely client-server topologies,” Thanawala said.

    Pain Points Driving Health Care’s Shift to the Cloud

    In the old billing model, the hospital or physician billed the patient’s insurance or Medicare for each individual visit and procedure, treating them as separate, disconnected events even when the patient had the same treatment twice for a single need. “Insurance companies and specifically Medicare are now saying, ‘Hospitals, we don’t really care about all these complications, we care about the overall cost. If your patient had to go to the emergency room and had to have you perform the same surgery over again, that’s on you. We’re only paying for that initial surgery,’” Thanawala said.

    The result is that the hospital must know whether the patient had additional care or the same care twice for the same need, where they had the procedure or treatment and what that was. A closed system, electronic or paper cannot address these requirements. “That is driving the need for interoperability between systems across different organizations, which is driving the shift to the cloud to handle that need,” Thanawala noted.

    Delays in the Industry’s Shift to the Cloud

    There is a breadth and complexity to health care that requires a great deal of cloud infrastructure, which is creating delays in the industry’s move to the cloud. Health care and HIPAA regulations demand tight, auditable security, which is easier to achieve with the current on-premises systems. “There’s not a lot of confidence in cloud security. It’s getting better, but hospital CIOs have great responsibility and want to be the kings of their domains where security and control are concerned,” Thanawala said.

    At the same time, and because it can be problematic for hospitals to update workflow technologies, the cloud must support legacy connections such as HL7 that are not going away. “HL7 is designed to work within a network. Once you talk about cloud systems accessing HL7, you are talking about site-to-site VPNs for security and developing to embedded older technologies such as versions 8 or 9 of Internet Explorer,” he said.

    The fact that the data so heavily affects people’s lives, including life-or-death situations, makes the proper scalability, security, stability, uptime and privacy, as well as all the other goals of the cloud and DevOps, doubly important. “It adds another level of scrutiny and pressure to the move to the cloud,” Thanawala said.

    Take, for example, auditing, which touches every piece of healthcare data with such intensity. “From a systems perspective, you have to start thinking about auditing systems and tracking all transactions seven years back, which puts a lot of pressure on logging systems and databases,” he noted.

    As for uptime, if a healthcare system goes down, how does it affect life-or-death situations in that moment?

    How Health Care’s Transition to the Hybrid Cloud is Unique

    “The hybrid cloud gives the hospital CIO the ability to maintain control over what they need to control and still look to the cloud to enable the integration and interoperability with outside systems,” Thanawala said. This is important because of the many factors that make transitioning to the cloud unique in the healthcare industry, including:

    • Users in hospitals blame their systems—systems the CIO is responsible for—for any number of challenges and errors.
    • Healthcare facilities don’t really have to scale in the cloud quickly, because in health care there already is a defined patient population and facilities won’t have a large number of new users signing up that it can’t handle.
    • Service providers must convince hospital CIOs that collaborating with cloud technologies, including opening up VPNs and ports where necessary, is worth the risk.

    Digging into DevOps

    In part two of this series, we’ll explore how DevOps helps to enable the hybrid cloud in health care.

  • User Identity in Focus as Cloud, Device Use Grows

    User Identity in Focus as Cloud, Device Use Grows

    Recently, I had the pleasure of attending SailPoint’s Navigate ’16 identity management conference. I’ve always been interested in the identity management aspect of security. Many of the first wave of identity management vendors launched around 1999, when I started covering information security as a full-time focus. These vendors dealt with all of the identity challenges that we deal with today—provisioning employees and partners to the resources they need efficiently and effectively, keeping systems secured by making certain people can only access the right data and resources, improving user experience through efforts such as single sign-on, managing privileged accounts, and all the other associated challenges.

    While much of identity management is the same, it’s also quite different in how the challenges have evolved. Today there are more systems with identities to manage. As the number of devices increase, that will grow exponentially over the years ahead. Plus, with agile development methods and approaches to IT management such as DevOps, technology development is moving much more quickly. Finally, IT has much less centralized control now over users than it did 16 years ago.

    Here’s how the rapid expansion of cloud and devices are creating a greater need for identity governance.

    Identity Management and Cloud Growth

    I thought managing identities was as complex as it was going to get when corporate LANs erupted into intranets and the web. At the time, re-tooling and scaling directories made for LANs/WANs to the web and web-based services was a challenge. But today, the burst of cloud-based systems, both private and public cloud and software as a service (SaaS), coupled with the many existing on-premises and web-based systems, have made identity management efforts more complex than ever.

    According to the most recent Bitglass Cloud Adoption Report, 48 percent of enterprises now use a cloud-based productivity suite, up from 28 percent in 2014. In EMEA, 59 percent are adopting cloud suites, as are 61 percent in APAC, compared to 48 percent in North America.

    There’s no doubt cloud-based services have increased the number and nature of how identities are created, managed, deleted and governed. And the security of these data and resourced likely comes down to how well identities are managed.

    The Long Shadow Within IT

    As I mentioned above, there is a lot less (like it or not) centralized control in enterprise technology today. More users are bringing their own devices, as we know, and more business managers and users are procuring their own cloud services, from storage and infrastructure to cloud platforms and SaaS applications.

    According to an CIO.com article published last year, CIOs vastly underestimate extent of shadow IT, analysis by Cisco Systems found the typical enterprise has up to 22 times more cloud services running than it believes. The CIOs surveyed estimated their enterprises were running about 51 cloud services. Reality? There were 730 cloud services running.

    I’ve always argued that Shadow IT should be embraced, not fought. It is a way to see what users need and the services they want to use, and it’s a way to engage with users. That’s not to say IT needs to—or should—let Shadow IT be a free-for-all. It shouldn’t. IT needs to manage the types of devices users can select, the services they can use and the data that can be used in those services, not to mention the regulatory and policy compliance checks. None of that goes away.

    But identity and access is a great way for CIOs and IT to help users safely embrace Shadow IT.

    The Challenge of Managing Device Identity

    It’s not just human identities and applications and services. There are more and more IoT devices whose “identity” and set of privileges must be managed as part of enterprise systems management, data, privacy, security and availability. Gartner expects there to be 21 billion connected Internet of Things (IoT) devices by 2020.

    Again, identity is central to managing access and the privileges IoT devices have, as well as auditing use and activity.

    Gartner agrees: “IAM leaders must reconsider how traditional approaches to cybersecurity and IAM work in a world where devices and services are so abundant, in so many different forms and positioned at so many different points within the IT ecosystem,” said Earl Perkins, research vice president at Gartner.

    Based on conversations at the show, it’s also top of mind with most CIOs.

    The Scale of Cloud Services Challenges Identity

    The dramatic growth of apps and cloud services are taking its toll. Years ago, securing apps meant throwing up some firewalls, setting up a virtual private network and providing access control to applications and networked resources. But with the explosion of apps, cloud services, devices and data, such approaches are antiquated. We didn’t know it then, but that was about as simple as security was going to get.

    All of this requires incredibly well-scaled identity management efforts. In addition to the traditional seasonal or event-driven surges in identity management issues, companies could see surges from devices or users accessing systems at all times and from anywhere. Scaling with this requires identity management and directories eventually to be in the cloud. While most enterprises are hybrid now—unless they formed in the past 10 years—there is plenty of legacy, on-premises and cloud needing management.

    However, in a decade, there will be many more large enterprises fully in the cloud, and identity management technology will follow.

  • Best Practices for Cloud Deployment

    Best Practices for Cloud Deployment

    When it comes to deploying apps in the cloud, the conventional wisdom is that DevOps is not required if you’re a small or midsized company. The cloud is designed to enable virtually anyone to set up the infrastructure they need, quickly and easily, without a lot of experience in the intricacies of managing data center operations or hardware. But then, by the time the company realizes it needs DevOps, it’s almost too late—the cloud infrastructure has become complex, costs have exceeded expectations and precious time is being wasted.

    To ensure you’re able to optimize cloud, or hybrid cloud, deployments—and avoid many of the problems that other companies have experienced—there are a number of best practices you must employ within your company.

    Best Practice No. 1: Realize You Need DevOps from the Start

    When you manage your own hardware, you know exactly what the deployment costs are and what resources are available, and have a clear understanding of how they are interrelated. The abstraction of the cloud makes the initial setup a snap. But typically there is no formal structure involved: Individual developers can initiate new instances whenever they need them, without considering the bigger picture of the resources already in use, how they’re used or what is being planned by the organization.

    Spinning up unnecessary cloud resources not only costs a company more in the long run, it also leads to a cloud sprawl that requires time-consuming, manual processes to maintain. Think about the challenges your company would face if it had thousands of machines created by different processes but you didn’t know who created them, what they were being used for or  even if they were still needed.

    By employing DevOps from the start, you will be able to balance the needs of developers with the availability of resources in the cloud to ensure expenses do not run out of control and you understand exactly how the cloud infrastructure is being used now and into the future.

    Best Practice No. 2: Have a Process and Purpose

    As you create your cloud infrastructure, you need to have a clearly defined purpose, as well as a process in place for deployment and ongoing management. You will need to keep track of what resources are being used and their designated purposes. Some cloud vendors, such as AWS, offer tagging capabilities that help you keep track of what services each machine was created for and whether they are being used for production, backup, load balancing or testing.

    Regardless of whether you use tagging capabilities or maintain a separate database of cloud resources, knowing what you have purchased and how it’s being used will provide needed visibility, ease of management and greater control of costs.

    Best Practice No. 3: Leverage Automation

    Automation is important—both in the cloud and on premises—to streamline routine practices, make sure they’re applied consistently and eliminate potential errors that are common with manual processes.

    When deploying on multiple clouds or in a hybrid environment, it is important to understand that tags used for one cloud provider won’t work on another. The process is typically broken into three steps: inventory creation, system preparation and automatic deployment. Many cloud systems allow you to create a dynamic inventory directly from the servers, eliminating the need to create or maintain it manually. However, deployment over multiple clouds require an abstraction of this inventory; it is common to create a static inventory in a unified format from the various dynamic inventories.

    Best Practice No. 4: Design the Network with Security in Mind

    Poorly designed networks are almost impossible to secure. Problems often arise when companies allow their networks to grow organically rather than being planned out in advance. A number of companies have employed the best practice of segregating the services network and the management network. The services network includes all the components required to provide services to your customers and users, while the management network is comprised of what you need internally to manage the network. Separating these networks provides greater assurance that unauthorized users cannot gain access to your systems via the Internet.

    Another part of security is key management. Depending on the cloud provider, keys can be widely available or have a root password that is open to the world. In fact, many hackers routinely scan Github for keys developers accidentally check in. With having to manage so many keys, you may struggle to ensure that they’re not exposed. DevOps plays an important role in keeping the keys locked away, then providing them to a select group of individuals who can use them only in production.

    Best Practice No. 5: Be Strategic About Monitoring

    When you’re deploying hardware on-premises, it’s a common practice to define in advance what to monitor. But monitoring in the cloud is quite different, with basic defaults that produce vast quantities of data—much of which may not be relevant to your organization.

    Taking a cue from how you handle monitoring on-prem, you’ll want to design cloud monitoring to fit your company’s unique needs and filter out the information that isn’t pertinent to your network or operations.

    Best Practices Go a Long Way

    Keeping in mind these best practices and employing a DevOps model from the start will enable your company to get the most benefits from its cloud deployment regardless of its size, and avoid the costly and time-consuming challenges that arise without proper advanced planning and a strong focus on network design and processes.

    About the Author/Roger Fulton

    roger-headshot

    Roger Fulton is Iron.io’s director of Infrastructure and Operations. His background includes broad experience in software architecture and operational planning. He manages international teams to deliver highly available products for end users and highly performing in-house infrastructure. Prior to joining Iron.io, Roger served as head of application and infrastructure engineering at Kaybus. He has also held technical leadership roles at JRV Consulting Group, Cybrata, Invisible IT and Clarus Systems. Roger holds a Masters from the Queens University of Belfast, having graduated with highest honors.