devopsy.kinsta.cloud

Category: Blogs

  • How Service Mesh Enables a Zero-Trust Network

    How Service Mesh Enables a Zero-Trust Network

    How can we achieve high-grade security within a microservices ecosystem? Service mesh may be the answer

    Enterprise applications often assign an identity to humans to determine security clearances. Using standards such as OpenID Connect and SAML, we can delegate authorization scopes across a suite of applications to determine precisely what a user is permitted to do. But what about the applications themselves? Some believe that microservices should follow this identity-first approach, too.

    For large-scale application ecosystems, microservices is arguably more secure than a monolith. As Marco Palladino, CTO of Kong, described, “A monolithic application is a security incident waiting to happen.” Yet, microservices shouldn’t necessarily trust one another. So, how can we achieve high-grade security within a microservices ecosystem?

    The answer may lie in service mesh. I recently met with Palladino to discover how a service mesh can achieve bank-grade security for established and emerging microservices patterns. Similar to how enterprises assign an identity to end users, a zero-trust security network gives an identity to microservices themselves.

    A Zero Trust Network Trusts Nothing

    With more personal data used in cloud operations, more and more personally identifiable information (PII) is vulnerable to attack, and the costs are high. In Q1 2019, the cost of breaches in the financial sector was $6.2 billion. In 2018, 97% of breaches exposed PII, a top target for cybercriminals.

    Organizations are thus feeling pressure to secure their assets. The most precious data is held within financial services and health care, where GDPR and HIPPA enforce strict accordance with data privacy. This pressure is leading U.S. banks to clamp down third-party application access. Simultaneously, European banks face similar security dilemmas while appeasing PSD2 regulation for open banking.

    To safeguard user data, organizations are adopting a zero-trust security model. “Zero trust security means that we’re not trusting anybody,” said Palladino. “We don’t trust our own services. We don’t trust our own team members.”

    Placing too much trust in users, services and teams could cause a catastrophic failure. And, “there is no bigger risk than thinking you are secure, while in reality, you are not,” he said.

    Monolith ⮕ Microservices

    In practice, architecting a zero-trust application network runs parallel with the trend toward microservices designs. Though it may seem counterintuitive, a microservices system is arguably more secure than a monolithic architecture.

    In a monolithic application, every resource—whether invoices, billing orders or transaction history—is always just one line of code away. Since everything is connected, you could expose a resource with a single access.

    A microservices architecture decouples applications into separate services, each responsible for specific resources that make up the smaller applications, which can be secured and provisioned over a network.

    Segmenting the service landscape avoids one cascading defect affecting all applications. It also enables more control. “By decoupling from monolith to microservices, we can apply fine-grained security permissions and rules to determine what’s possible in the context of the application,” Palladino said.

    Microservices ⮕ Service Mesh

    Implementing these permissions is where a service mesh comes in. The application teams often build security, yet it’s generally bad practice to build your own cybersecurity. Security for microservices requires high expertise, and standardizing connectivity between various microservices can easily result in fragmented security implementations.

    Instead of building your own security infrastructure, Palladino recommended utilizing service mesh. By using service mesh as a control plane for microservices, platform architects can specify specific rules and attributes to generate an identity on a per-service basis.

    Service mesh also removes the burden of networking from developers, enabling them to focus more on their core logic. “Application teams become consumers of connectivity, as opposed to the makers to this connectivity,” Palladino noted.

    Identity for Services

    In the same way enterprises create identity profiles for humans, Palladino said we must extend the same model to services, pairing each service with authorization scopes. “Services must provide their own identity,” he noted. “Based on that, they can do certain operations in their infrastructure.”

    When we think of identity for services, attributes such as geographic location and certificates come to mind. These and other traits could make up a microservice’s authorization scopes. Utilizing a service mesh, architects could assign such attributes to each workload and service to determine what operations are allowed or what requests can be made to what other services.

    For example, services from one region could be designed not to consume services from another region. Such attribute-based control for microservices is a novel, well-needed strategy, Palladino said. “It was crazy this was not in place before; not having zero trust is like having all your money in an open vault.”

    Best Practices for Service Identity

    Knowing that we want to implement identity-based security for microservices, where are some best practices? Palladino recommended:

    • Automate some attributes: A service mesh could automatically assign identity attributes upon creation, such as cloud zone or geographic boundaries.
    • Provide a way to manage attributes: A dashboard would be necessary for DevOps engineers to create, edit or delete service-level attributes.
    • Governance: Architects must manage the control plane and create identities for the right parties to access this capability.
    • Secure encryption and Mutual TLS: Teams will require a system for rotating and expiring certificates for hundreds of thousands of workloads running across an organization, noted Palladino.
    • Standardize: Secure Production Identity Framework For Everyone (SPIFFE), a CNCF incubating project, is a good framework for service identity, making it operable with other cloud-native projects.
    • Play nice with CI/CD: Attribute creation could be hooked into an underlying CI/CD process to occur when deploying products automatically.

    Once microservices have identities, the services they interact with can determine what actions they are authorized to perform. This limits machine-to-machine communication to allowed operations.

    Zero-Trust Security With One Click

    Many teams have transitioned to a microservices architecture to deploy more often, be more distributed, decoupled or improve overall reliability. While these innovations are largely positive, Palladino cautioned against innovation without the proper security forethought: “It’s silly to transition to a microservices strategy without thinking about how these services are secured,” he said.

    Teams will also be running microservices on various environments—VMs, cloud or physical data centers. Thus, considering microservice-to-microservice communication, how do we route traffic between such environments? How do we observe metrics and logs? Palladino believes an agnostic service mesh that supports multi-region, multi-zone, multi-cloud and hybrid modes is the answer to these dilemmas.

    If used correctly, service mesh can plug security gaps for internal service-to-service communication, thus obfuscating high-value resources, such as financial services data, from nefarious players. The key to solving the service identity challenge is a usable dashboard. As Palladino described, “Zero trust security with one click.”

  • Progress Reorganizes Chef Portfolio

    Progress Reorganizes Chef Portfolio

    Progress today announced updates to the portfolio of DevOps offerings it gained with the acquisition of Chef earlier this month.

    A Chef Infrastructure Management offering now combines Chef Infra and Chef Automate into a single offering, while Chef App Delivery combines Chef Habitat, Chef Infra and Chef Automate into a single offering that creates an artifact that can run anywhere.

    Sundar Subramanian, senior vice president and general manager for core products at Progress, said the goal is to organize the Chef portfolio around a set of distinct offerings for automating the management of infrastructure and continuous delivery (CD). Instead of simply offering a set of tools for power uses, Subramanian said Progress is moving toward creating a set of platforms.

    At the same time, Progress announced a dedicated content delivery service that provides access to a library of curated Chef Premium Content for compliance scanning and remediation that complies with the best practices defined by the Center for Internet Security (CIS).

    Finally, Chef has added a Learn Chef Skills Validation Program based on the online Learn Chef platform and the OpenBadges Standard, a specification designed to promote certifications that teach skills in a way that are both verifiable and portable.

    CD and infrastructure management are typically handled by different members of the same DevOps team. By segmenting its offering around federated responsibilities, the Chef software portfolio will become easier for DevOps teams to consume based on specific tasks, Subramanian said.

    Progress expects its offerings in the months ahead to have greater appeal among midmarket IT organizations that are accelerating multiple digital business transformation initiatives in the wake of the COVID-19 pandemic, he noted. The company has been focused on providing application development tools to these organizations for decades, and as their IT operations mature, many are beginning to embrace best DevOps practices as they seek to reduce the total cost of IT operations.

    The challenge those organizations face today is they depend heavily on ITIL-based frameworks for managing IT. Beyond acquiring DevOps tools and platforms, most of those organizations are at the beginning a what has been shown to be a transformation to DevOps processes that requires major changes to the culture of an IT organization. The cultural transition often involves years of effort as organizations try to balance different approaches to managing IT based on ITIL and DevOps processes.

    It’s too early to say what path to automation of IT those organizations will take. The Chef tools still require a level of programming expertise to employ that is often lacking among traditional IT operations staff. Not to mention the fact that Progress is not the only DevOps platform provider looking to expand its reach into the midmarket.

    Regardless of the path chosen, the number of organizations that embrace DevOps principles is about to expand significantly.

  • How to Build a Highly Qualified Cloud-Native Team

    How to Build a Highly Qualified Cloud-Native Team

    If you’re reading this, you more than likely are aware of the struggles involved in recruiting and building a team of technical professionals skilled in cloud-native computing technologies such as Kubernetes, Helm, Prometheus and service mesh. The Linux Foundation and edX’s “2020 Open Source Jobs Report” found 93% of hiring managers are having difficulties filling open positions that require open source skills like these. The report also found that cloud and container technologies are the most in-demand aside from Linux, which itself is a necessary basic skill for cloud professionals.

    So the question is, between a talent shortage and rapidly increasing adoption of these technologies, how can any leader manage to build out a qualified team? Based on our experiences as a vendor-neutral provider of training and certification for many of these technologies, we’ve learned some tips and tricks that, with some effort, can help you establish a team that can handle any cloud-native project you throw at them.

    Don’t Rely on the Same Old Recruiting Strategies

    It’s important to understand that the most promising cloud developers, engineers, administrators and other team members may not have the most prestigious qualifications. You may often find a high school dropout who has completed a cloud computing bootcamp and passed a Kubernetes certification exam will be far more able to perform the functions of a job than a recent graduate with a computer science degree. This means you need to set the job qualifications appropriately.

    If your organization is going to limit these hires to those holding a college degree, you are reducing an already limited talent pool even further and you’re going to miss out on some incredibly talented individuals. Instead, look at demonstrable skills and experience rather than only education. Some of the most talented contributors to these open source projects themselves don’t hold degrees; if the people building these technologies don’t need one, then neither do the ones implementing and administering them.

    Of course, you should continue to look at candidates with computer science degrees—after all, you’re trying to expand your access to talent, not reshuffle your sources. College degrees are a valuable indicator of intelligence and tenacity in completing a rigorous program. But do not take the degree as evidence they can perform any cloud-native tasks. Look at their specific skills, what the university curriculum was, any external training or self-study they conducted and verifiable credentials.

    Structure job postings as a list of skills and abilities necessary to be successful in the role rather than specific educational requirements or years of experience—in fact, many of these technologies are so new that no one has more than a year or two of experience with them. Avoid absolutes where possible and encourage folks to apply even if they don’t check every single box. And if you do want to ensure that someone is up to the challenge, you can require they hold verifiable certifications. The trick here is to find the certification most respected in the industry for a particular skill set, and prioritize performance-based certifications that test abilities to knowledge-based ones that only test memorization.

    Go Where the Developers Live

    Do not expect a wave of great applications just because you throw a well-crafted job posting on the internet. Many of the most talented cloud-native pros out there don’t bother looking for opportunities as they are already constantly bombarded with them. You need to find them where they are spending their time, which is likely message boards and forums such as StackOverflow and Reddit. LinkedIn can sometimes be effective, but experience shows this type of expert is less likely to spend time there than individuals working in many other professions.

    And do not simply bombard these professionals with job opportunities, either. You need to make your organization stand out and demonstrate why they should want to come work for you. The Open Source Jobs Report asked opens source pros what would make them consider moving jobs—while to no one’s surprise salary is a top incentive, 65% stated the opportunity to work on interesting projects would tempt them, while 58% desire a better work-life balance and 51% want more flexible schedules or telecommuting options.

    Talk these up in your outreach, but you should also put your money where your mouth is. Sixty-three percent of the hiring managers surveyed this year said their company has decided to contribute to an open source project, either financially or by contributing actual code, for the express purpose of attracting developers who work on that project. If your organization is using open source technologies but not giving back to the community, you will miss a significant opportunity to raise your profile among target recruits.

    Take a Chance and Help Them Grow

    Let’s say your company is participating openly in open source, being a good community member, offering competitive benefits, being open to considering individuals from a broad range of backgrounds and advertising your open roles in ways that both get in front of and attract talented cloud-native professionals. Despite all this, you still have not been able to recruit enough talent to tackle the needs of your organization.

    The first recommendation when you’re at this stage is to look at your existing resources: Are there talented individuals in your organization who do not possess the skills necessary to perform these tasks but possibly could with the proper training? If so, give them the chance to grow into such a role. You can start off with very little risk involved; ask them to complete a free introductory training course and report back after. Consider how long it took them to complete the course and ask them how they feel about this type of work. It should be apparent quickly who is enthusiastic and excited about learning a new technology and who is not. You can then sponsor those who show promise for more advanced training courses and certifications to demonstrate they have gained the necessary skills to move positions.

    If you don’t have anyone internally who either can or wants to train for a cloud-native role, look at recruiting someone with promise and make clear from the start that the expectation is they will complete training and gain an appropriate certification within a specified time period. Look for individuals who have already demonstrated both grit and an ability to learn. Perhaps they have an entry-level IT certification that is far from meeting your requirements but still shows that they can complete this kind of program. You’ll find many candidates are strongly enticed by such an offer.

    This is reinforced by the data from the Open Source Jobs Report: 54% of developers surveyed stated that lack of employer-sponsored training is a major hindrance to their job and 70% stated that increased training would make them more effective. 70% of hiring managers also stated that they’ve seen more requests for training from employees the past year.

    Summary

    We know it’s a tough job finding qualified cloud-native talent to fill open roles, and it is not expected to get easier anytime soon. By putting in a concerted effort, being thoughtful and flexible about the types of applicants you consider and giving existing and potential employees the opportunity to grow into these roles, you and your team can be successful.

    This article is part of a series of articles from sponsors of KubeCon + CloudNativeCon 2020 North America

  • From ‘Cloud Zombies’ to Multi-Cloud Experts

    From ‘Cloud Zombies’ to Multi-Cloud Experts

    A significant risk associated with cloud migration is overspending. For years, zombie-like obedience to AWS has resulted in price lock and escalating fees at many organizations – this is a tale for fighting the cloud zombies.

    Cloud costs around data egress or application bandwidth are often not transparent and thus highly unpredictable. Complicating this problem is the need to rapidly develop and scale large application ecosystems, leading to snap judgments by project engineers to buy tools. Or, cloud purchasing decisions fall to the CFO, who may not be qualified to understand technical nuances. Without cloud spending forethought, it’s easy to enter unsavory pricing commitments.

    However, all of this is changing. As cloud service providers (CSPs) mature their offerings, competitors have risen to take on AWS, unlocking more consumer choice throughout the cloud landscape. In a multi-cloud world, cloud economists are leveraging lift and shift capabilities to optimize performances. Ironically, companies are also seeing cost benefits in retaining some on-premises servers.

    I recently spoke with Scott Leatherman, chief transformation officer at Virtana, to get his take on preventing cloud cost escalation and how IT should streamline multi-cloud environments to stem the cloud budget.

    Zombies in the Cloud

    When the cloud arrived, it promised high performance at a low cost. Companies were happy to offload on-premises servers for the ease of cloud arrangements. For years, there was a “zombie-like loyalty” to cloud platforms, said Leatherman.

    Not much was thought about costs rising in parallel with the astronomical growth of the web, mobile and IoT. Executives penned long-term pricing contracts without a second thought. Then eventually, reality hit. And it came with a hefty price tag.

    Spooky Cloud Expenses

    As cloud-enabled businesses took off, they encountered many hidden fees. To name a few:

    • Egress: Fees for moving data in and out of the cloud.
    • Application: Fees for moving applications to new instances.
    • Workloads: Mounting costs for increasing bandwidth and workloads.

    These and other add-ons could quickly raise a cloud bill from $200 to $200,0000 a month. Complicating this was the fact that many large organizations purchased multi-year cloud subscriptions locked at a specific price.

    Counterintuitively, such cost-savings guarantees are “completely backward,” Leatherman said. As computing power increases exponentially per Moore’s Law, servers become cheaper to maintain.

    “You’ve literally committed yourself to something that is depreciating in cost,” he said. Yet, those savings don’t trickle down to consumers and SLAs have remained pretty flat. “What’s the most frustrating thing for me is that the best way to run this business is to lock people in.”

    The Zombies Become Aware

    In the wake of COVID-spurred digital transformation, cloud zombies are waking up to slash the budget, finding cost-effectiveness in multi-cloud strategies. Some companies are even reaffirming ties with on-premises servers.

    If performance isn’t worth the cost, it defeats the benefits of moving to the cloud entirely. So, how can we smartly optimize our environments for quality performance with leaner financial margins? Leatherman shared some steps to become cloud-smart:

    1. Baseline Your Footprint

    The first step toward reducing cloud costs is maintaining a baseline of all cloud dependencies and private data centers. This should cover a deep cost and performance analysis.

    “You need a baseline in your data center—what can I move? what should I move?” said Leatherman. Unless you have a baseline of what your data center is doing (both private and public), you can never understand what resources should move to the cloud. Powerful apps for monitoring the public cloud could help audit architecture.

    2. Optimize, Optimize, Optimize

    Only after mapping topologies can you can begin to optimize performance. Since usage differs depending on the season, cloud optimization should respond to yearlong application usage trends. For example, ERP software may pull significantly more requests at the end of the quarter and e-commerce spikes during the holidays.

    Organizations should ensure the elastic cloud truly scales to application needs, but only when needed. Ensure systems of orchestration and routing function properly, and diligently search for hidden fees. “The cost that sneaks up on you in the elastic cloud always comes up,” said Leatherman.

    Heavy computation processes, such as machine learning over big data, will require enormous server consumption. For resource-intensive scenarios, Leatherman recommended keeping code lean. Whereas CPU relies on the laws of physics, “application developers are only limited by creativity,” he said.

    Application developers should whenever possible optimize code to avoid redundancies. Doing so yields a positive environmental impact, too—eliminating operational inefficiencies can decrease cloud server usage, which reduces carbon footprint.

    3. Lift and Shift

    GCP, Azure, Oracle and other CSPs mature, the zombie-like loyalty to AWS is fading. “Now that there is parity across cloud vendors, things are changing,” he noted. “The flexibility in the marketplace is really dynamic.”

    In a multi-cloud world, organizations can leverage a hybrid cloud and “lift and shift” toward the best cost optimizations for their specific needs. This is where maintaining a strong baseline will help continually test cost efficiencies among cloud competitors.

    4. Refine Your Tool Procurement Process

    Another solution involves scrutinizing your cloud procurement process and the personnel involved. As mentioned above, new cloud tooling purchasing may occur at the DevOps level or large contracts may be escalated to the CFO office.

    Unfortunately, neither side is in an ideal situation to make smart financial decisions—one faces an urgency to deliver and the other can’t see the trees from the forest (and especially not the termites eating the valuable wood). Many believe a new role is required—The Other CFO—to help optimize the cloud stack.

    5. Retain and Optimize Private Data Centers

    By now, most enterprises are accustomed to having both a data center and a cloud center team. Leatherman argued that ultimately these two teams must work together to avoid outages and reduce the exponential costs of moving to the cloud.

    From a security perspective, there are still significant advantages to maintaining on-premises private clouds and data centers. By leveraging private assets, you aid data protection efforts and have increased control. “Datacenters have value,” he said. “Unless you started there, no one will be 100% cloud.”

    Although cloud migration seems inevitable, through software automation and Moore’s law there is value in on-premises. As a result, Leatherman predicts a rise in private cloud data center optimization and management services by third parties.

    6. AI for Cloud Cost Awareness: DevFinOps

    Finally, AI soon may be available to automatically scan for cloud cost impacts before applications are even deployed.

    Most DevOps teams use container security scanning or automated compliance validation as part of their DevSecOps process. Leatherman suggests future DevOps tooling will similarly evolve to display automated notices that estimate cloud costs for your actions in real-time.

    For example, as engineers are pushing code, a popup could display server utilizations and a cost impact breakdown. Or, inline code validation suggestions could find redundancy to decrease waste. Perhaps we’ll need a new abbreviation for this Developer + Finance + Operation focus—DevFinOps, anyone?

    Such an AI would likely not negate the need for a cloud economist to help manage multi-cloud complexity. “We’ll still need them,” said Leatherman.

    Surprises in a Multi-Cloud World

    COVID–19 is accelerating deployments tagged for much shorter timeframes, forcing many more instances to public clouds. Businesses need to get performance up and meet their SLAs. To reduce expenditures, they must quickly lift and shift to the most performant and cost-efficient environments.

    The gravity is apparent, yet hasty moves now could be disastrous for security and data privacy in the future. Besides rising cloud costs, there are other risks associated with the cloud that increase exponentially with multi-cloud environments. “It’s not just the financial spend, but the security and risk that goes along with it,” Leatherman said. “It becomes really scary how many people have access.”

    Not only must cloud spending decisions shift left, but security must arise earlier in the development cycle as well. Compliance with regulations such as GDPR or HIPPA should also influence how teams manage data between clouds.

    The Cloud is Relatively New

    An IBM-endorsed study found enterprises are only 20% of the way through their cloud journey. Many enterprises are still just at the beginning of their cloud transformations, and 90% plan to expand their cloud footprint in the next year.

    The cloud journey is quite long, and many have quite a way to go. Knowing this, cloud economics will only rise in importance. As I recently covered, cloud expertise is now superior to a university degree.

    Current forces, like the COVID–19 crisis, will no doubt continue to “drive people to the cloud,” said Leatherman. While responding to the imperative to deliver more and higher speeds, it’s essential to be smart, leverage existing assets, and optimize cloud usage at every level possible. “It shouldn’t just be cloud-first,” he said. “It should be cloud-smart.”

  • Survey Sees Surge in Demand for DevOps Skills

    Survey Sees Surge in Demand for DevOps Skills

    The Linux Foundation and edX, a provider of an online learning platform, jointly released an annual report today that for the first time finds organizations are looking for more IT personnel with open source DevOps skills (65%) than developers (59%).

    Clyde Seepersad, senior vice president and general manager for training and certification at The Linux Foundation, said the 2020 Open Source Jobs Report shows that as more workloads shift to the cloud in the wake of the COVID-19 pandemic, the number of organizations looking to embrace best DevOps practices is increasing substantially. The report finds 70% of respondents are more likely to hire an IT professional with cloud computing skills.

    Based on a survey of 175 hiring managers and 900 IT professionals with open source expertise, the report also finds that despite the economic downturn brought on by the pandemic, 37% of hiring managers say they will be looking to hire more skilled IT professionals in the next six months.

    The report also finds 93% of survey respondents report difficulty finding open source talent. An overwhelming 81% of hiring managers said recruiting open source talent is a priority for 2020, with 56% looking to recruit IT professionals with open source skills in the next six months. Nearly two-thirds (63%) said their organizations have begun to support open source projects with code or other resources for the explicit purpose of recruiting individuals with open source software skills.

    More than half of hiring managers (52%) said they are more likely to hire someone with a certification, while 80% of survey respondents reported providing online training courses for employees to learn open source software. Nearly three-quarters of organizations (74%) are now offering to pay for employee certifications, according to the survey results.

    An equal percentage of hiring managers also identified Linux as the most in-demand skill they seek in new hires, while 63% also said they are looking for IT architects who can build solutions based on open source software.

    Seepersad said given the difficulty attracting talent many organizations are focusing more on training their existing staff rather than trying to recruit talent in an open market where demand for open source expertise remains high.

    The talent pool is more global as more organizations become accustomed to working with staff who are now largely working from home, which, coupled with current economic realities, is limiting how much organizations are willing to pay for IT talent. More organizations are also keen to make sure there is a cultural fit with their organization when hiring, he added.

    In general, it’s clear more IT organizations will need to focus on growing their own farm teams with open source DevOps skills rather than hoping to lure away superstars from other organizations. Rather than simply trying to fill open positions, organizations need to think more in terms of acquiring and managing talent in the same way a general manager manages a sports franchise.

  • How To Foster a Culture of Citizen Developers

    How To Foster a Culture of Citizen Developers

    What can leaders do to encourage a thriving environment of citizen developers in their organization?

    Over the last several months, businesses of all sizes have had to embrace rapid change, as have their workforces. When it comes to IT departments, even before the pandemic many were fielding requests to implement new technology, customize solutions to niche problems and automate specific tasks at rapid rates. Unfortunately, these increased demands coincide with an ongoing developer drought, and paired with the pandemic, have put an added strain on some already thinly stretched IT teams.

    As a result, there has been an observable rise in citizen developers being tapped to relieve some of this burden. “Citizen developer” here is used to describe any programming novice or non-developer who has been tasked with the responsibility of constructing custom applications. This person has some familiarity with programming but may not have been formally trained and, therefore, may use their own set of tools and workarounds to complete the work. If leaders don’t make an effort to create standardization, the consequence can be high variability in quality, efficiency and output.

    And it doesn’t seem to be a passing phase. I recently spoke with Dave Garrett, chief strategy and growth officer at Project Management Institute. As we look to the future, Garrett believes that citizen developers will play a critical role in helping organizations stay agile and ensuring projects move forward without delay.

    Though it won’t solve all of your pressing technical concerns, the good news is that with smart strategies, a citizen development emphasis can ease the burden on IT teams and improve output. Garrett spoke to a sea change of new techniques being developed to respond to these growing needs.

    4 Elements of Strong Citizen Developers Culture

    1. Standard Toolsets

    To start, build a solid foundation by choosing standard toolsets. Be sure all development staff is made aware of the toolsets you’ve deemed standard, clarify that adopting them is a priority and ensure that all have access. Standard approaches to using these tools should also be established and communicated, such as programming languages and stylistic standards.

    “If you have a standard toolset and approach, then you have many people who can help,” Garrett said, emphasizing that by doing so, you are building a broader base of support. Having a larger pool of employees to pull from helps an organization stay on top of the necessary tasks and creates more freedom for open collaboration. As a bonus, professional developers are freed to focus on more advanced tasks, such as machine learning and or mathematically complex programming.

    In deciding which toolsets and approaches to make standard, Garrett said, ask yourself, What will get them from A to B most quickly and efficiently? One approach to this is low-code or no-code tools. He considers such platforms transformational in empowering citizen developers to express ideas tangibly without a great deal of coding.

    2. Technical Training

    From here, ensure that all development staff, especially your citizen developers, have been technically trained in adopting the standard toolsets and approaches that have now been established. This may include offering training in low-code UIs and instilling fundamental programming concepts such as boolean logic and conditional programming such as If/Then/Else statements.

    “Set a foundation of success for learners,” he said. For example, have them build something of value during training, making it as actionable as possible for the use cases they will be handling.

    3. Discouraging Use of Shadow IT

    Once you have established toolsets and training, leadership should do whatever’s possible to enforce the established standard’s adoption and discourage the use of shadow IT—tools, applications and workarounds not approved by leadership within the company. Shadow IT, as a practice, can be messy, creating high variability and making quality-control and collaboration difficult. “You want to avoid it,” Garrett said.

    4. Good Mentorship and Governance

    Another benefit of having a standard toolset and eliminating shadow IT use is that it enables more efficient mentorship—critical for inspiring citizen developers. IT teams are advised to identify a mentor or group of mentors proficient with the standard tools to offer ongoing guidance to the team’s citizen developers.

    Quality mentorship goes hand in hand with quality governance. “Governance” here is used to mean technical admins who enforce the use of technical standards, tools and programming styles. No-code tools require access control that reflects a citizen developer’s permissions.

    Building in Hyper Agility + The Two-Pizza Rule 🍕🍕

    One way to think, Garrett said, is hyper agile. This can apply to team size, smart PM cycle lengths and brainstorming techniques.

    Team Size

    In thinking of ideal team size, ask: How can the team be optimized to work quickly and efficiently? One suggestion Garrett had was to ensure your team isn’t too large to effectively train, mentor and govern. For example, as a rule of thumb, he mentioned the old “two pizzas” rule: A team size shouldn’t require more than two pizzas to share a meal.

    Sigh. Remember things like in-person training? Pizza parties? Obviously, old meetups can’t be executed exactly to plan these days, except with some clever contactless delivery tactics, but you get the idea. A strategically sized team will create an efficient workflow and collaboration in your team dynamics.

    Cycle Length

    As projects come in, an agile cycle length, or sprint, for completing a specific set of work should be determined. What is an efficient amount of time to expect a team to turn out a project? To determine this, sprints should start with a planning meeting during which the product owner defines the work criteria. The development team determines a realistic time frame to complete the work scope. The scrum master or team manager typically makes this call.

    The lengths of agile sprints can vary—some may be two weeks, while others may be 30 days—depending on the scope of work. Most project management experts recommend you aim to keep your cycle length consistent, so teams get used to working within a set amount of time.

    Brainstorming

    Bringing attention to how your team ideates together is an excellent way to build agile collaboration. Brainstorming as a development team fosters collaboration and allows you to aggregate the talent and pull ideas together for the best output, whether you’re trying to solve a niche problem or simply join forces around how best to complete the sprint.

    Garrett recommended bringing people together in smaller groups for round-robin brainstorms and creating as you’re ideating. “Thus, iterations happen fast,” and all team members are given a chance to express their ideas in a tangible way.

    Once citizen developers see success, you can then use the excitement of building applications and solutions together to drive things forward, he said. Your anchor is a cool product that gets people excited—great for both morality and high-quality output.

    When Are Citizen Developers Appropriate?

    The question should be asked: When is it appropriate to call on citizen developers? Garrett recommended performing a suitability assessment to identify the best use cases for citizen developers. This will help the organization understand when it’s smartest to bring them in.

    The easiest way to answer this may be to consider: When isn’t it appropriate? Assigning advanced processes to citizen developers such as algorithm-heavy, mathematically complex programming or core product functionality is usually inappropriate, he said. Instead, utilize citizen developer talent for constructing internal business utilities that can be automated easily.

    There are formal suitability assessment tools that look at scores and attributes for applications to help you make this call. “However, everyone’s different,” Garrett noted. “You have your own ways of working, given context, goals and what you’re trying to accomplish.” So it may make the most sense to perform your own internal suitability assessment.

    Do the Numbers

    According to a recent PMI report, over the past five years half of organizations have recognized team members who brought innovative ideas forward, but:

    • Only 25% required dedicated time for innovative thinking and idea generation.
    • Only 19% hosted a creativity lab or hackathon facilitated by a professional.

    By contrast, their research shows that those promoting innovative practices see stronger results. Case in point:

    • 80% of organizations that encourage project leaders to expand their role have invested in at least one innovative idea in the past five years.

    The trend PMI sees on this front is that organizations want to innovate, but don’t always know how to make it happen. So, how might they push innovation forward? If you can implement a strong citizen development toolset, you can “enable innovation to bubble up,” Garrett said. The hope is this guide offers a blueprint for fostering just that.

  • Winning DevOps in the Finance Industry

    Winning DevOps in the Finance Industry

    DevOps is making its impact on almost every software-powered company these days. Companies in the finance domain, however, have been perhaps the slowest to embrace DevOps because of concerns regarding governance, security, compliance and regulatory regulations. But it is safe to say that they are currently leading the digital innovation in this era of cloud-native technology.

    Why DevOps for the Finance Industry?

    Higher customer engagement and continuous transactions make the finance industry one of the busiest and naturally requires uninterrupted infrastructure. DevOps implementation helps companies develop fast, fail fast and learn fast so they can fulfill customer expectations and deliver features faster to market than their competitors. Technology in the finance sector has not evolved much, as many companies have assumed it is too risky to embrace DevOps because of the amount of legacy code and old methodologies still in practice. As a result, digital innovation has lagged.

    Challenges of Implementing DevOps in Finance

    Unlike other industries, regulated industries imply several challenges:

    • Strong restrictions on secured networks.
    • Fine-grained audit trails.
    • Strong ACLs models.
    • Full lifecycle governance.
    • Integration with third parties.

    Security and compliance are top concerns in financial companies, and DevOps practices are, in fact, viewed as risk factors to security. Plus, the increased frequency of software releases in DevOps is seen as a threat to governance and regulatory controls.

    Is the Needle Moving Toward DevOps?

    According to the “2020 State of Database DevOps report“ by Redgate, financial services organizations report the highest levels of database DevOps adoption this year, with proportionally higher rates than any other sector. That shows financial sector maturity in terms of digital innovation. 

    Image source credit: Redgate

    Case Studies of Financial Companies and Their DevOps Journey

    Barclays DevOps Adoption

    In 2015, Barclays announced it was adopting DevOps for its digital transformation journey. Barclays processes payments that equate to around 30% of the UK’s gross domestic product. The developers’ morale and quality of code increased because of DevOps and the leadership team at Barclays credited DevOps with significantly reducing the complexity of its codes, which has allowed the company to reduce delivery risk and ultimately, improve the quality of services.

    DevOps the Lunar Way

    Lunar Way’s journey shows you don’t have to be big to use Kubernetes. The company started its cloud-native DevOps journey by splitting its massive monolith application into smaller microservices. To spin up these microservices, Lunar Way used Ansible, Terraform and Jenkins and to deploy these microservices as a whole unit. 

    Then Lunar Way started to experience scaling issues with microservices and didn’t experience any of the microservices benefits. 

    The company started looking for ways to move past this complexity by shifting its focus from machine-oriented to application-oriented architecture. It chose Kubernetes as the abstraction layer along with AWS, not worrying about where the containers are running, and this is how it was able to manage and unlock the velocity of microservices. Lunar Way also chose Kubernetes from a security perspective and to specify how the applications should run. Now the company runs around 80+ microservices in production with the help of Kubernetes. Watch and learn how Lunar Way did it: “Two years in production with Kubernetes.”

    Italy’s Biggest Bank Embraces DevOps

    A conventional bank running its real business on such a young technology? No way, are you kidding me? Nope, I am not kidding. 

    Italy’s banking group, Intesa Sanpaolo, has made this transition. Banks still run their ATM networks on 30-year-old mainframe technology, so embracing the hottest trend and tech is nearly unbelievable. 

    Even though ING, the banking and financial corporation, changed the way the banks were seen by upgrading itself with Kubernetes and DevOps practices very early in the game, there was still a stigma attached to adopting Kubernetes in highly regulated and controlled environments including health care and financial services. The bank’s engineering team came up with an initiative strategy in 2018 to throw away the old way of thinking and started embracing microservices, container architecture and migration from monolithic to multi-tier applications. Today, the bank runs more than 3,000 applications. Of those, more than 120 are now running in production using the new microservices architecture, including two of the 10 most business-critical for the bank. 

    HSBC Going the DevOps Way

    HSBC is changing the way typical banking works with cloud-native technology and DevOps, as an early user of Google’s Cloud Services Platform to provide core banking services to its business customers. 

    HSBC plans to build its all-new business banking service to run on a Kubernetes-managed container infrastructure using Google’s toolset. HSBC also has excellent partnerships with AWS and Microsoft, favoring the multi-cloud strategy. The company’s engineering team believes that moving to a container model under the umbrella of Kubernetes is significant because it means the environment is similar across different clouds. HSBC also hired people from outside of the banking sector because it wanted to change its culture from within. HSBC is said to have invested $10 million so far in its DevOps initiative.

    Capital One’s Agile to DevOps Journey

    Capital One’s Agile journey started in late 2011, with just two teams, which expanded slowly as more teams were trained in Agile development. The developers at Capital One were following the Scaled Agile Framework (SAFe). Initially, since the teams were new to the automation methodologies, integration testing, security testing, unit and performance testing were all done outside of development sprints by separate test teams. Later, once they understood how important it is to have a collaborative culture, they integrated testing into the dedicated DevOps teams and automated it well. Then slowly, they moved all testing into development sprints, adopting a culture of DevOps and uniformity and wiring integration, security and performance testing into a continuous delivery pipeline. As of 2016, the company had more than 700 Agile teams following continuous delivery.

    DevOps at Bank of America

    Bank of America is one of the world’s biggest financial institutions following DevOps practices, and its digital enterprise operations support nearly 4,300 retail financial centers and approximately 16,600 ATMs worldwide. The award-winning digital bank has nearly 38 million active users and approximately 29 million mobile users.

    Automating Security and Compliance

    As DevOps continues to evolve and become mainstream, it will continue to be of interest for many senior managers across financial services. The above case studies show they are already making DevOps a priority. But security and compliance remain barriers slowing the innovation in the financial companies. 

    Organizations should start small and keep security high priority, first using continuous integration and then adding continuous delivery and deployment using the available tools in the market. Over time, the more a company automates, the higher return on investment it can achieve.

    Automating security permissions and controls will help financial companies remove some of their most common software barriers and get their releases out quicker than before, while still maintaining the necessary governance and compliance. 

    Take a Step Forward

    Financial services companies need complex and rigorous software development processes to meet intense corporate and regulatory requirements. Yet, they are expected to release software faster than ever to meet expectations and beat the competition. By starting small and keeping security and compliance in mind, DevOps in finance can be achieved with great success.

  • Bootstrapping DevOps for the Internet Computer

    Bootstrapping DevOps for the Internet Computer

    All startups face unique sets of challenges and issues at each stage of their existence. The DFINITY Foundation, a Swiss-based not-for-profit with the remit to extend the functionality of the internet, is no exception. Like any new organization, we have had to evolve and adapt our DevOps and other practices in response to different obstacles.

    After learning to clearly communicate our vision for an open development platform where software runs directly on the internet, we created a road map to translate this vision into a decentralized architecture with distinct features that could be built and shipped on a periodic cadence. Consisting of five milestones, the road map culminates in a public launch in Q4 of 2020—and we haven’t missed a single deadline despite the disruptions of a global pandemic.

    Incorporating a focus on operational responsibility on top of our focus to ship features without losing speed was a major challenge this year ahead of the release of our developer network, called Tungsten. Sustainably bootstrapping operational responsibility for this release, as well as future releases, was imperative.

    We’ve all heard horror stories about operational teams that run through a checklist in a perfunctory fashion without identifying the root cause of a problem. Even worse, they are quick to give up and are resistant to launching changes, increasing the complexity of their workload.

    Pushing Forward With DevOps

    We were determined that this wasn’t going to happen to DevOps at DFINITY, so we embedded these core cultural touchstones into the organization.

    1: Create SLOs and SLIs for cross-functional clarity

    At the end of 2019, well before there was a service to operate, we started thinking about service level objectives (SLOs) and service level indicators (SLIs). These are critical for providing a common, quantifiable vernacular for R&D and Operations teams to measure how a service should be performing.

    An SLI is a clearly defined indicator for one aspect of a service’s performance, ideally focusing on user-visible service behaviors. The SLO sets the target for the indicator, as well as a window of time over which it is measured.

    To communicate the SLIs and SLOs, training materials were presented across the organization, including speaking with various engineering teams, discussing best practices at management meetings and giving an overview at an organizationwide meeting. In addition, the leadership of R&D teams signed off on the approach.

    To provide clear operational expectations as well as a forcing function, we established an early version of the SLOs as a high-priority OKR.

    As an additional consideration, it is crucial that your SLOs reflect aspects of your service that are visible to the user. As Charity Majors, CTO of Honeycomb.io, noted, “Nines don’t matter if your users aren’t happy.” SLOs are not a one-and-done deal; they need to be updated as the service evolves and as your ability to capture the UX improves.

    Having SLOs in place gave staffers with operational responsibility clear expectations to fulfill. If the service is performing within the SLOs (and if the SLOs are representative of user happiness), then things are running smoothly.

    #2: Engineers developing the software should also help run the software in production

    To bootstrap the team with operational and support capabilities, we needed to ensure that we didn’t create a siloed team lacking engineering skills. Operations in this context is a software engineering problem, which meant we needed to involve developers.

    The same engineers developing the software should have skin in the game in terms of how the software is running in production. If there are problems, these engineers are in the best position to diagnose them and propose and implement the necessary fixes. Getting organizational buy-in for this took some time.

    We emphasized the need for senior engineers to be able to operate the software being built. We didn’t want to fall into the trap of throwing a handful of junior engineers into the deep end without proper supervision or training, so we also established an expectation that all engineers should be able to operate the software irrespective of seniority.

    We built two teams of between five and 10 developers in each of our office locations in Palo Alto, California; San Francisco; and Zurich, which were initially staffed with volunteers across the organization. Each week, one team member at each site managed primary operational responsibility for the service instead of their regular project work, and they knew that the entire staff would help if there were any problems. We plan to rotate new people into these ad hoc teams every six to nine months.

    #3: Create optimally sized, communicative teams

    Operational team size is also important. If the team is too small, people are on-call too frequently. If the team is too large, they will be on-call so infrequently that they will either forget how to operate the production stack or their knowledge will become outdated.

    We share status updates through a daily log and informal shared communication channels such as Slack. There is also a weekly meeting to relay responsibility from one person to another.

    This has helped us identify capability gaps. With team members based across the world, we’ve come to acknowledge that if a problem requires a team’s expertise outside of its working hours, then rolling back first and investigating later is absolutely fine.

    The DevOps team decided early on that we weren’t going to start with 24/7 support, instead providing support during business hours in Europe and the U.S., where the majority of our early-access customers are located.

    We’ve been transparent to the wider organization throughout this process, calling out what is and what is not working well, and how we’re working to improve our processes.

    Conclusion

    We were fortunate that unprecedented global events like COVID-19 didn’t significantly affect our operations. That said, shaping our processes in-person and being physically present for people when they first go on-call would have been valuable, and we definitely felt the impact of losing that. But we’ve successfully bootstrapped our DevOps teams nevertheless, and we look forward to continuing to evolve as a team.

  • Code Security: SAST, Shift-Left, DevSecOps and Beyond

    Code Security: SAST, Shift-Left, DevSecOps and Beyond

    Automation is the key to pushing code security beyond DevSecOps

    In virtually every industry, developers are dealing with ensuring the safety of their code. Regardless of whether it’s enterprise or applications for automotive, aviation or industrial controls, small systems or large systems, every organization struggles with security. In response, the industry continues to evolve to improve automation, speed of delivery and code reliability. Let’s consider the key building blocks of today’s application development ecosystem.

    Static analysis or static application security testing (SAST) helps enforce coding guidelines and detect undefined behavior, and is commonly used across the development ecosystem. One advantage of SAST is automation. Since it doesn’t require input, SAST is easy to introduce into an existing software development process where it will inspect source code builds and generate warnings. Coding guidelines such as MISRA, CERT C and others help avoid common programming mistakes. However, a second category, undefined behavior, is a bigger issue. That’s because undefined behavior introduces risk. For example, a null pointer dereference, buffer overrun or uninitialized variable are exploitable vulnerabilities. Depending on their severity, they typically will require issuing a patch for customers.

    Next, let’s consider DevSecOps, which spans development, security and operations. Although it is becoming widely adopted in enterprise markets such as financial services, markets such as automotive and industrial are lagging. Nevertheless, the emphasis on security is driving the DevSecOps push to make life easier for software developers and to bring them more in line with security and operations functions.

    For example, quick feedback cycles are extremely important in every industry. Consider a use case in which it can take up to an hour to deploy a newly developed feature or a fix once it is written. That’s unthinkable in today’s day and age when new source code must run almost instantaneously.

    One of the most important elements in DevSecOps revolves around a project’s branching strategy. In addition to the main branch, every developer uses their own separate branch. They develop their code and then merge it back into that main branch. A key requirement for the main branch is to maintain zero unresolved warnings so that it passes all functional testing.

    Therefore, before a developer on an individual branch can submit their work, it also needs to pass all functional tests. And all static analysis tests need to pass. When a pull request or merge request has unresolved warnings, it is rejected, must be fixed and resubmitted. These include functional test case failures and static analysis warnings.

    Functional test failures must be fixed. However, the root cause of the failure may be hard to find. A functional test error might say, “Input A should generate output B,” but C comes out instead, but there is no indication as to which piece of code to change. Static analysis, on the other hand, will reveal exactly where there is a memory leak and will provide detailed explanations for each warning. This is one way in which static analysis can help DevSecOps deliver the best and most secure code.

    Finally, let’s review Lean and shift-left, and see how they are connected.

    Lean development is focused on delivering value to the business. If we fix this defect, is the business going to get value? If we fix this warning, is the business going to get value? In other words, eliminate waste, make the right decisions, at the right point in time.

    Zero defects is actually one of the big underlying philosophies behind Lean. And of course, zero defects never means zero defects. It really means near-zero defects. For example, when there’s no value in fixing a particular defect, it can be classified as don’t fix. Meanwhile, if a defect exists such that an application will behave badly when a user inputs a negative value in the add function, there are two options: Fix it or don’t fix it, if the expectation is users will never input negative values.

    The objective is to achieve a zero-defect baseline and continuously improve. In this scenario, zero defects refers to source code problems as well as warnings from static analysis.

    Last is the concept of shift-left, which is a movement to bring certain processes closer to the developer—testing, in this case. By providing faster feedback cycles on new code, developers can avoid slowdowns. When working on source code projects, if regression tests run several days after a block is submitted and reveal defects, it can be difficult for developers to recall what they worked on at the time. Faster feedback that occurs in minutes, not days, improves quality and security. In this way, shift-left is a central component of DevSecOps and Lean. And static analysis is part of that shift-left.

    So what’s next? In a word, more automation. Making sure developers have as little cognitive overhead as possible will allow them to build code faster and more securely. This can be achieved by combining the concepts described above as well as analyzing code as soon as new functionality is developed using behavior-based testing or test-driven development. The ultimate goal is faster feedback cycles that accelerate development cycles.

  • ServiceNow Partners With IBM on AIOps

    ServiceNow Partners With IBM on AIOps

    ServiceNow and IBM this week announced that the Watson artificial intelligence for IT operations (AIOps) platform from IBM will be integrated with the IT service management (ITSM) platform from ServiceNow.

    Pablo Stern, senior vice president for IT workflow products for ServiceNow, said once that capability becomes available later this year on the Now platform, IT teams will become on average 20% more productive. This is good timing, as most teams currently are struggling to adapt to the increased demand for IT support from employees working from home and in the office in the wake of COVID-19 pandemic.

    Mike Gilfix, vice president of cloud and integration and chief product officer for Cloud Paks at IBM, said IT teams will also be able to proactively identify anomalies in IT environments 60% faster.

    IT teams interact with IBM Watson AIOps via a chatbox interface such as Slack. The platform will surface recommendations based on ServiceNow data and enable IT teams to query IBM Watson AIOps to provide a faster mean time to resolution for IT incidents, said Gilfix. That capability is provided via a configuration management database from ServiceNow that tracks the operational state of all business services in real-time.

    IBM Watson AIOps, which can be deployed in the cloud or on-premises, also can surface additional context the next time similar issues arise, in addition to being able to record actions and insights for auditing purposes. IT teams can also feed data from other systems into IBM Watson AIOps to automate a wider range of processes.

    AIOps platforms won’t be replacing IT professionals anytime soon. However, IT teams that are not able to access AIOps platforms will find themselves working harder than their peers. As IT environments become more complex, the number of potential issues that can adversely impact end user experience are moving beyond the capabilities of an IT team’s ability to track manually.

    AIOps platforms, meanwhile, continue to advance in sophistication. Given the structured nature of IT environments, IBM Watson AIOps is now able to have a significant impact on productivity with 24 hours of being deployed, Gilfix said. As part of this launch, IBM is expanding its global ServiceNow services offering to include additional capabilities that provide advisory, implementation and managed services on the Now Platform. IBM also has formed an AIOps Elite Team that will help prepare organizations to transition to AIOps free of charge.

    IT may take a while still for IT teams to adjust culturally to AI, but there are many rote tasks that many IT professionals would like to see eliminated. Over time those tasks conspire to increase the level of drudgery IT professionals experience as they perform the same repetitive tasks. In theory, at least, AI should reduce overall IT staff fatigue in a way that could reduce turnover while at the same time freeing up resources to tackle more complex IT challenges.

    It may take some time for IT teams to trust the recommendations surfaced by an AIOps platform, but the more those platforms learn about the IT environment, the more accurate those suggestions become.