Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.
This week: PyPI complies with a “string of subpoenas,” and LTO continues to grow, despite predictions of its demise. (more…)
Amazon Web Services relies on the AWS IAM service to govern who is authenticated and authorized to use AWS resources. It plays a hugely significant role in AWS security–and so do its various identities. Let’s look in more detail at IAM roles and how they work.
When you create an AWS account, it has a single sign-on (SSO) identity called the AWS account root user, which has complete access to AWS services and resources. To avoid potentially disastrous issues arising from this kind of unfettered access, IAM allows shared access to the AWS account via identities.
Based on the security-first model, AWS restricts all actions for all identities by default–apart from the root user. This restriction enables us to manage granular access for identities following the principle of least privilege, so permissions are granted only as required to execute daily tasks.
AWS users, federated identities and AWS roles are the three major categories of AWS Identities. IAM roles are conceptually similar to AWS users, but whereas a user is uniquely associated with a principal(users/apps/etc.), a role can be assumed by anyone who needs it.
Rather than passwords or access keys, temporary security credentials are provided to whoever assumes the role. This eliminates the overhead of managing users and their long-lived credentials.
Any of the following entities can assume a role to use its permissions:
● AWS user from the same account
● AWS user from a different account
● AWS service
● Federated identity
The two essential aspects of an IAM role are the trust policy (who can assume the IAM role) and the permission policy (what the role permits the user to do).
Trust policies use specified conditions to define and control which principals can assume the role. They prevent the misuse of IAM roles by unauthorized or unintended entities.
Permission policies define what the principals assuming the role are allowed to do.
AWS IAM roles fall into one of the following major categories based on their trust policies:
● Service role
● Service-linked role
● Web Identity role
● SAML 2.0 federation role
● Custom IAM role
Service Roles
AWS services are the trusted entity type for these roles, which are created to allow AWS services to perform actions on the user’s behalf. They do this by inheriting the permissions assigned to the service role.
You might wonder why AWS services need permission to access each other. It’s because, by default, even AWS services have no access to the resources in the AWS account. However, service roles allow AWS services to access resources based on their requirements.
Most AWS services rely on service roles for optimal functioning. For example, they allow Cloudformation to create and delete resources on the user’s behalf based on a YAML or JSON file. The Amazon EC2 IAM role is a special type of service role. EC2 relies on an instance profile as a container for an IAM role, which is then assumed by the applications running within the EC2 instance to perform actions the role allows.
Service-Linked Roles
A service-linked role is a unique kind of IAM role linked to an AWS service. It simplifies the process of setting up a service by automatically adding all the required permissions for a service to perform actions on the user’s behalf. It is predefined by the service. Most service-linked roles do not permit changes to trust or permission policies.
Web Identity Roles
A user assumes a web identity role when they log in to AWS using an identity provider (IdP) such as Amazon and Facebook. Users do not have an identity within AWS itself; in exchange for an authentication token, they get temporary security credentials in AWS that map to an IAM role that is authorized to use the resources in the AWS account.
SAML 2.0 Federation Roles
These roles are assumed by users who are included in an external user directory, typically within organizations. This enables federated single sign-on (SSO), so that organizations can give users access to the AWS console and CLI without having to create a separate IAM user for each person. An organization that manages its employees in Microsoft Azure Active Directory could connect to AWS directly and give its users access to the AWS console and CLI based on the permissions provided to the SAML 2.0 federation role.
Custom IAM role
Custom IAM roles do not fit any of the other categories and support scenarios other than the ones listed above.
An entity can assume a role for as long as the IAM role’s session duration property dictates. For example, if the session duration for a role is set at 12 hours, the temporary credentials will expire 12 hours after they are issued.
If you assume the role using the assume-role* command, you can specify a value for the session duration using the duration-seconds flag, ranging from 15 minutes to the maximum session duration permitted for the role. The default duration is one hour, but it can be extended to up to 12 hours. The session duration can be adjusted by editing the role.
One role can assume another role in a process known as role chaining. Role chaining cannot extend beyond an hour on the user session; it doesn’t follow the role’s maximum session duration field.
IAM roles are often leveraged to enable cross-account access – the process of leveraging a principal in one account for access to resources in a different account. Organizations often maintain multiple AWS accounts to segregate environments such as development, staging, test, UAT, and production, and they grant identities from one account permission to access resources in another.
This could be to process data in the production account and then anonymize and copy it to the UAT account, so that the frequency and attributes of data are the same. This keeps the UAT environment as close to the production environment as possible.
Both identity-based policies and resource-based policies are permission policies, but identity-based policies are attached to identities such as users, roles, or groups, and resource-based policies are attached to resources such as S3, Amazon SQS queues, or VPC endpoints.
Identity-based policies specify the actions the identity can take and on which resources, whereas resource-based policies determine who is allowed to access the resource to which they are linked and specify the actions they can perform. Resource-based policies must be inline and not managed. This list provides details of which resources support resource-based policies.
Where both identity and resource-based policies are present, AWS evaluates them together. An action is permitted if it is allowed in either or both policies. If either policy contains an explicit DENY, it overrides the ALLOW.
You can provide cross-account access using just the resources policy rather than using a role as a proxy. This is achieved by attaching a resource-based policy on the resource you want to share. This resource policy comprises all the principals allowed to access this resource, in contrast with an identity-based policy, which specifies the resources a principal has access to.
Using a resource-based policy for cross-account access rather than roles has advantages.
When a user assumes a role, they cede the permissions they have in the trusted account so that they can secure permissions in the trusting account (account with shared resources). With a resource-based policy, the user keeps their permissions in the trusted account, and they also secure access to the shared resources in the trusting account. The principal can access both accounts.
AWS IAM Roles Anywhere is a kind of service role that permits on-prem machines or workloads external to AWS (such as servers, containers, and applications) to access resources on AWS by acquiring temporary security credentials. This completely eliminates the headache of managing long-term credentials.
Workloads are required to have X.509 certificates issued by a certificate authority, which must be registered with IAM Roles Anywhere as a trust anchor. Though a relatively recent use case, it marks a move toward minimizing the usage of long-term credentials.
● No management of long-term security credentials
● Support for Single Sign-on (SSO) using SAML 2.0
● Support for web identity federation, allowing users to log in via popular external identity providers (IdPs), such as Amazon, Facebook, and Google
● Enhanced security posture because you don’t have to rotate and replace short-term credentials (An expiry is already associated with them).
● Support for use cases such as cross-account access, Identity federation, AWS IAM Roles Anywhere
IAM roles are key pillars supporting, not just the IAM service, but the entire authentication and authorization flow. Their value extends beyond applications such as cross-account access, AWS IAM Roles Anywhere, and identity federation, being important for enhancing any organization’s security posture substantially. When used to their full potential, they can be extremely useful in IAM.
Dell Technologies has quietly launched a training and education initiative to significantly expand the number of IT professionals that have DevOps skills.
Brad Maltz, senior director for DevOps and developer relations ecosystems for Dell, said the next era of IT will require IT professionals that understand how to automate the management of application development and deployment. Given its long infrastructure history, Dell is especially interested in helping IT teams to use a mix of graphical tools and command line interfaces (CLIs) to programmatically deploy software at much faster rates and at scale, he added.
Dell has already hired more than 200 specialists to provide that training as part of a DevOps Transformation Services consulting services engagement, said Maltz. The primary area of focus will be working with IT organizations that are embracing platform engineering to centralize the management of DevOps and GitOps workflows, he noted. Many of those teams are now relying on declarative automation tools that ultimately make DevOps more accessible to a wider range of IT professionals, added Maltz.
The number of IT professionals that employ graphical tools based on IT management frameworks such as ITIL still far outnumber the number of IT professionals that have DevOps skills. The ongoing debate between these two camps essentially comes down to the need to be able to consistently manage IT at scale versus being able to adroitly keep pace with the needs of organizations that are increasingly dependent on software. In the digital transformation era, it’s become clear that more of the organizations that hire IT professionals are placing a higher value on agility, said Maltz.
Bridging the divide between ITIL and DevOps to address that requirement is largely a psychological challenge that requires IT teams to address a range of cultural issues in addition to mastering different types of tooling, he added.
Regardless of whether organizations continue to manage IT infrastructure internally or rely more heavily on managed services provided by Dell, the need to continuously deliver secure software faster has become a business issue. The trouble is the level of DevOps expertise within organizations tends to vary widely. A recent survey published by the Continuous Delivery (CD) Foundation found only 22% of respondents are using both continuous integration (CI) and continuous delivery (CD) to automate all building, testing and deployment of code in a production environment. Most organizations that have adopted DevOps make extensive use of CI to build applications, but CD has proven to be more challenging to master. In fact, the CD Foundation survey found only 7% of respondents described themselves as DevOps engineers or specialists.
The bulk of the managed services that Dell provides are focused on managing infrastructure rather than deploying software, but the company does have a managed DevOps offering based on the application development and deployment platform that an internal Dell DevOps team built to construct and deploy applications. Naturally, Dell has a vested interest in accelerating the rate at which software can be deployed on IT infrastructure it provides and manages.
It may be a while before the overall size of the DevOps community expands, but the training initiatives such as the one launched by Dell will help. However, it may become a lot easier to find and retain that talent in the months and years ahead. In the meantime, organizations would be well-advised to invest in upskilling their own IT teams. After all, it’s less expensive to train someone that already works within an IT organization than it is to compete against thousands of other organizations willing to pay top dollar for the best DevOps expertise money can buy.
I have written this blog half a dozen times over the years, but every once in a while something comes up that makes me realize I should revisit it once more for those in the back. This time, it was a social call with a group of friends, some of whom are in high-tech, all of whom are smart and often insightful. One member of the group, a tech lead for a company that you do business with, was talking (in vague terms, of course; this person is a professional) about work he is doing, and the group started to make fun of documentation.
Let’s face it—since the very first line of code was written, we have sucked at documentation. It is a rare project that (a) gets just enough documentation and (b) keeps that documentation up-to-date. What we most often get is no documentation or documentation that records the initial design but, by the end, bears little resemblance to the code’s actual workings. The worst of all types of documentation is the “It’s self-documenting!” kind, which has gotten another boost based on the idea that OpenAPI/Swagger will be a tell-all about what is actually happening with the API. Like far too much API testing, it does a good job of telling you about inputs and outputs and, in most cases, absolutely nothing about what is done with the inputs or what magic happens to get you the outputs.
After our call, I got to thinking. What can we do to adequately document the code? I mean, with AI becoming all the rage and the set of lines of source code available for training being so massive, you would think we could do better. Maybe we can—but right now, the language processors that we currently pass off as AI are too busy being conversational or generating the code itself. Both of those spaces are going to take a while to get “good enough” to rely upon without review (not as long as many think, but certainly not tomorrow). Maybe that point will be when some enterprising engineer will turn it on source. Though I think that it will take a documentation specialist and not a coder—after all, we coders created this problem and haven’t fixed it in decades.
Many of us truly believe the “self-documenting” part when that is and absolute lie. It may be self-documenting to someone who wrote it or, in the best cases, someone with deep domain knowledge, but certainly not to the person who has to pick it up. We all have horror stories of working on that application—or porting/rewriting an application—that claimed to be documented or self-documented and was not even close. The saving grace in all of the cases that I worked on was that the code actually worked and performed the job required. Given some extra time to step through it, it could be rewritten even without good comments and docs. (Hat tip to the dev that had to pick up one massive porting project from me because I left to go write about storage—the project was huge and was a blast, but having to hop in halfway through probably made him crazy.)
If you are planning for the future, I recommend a ton of things, but for documentation, only two. First, write design docs, discuss decisions and put them in Git with the source code. Require that they be reviewed and updated after each sprint. Then, make sure that there are comments in the code. Using naming conventions to document code is helpful, to be sure. But more is needed—particularly as code morphs into whatever it is going to be in long-term final form. We have all seen that method or function that has a name … and then doesn’t do what the name implies. Or worse, that does what the name implies but also applies half a dozen side effects because devs needed somewhere to put new functionality. And we haven’t even talked about complexity like feature flags.
So do both, and require that devs spend a bit of time on each during each sprint—not to be a jerk, but to pave the way for dev #327 that inherits this code and has to make sense of it. Help them get productive quickly.
And keep rocking it. We have documentation problems because we solved business problems rapidly. That’s a good thing, we just have to think about the next gen of devs that will be on this project, and give them that little boost into productivity that documentation offers. And maybe you’ll help the compliance team while you’re at it, but that’s a different blog.
Red Hat today unveiled a technical preview of Ansible Lightspeed with IBM Watson Code Assistant, a generative artificial intelligence (AI) service specifically trained on domain-specific data collected via the Ansible automation platform.
At the same time, Red Hat is also making available an event-driven capability within version 2.4 of the Ansible platform that enables runbooks and other tasks to automatically execute when received from a third-party tool or application.
Both capabilities were announced at the Red Hat Summit and are part of an ongoing Red Hat effort to democratize IT automation.
First previewed within the Project Wisdom initiative launched by Red Hat and IBM Research last year, Ansible Lightspeed IBM Watson Code Assistant is based on a large language model (LLM) that was developed using Ansible data. Watson Code Assistant, currently in technical preview, is a service that automatically generates suggestions to improve code as developers create an application. IBM promised to make IBM Watson Code Assistant available later this year.
Red Hat is now applying that generative AI capability to the YAML code that DevOps teams need to create Ansible playbooks. IT teams can create a simple natural language prompt that will translate their domain expertise into YAML code for creating or editing Ansible Playbooks. Red Hat and IBM, however, will not automatically use the code from an organization to fine-tune their AI model without express consent.
Rich Henshall, director of product management for Red Hat Ansible, said as IT environments become more complex, it’s clear many are hitting a ceiling in terms of what can be automated using legacy custom scripts. Ansible Lightspeed IBM Watson Code Assistant, along with an event-driven capability, will make it much simpler to automate the management of IT environments at scale using a declarative framework, he added.
Ansible, for example, can now receive events from third-party tools, determine next steps and consistently initiate an automated response using Ansible playbooks. Ansible can also consume events from third-party monitoring, observability and IT analytics tools from providers such as Cisco ThousandEyes, CyberArk, Dynatrace, F5, IBM Instana, IBM Turbonomic, Palo Alto Networks and Zabbix with more to come, noted Henshall. Red Hat is also making available supplementary content for Red Hat OpenShift, Red Hat Insights, Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform and ServiceNow, in addition to enabling IT teams to create their own custom integrations.
At this juncture, it’s not clear why more IT processes have not been automated using Ansible or other frameworks. Many DevOps teams continue to rely on custom scripts that typically don’t scale and are generally poorly documented. Any time the individual who wrote those scripts leaves an organization, it’s incredibly difficult—if not impossible—for the original scripts to be maintained, noted Henshall.
One way or another, it’s clear that a more scalable approach to IT automation is required, especially with the rise of more complex cloud-native application environments. It’s no longer economically viable for most organizations to rely on legacy processes that require a small army of IT professionals to maintain.
Red Hat today announced a portfolio of cloud services designed to better secure software supply chains. The expanded portfolio includes Red Hat Trusted Application Pipeline to secure continuous integration/continuous delivery (CI/CD) workflows and Red Hat Trusted Content offerings to better secure software components.
Announced at the Red Hat Summit, the Red Hat Trusted Software Supply Chain combines a suite of curated open source tools into a cloud service managed by Red Hat. The Red Hat Trusted Application Pipeline service is based on sigstore, an emerging de facto standard for cryptographically signing code to ensure immutability that Red Hat helped drive, while Red Hat Trusted Content provides access to a set of software components that Red Hat has vetted.
Red Hat Trusted Software Supply Chain also provides access to existing tools such as Quay, a registry for containers, and the Advanced Cluster Security (ACS) platform for securing Kubernetes clusters.
Red Hat Trusted Software Supply Chain is scheduled to be made available in preview in the next few weeks, and will also track the provenance of components. In addition, the service will monitor emerging threats to software components and alert users of new and emerging risks. The service will also surface recommendations for remediating vulnerabilities it identifies based on best practices defined by Red Hat.
Finally, Red Hat Trusted Application Pipeline also promises to make it simpler to automate the deployment of secure code with a few clicks within the context of an integrated continuous integration/continuous delivery (CI/CD) pipeline. DevOps team can also import Git repositories and configure container-native continuous build, test and deployment pipelines, inspect source code and transitive dependencies, autogenerate software bill of materials (SBOMs) within builds and verify and promote container images. An enterprise contract policy engine can consistently enforce compliance with the Supply Chain Levels for Software Artifacts (SLSA) framework.
Finding ways to implement DevSecOps workflows is a major challenge for any organization. Red Hat is making a case for a managed cloud service based on a collection of security software and services that a DevOps team would otherwise have to implement themselves.
Tasked with securing increasingly complex software development environments, it’s not clear whether DevOps teams will opt to rely on a cloud service, but given the general lack of security expertise among software development teams, the fastest way to secure a software supply chain for most organizations is to rely on a service.
Regardless of approach, of course, DevSecOps will soon become a standard element of application development workflows. Once that occurs, it’s not clear whether DevOps will simply become DevSecOps or whether security will be assumed to be an integrated element of any set of DevOps best practices.
Sarwar Raza, vice president and general manager for cloud services at Red Hat, said regardless of approach, it’s clear that implicit trust is no longer good enough. Organizations of all sizes will need to be able to demonstrate that they have maintained a chain of custody for the software components used to build an application, he noted.
One way or another, the overall security of downstream applications should eventually improve as the processes used to construct software are increasingly locked down.
The software supply chain has become increasingly complex and dynamic with the rise of cloud computing, open source software and third-party software components and APIs. Widespread damage can occur if third-party APIs, cloud services, SDKs and open source software have security flaws. As a result, software supply chain security has emerged as a critical concern for organizations across industries. Supply chain attacks have become more frequent, sophisticated and devastating, as evidenced by high-profile incidents such as SUNBURST, Kaseya and Apache Log4j.
To address the growing challenge of software supply chain security, organizations have turned to leveraging software bills of materials (SBOMs). SBOMs are a standardized inventory of software components used in a particular product or system, including their versions, dependencies and sources. SBOMs provide transparency and visibility into the software supply chain. They can be an effective approach for identifying and mitigating security risks, compliance issues and operational challenges – assuming organizations have the right tools to fully benefit from SBOMs, including runtime discovery, in place.
SBOMs are not a new concept, but their adoption has accelerated in recent years due to a number of important factors, including the following.
First, regulatory agencies, such as the U.S. National Institute of Standards and Technology (NIST) and the Cybersecurity and Infrastructure Security Agency (CISA), have been advocating for SBOMs as a best practice for software supply chain security. NIST has released a draft guidance document on SBOMs, and CISA has mandated SBOMs for federal contractors and subcontractors.
Second, industry initiatives, such as the Open Source Security Foundation (OpenSSF) and the Software Assurance Forum for Excellence in Code (SAFECode), have promoted SBOMs as a key element of software security. OpenSSF has launched a working group on SBOMs and published the Open Source Software Security Mobilization Plan to improve the resiliency and security of open source software (OSS), including guidelines for SBOM management; SAFECode published a whitepaper on SBOMs.
Third, supply chain attacks have highlighted the need for SBOMs as a proactive and preventive measure. SBOMs can help organizations detect and respond to vulnerabilities, malicious code and unauthorized changes in the software supply chain. SBOMs can also support incident response and recovery efforts by providing accurate and comprehensive information about the affected software components.
Fourth, customer demands and market pressures have influenced the adoption of SBOMs. Customers are increasingly aware of software supply chain risks and are asking for SBOMs as part of their procurement and evaluation processes. Some industries, such as health care and automotive, have mandated SBOMs for safety and liability reasons. SBOMs can also enhance brand reputation, customer trust and competitive advantage for software vendors.
Fifth, technological advancements have facilitated the creation and consumption of SBOMs. The emergence of open standards, such as CycloneDX and SPDX, has enabled interoperability and integration of SBOMs across different tools and platforms. The development of automated tools, such as software composition analysis (SCA) and vulnerability scanners, has enabled the generation and validation of SBOMs in a timely and accurate manner. The integration of SBOMs with other security measures, such as identity and access management (IAM) and security information and event management (SIEM), has enabled an effective view of software supply chain security.
However, SBOMs are not a silver bullet for software supply chain security. They are only as good as the data they contain, and the quality of data can vary depending on the source and the method of collection, particularly around the application software stack of APIs, cloud services and SDKs. SBOM inventory is constantly changing, and being able to leverage their up-to-date data requires more than a snapshot from traditional source code static analysis approaches, but rather continuous runtime analysis and dynamic inventory.
For example, common vendor management and software composition analysis (SCA) approaches often lack source code access for mobile, web, cloud and commercial-off-the-shelf (COTS) software, as well as third-party API services.
For a more effective approach, organizations can benefit from a full-stack attack surface management (ASM) software supply chain solution that delivers continuous third-party application asset discovery and dynamic tracking of third-party vendors. These newer approaches automatically categorize assets under known vendors, allow customers to add additional new vendors, curate individual assets under any vendor, and alert on increases in policy violations and high embed rates of third-party vendors within key applications. These automated capabilities allow vendor management teams to remedy supply chain security problems more quickly and easily and much more accurately than older approaches.
It is important to note that SBOMs also require collaboration and communication among multiple stakeholders, including software vendors, suppliers, customers and regulators. They may also pose some privacy and intellectual property concerns since they can reveal information about the software components and their relationships.
Overall, organizations need to adopt a comprehensive and risk-based approach to software supply chain security, of which SBOMs and tools to best leverage their information are an important component. Organizations should establish policies and procedures for SBOM creation, management and sharing, as well as for validation and verification. Organizations should also integrate SBOMs with other security measures and practices, such as threat modeling, penetration testing and code review to best ensure the security of their software supply chains.
Is the DevOps world slowly backing away from microservices and to a monolithic renaissance? After years of proselytization about the benefits (Flexibility! Resilience! Support faster release speeds!) of microservices and serverless architecture, there are signs of a potential backlash against the singularly fragmented microservices architectural approach.
The argument in favor of rolling back to monolithic architectures feels similar to the cloud repatriation debate. Just as with a lot of cloud-first transformations—often in lockstep with them, actually—many organizations moved headlong toward microservices without a case-by-case business or technical rationalization for the change. As the shiny newness of microservices has dulled under that approach’s daily exposure to real-world deployment and business challenges, the monolith isn’t looking so bad in a lot of situations.
The monolithic versus microservices debate boiled over the last few weeks in the wake of the publication of a case study earlier this month from the Amazon Prime Video team. The analysis from Marcin Kolny, senior software development engineer for Prime Video, reported that his team was experiencing scaling bottlenecks in its audio/video quality inspection tooling, which was architected as a distributed system using serverless components. By rearchitecting the tool as a monolith, they were not only able to increase scaling capabilities but drastically reduced the infrastructure costs around that tool.
“Moving our service to a monolith reduced our infrastructure cost by over 90%,” Kolny wrote.
The announcement elicited a fair degree of hyperbole on the topic, but it also spurred a very organic airing of the pros and cons of monolithic, microservices and serverless architectures.
The following are 10 different perspectives spurred on by this online debate from a range of CTO and engineering experts. Some of them are diametrically opposed to one another, many are overlapping in their opinions. But as a body, they point to the general (and very pragmatic) consensus that there simply will never be a one-size-fits-all approach to software architectures.
“It’s clear that in practice, microservices pose perhaps the biggest siren song for needlessly complicating your system. And serverless only makes it worse. Replacing method calls and module separations with network invocations and service partitioning within a single, coherent team and application is madness in almost all cases.”
—David Heinemeier Hansson, creator of Ruby on Rails and CTO at 37signals (maker of Basecamp and HEY)
“Application architectures are neither good nor bad, nor are they appropriate for every use case. Organizations need to step back and consider carefully what it is they’re trying to achieve and which architecture might best serve that need instead of choosing the most “modern” architecture because, well, it’s in vogue.”
—Lori MacVittie, distinguished engineer in office of CTO at F5Networks
“In the ongoing debate between monolithic and serverless architectures, it is important to acknowledge that both approaches have their merits and considerations. Monolithic architectures have been prevalent in large human-centric organizations due to their management advantages and historical use cases. However, as the world continues its rapid progression towards a more AI-driven future, the landscape is shifting towards architectures that embrace fluidity, autonomy, and distribution.”
— Reuven Cohen, veteran CTO and entrepreneur, mentor for TechStars
“Over the last 15 years, everyone has wanted to move to serverless microservices-based horizontally-scaled architecture – but it is not ideal for every application’s specific needs. As with any choice in life and in business, there are always tradeoffs; you must evaluate and choose the appropriate architecture based on your business and technology needs. Another important dimension to this is workforce skills for application architecture.
—Hemang Davé, client technical leader and CTO at Kyndryl
“The problem is that they called this refactoring a microservice to monolith transition when it’s clearly a microservice refactoring step, and it is exactly what I recommend people do in my talks about serverless first. I don’t advocate “serverless only”, and I recommended that if you need sustained high traffic, low latency and higher efficiency, then you should re-implement your rapid prototype as a continuously running autoscaled container as part of a larger serverless event-driven architecture, which is what they did. If you built it as a microservice to start with, it would probably take longer (especially as you have to make lots of decisions about how to build and run it) and be less able to iterate as you figure out exactly what you are trying to build.”
—Adrian Cockcroft, longtime DevOps advocate and veteran architect with stints at eBay, Netflix, Amazon, and Battery Ventures, among others
“Microservices, while duplicative and inefficient from a compute standpoint, allow you to release stuff really quickly, which is a tradeoff that most organizations at scale are willing to make.
Microservices exist to reduce communication pathways—not to solve technical scaling challenges. At a certain point, teams stop functioning due to the communication overhead. A seven-person team has 21 communication pathways, whereas a 100-person team has 4,950. I work with large retailers selling billions of dollars a year online, and putting everything into a single monolith just doesn’t work. Could the entirety of Amazon.com be deployed as a single monolith? No way.”
—Kelly Goetsch, chief strategy officer at commercetools
“Speed to market and speed to develop does not always equate to long-term scalability and maintainability. You must actively balance your investments across these two critical pillars to build a viable product. Active rebalancing and reconsideration of the stack and technology mix for your FinOps and BizOps context is both art and science.
Cloud-native distributed systems paradigms are here to stay. The Prime Video folks have mistakenly labeled a properly factored, highly coupled data-intensive processing step a monolith—it’s at best a rocky outcropping in their distributed microservices forest. “
—Ajish George, managing director of cyber data & analytics for State Street
The Shiny Thing is Not Always the Best Thing
“The news that AWS discovered that serverless and microservices aren’t for everything is surprising. Not because it didn’t work out for that use case; but because the wrong use cases have been so well-known for the serverless/microservices pattern and should have been obvious to Prime Video. It’s worth remembering: That shiny new thing isn’t necessarily a better way of trapping mice. Right tool for the right job.”
—Doug Vanderweide, director of cloud architecture for Avanade
As organizations transform and grow, different methodologies and development tools can come together, creating an extra level of complexity for your mainframe application development teams. In a perfect world, developers would be able to focus on delivering solutions without worrying about all the infrastructure necessary to make that happen. Building a better developer experience, or DevX, can make that a reality.
DevX is about meeting developers where they are, at every stage of their career, delivering compatibility with and connectivity to the tools and devices that they want to use so they can focus on what they do best: Coding and releasing high-quality mainframe applications faster and more frequently and fostering better productivity and more innovation.
Automation is integral to removing friction for developers. A modern integrated development environment (IDE) with built-in automation allows developers to find and manage code more efficiently and with greater quality and move quickly and easily from coding to editing to debugging to testing.
Modern tooling can simplify and accelerate how developers work by giving them a visual representation of their code layout in a single-pane-of-glass view. They can move quickly in a natural, easy way with more information at their fingertips while built-in automation offers assistance at the right moment without getting in the way.
This can seem difficult when dealing with mainframe application development and longtime mainframe developers who are still working with tools from the 1980s haven’t directly benefitted from the intervening decades of improvements. What they might not realize is that the improvements made for other platforms are available to mainframe developers, too. By engaging them in your DevX modernization efforts, you can improve their experience—and that of their team members.
Once you have the tools, your mainframe developers need to be supported by a culture and education that teaches them how to use them and understand the benefits they offer. If you implement new tooling that nobody will use, the transformation is for naught. It’s important to start with the user experience and find the right tools to address it rather than starting with a tool and forcing its use just because it’s new. Continuous improvement is an important part of this; otherwise, you could bring the mainframe developer experience from the 1980s up to today’s standards only to have it freeze in place. This is why metrics and analytics are so important.
As you speed up delivery, you also want to make sure the quality improves. With an improved DevX, you’re consistently finding defects earlier in the life cycle, which results in safer, more efficient delivery of better-quality code. By tracking velocity with metrics such as user stories delivered in one sprint, you can gauge how those changes are making a difference. You can also demonstrate how automation helps eliminate human error, increases code drops, and improves business agility.
If you’re giving your mainframe developers everything they need to do their jobs easily and more efficiently, they’re exponentially more likely to adopt the tool and advocate for it—because a better DevX improves the day-to-day employee experience for longstanding and next-generation developers. By removing the drudgery of tasks that take developers away from writing code, you’re unleashing their creative and enabling them to achieve even greater levels of innovation—and you’re also improving your hiring and retention practices.
Improving the mainframe DevX pays dividends across your business, from your developers to your customers to your stakeholders, so you can bring more new features to market faster and enable a competitive response for revenue-generating solutions. The mainframe is here to stay. Make sure you’re building a modern development environment that empowers your developers to respond nimbly in the face of change and make change your advantage.
DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.
Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.
Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.
The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.
We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:
Samsung Electronics America
Mountain View, California
Lead Site Reliability Engineer (SRE) – DevOps Specialist
$180,000 to $210,000
Indeed.com
Federal Reserve Bank of New York
New York, New York
DevOps Engineer
$178,100 to $207,900
CareerBuilder.com
SAIC
Chantilly, Virginia
DevOps Advisor
$130,000 to $200,000
Dice.com
Saicon Consultants
Alpharetta, Georgia
DevOps Engineer
$110,000 to $140,000
Simply Hired.com
Hewlett-Packard Enterprise
Fort Collins, Colorado
DevOps Automation Engineer
$109,400 – $251,500