devopsy.kinsta.cloud

Category: Enterprise DevOps

Picking a Git for the Enterprise
Some of my Agile and DevOps transformation work, not surprisingly, involves helping clients with tooling adoption—to set up, scale and operate across the organization. This blog piece is about a recent experience of setting up Git across a client organization.

Git, as most of you are aware, is a fundamental building block for many of the modern engineering practices associated with Agile and DevOps. Software configuration management tools such as Git, in simple words, allow you to track and manage your software code and configuration changes in the most-effective, efficient and distributed way. Git, as the choice of source control, has remained steadfast over the years, while other tools across various categories of DevOps tools have seen their fortunes go up and down with the ever-changing technology landscape of DevOps tools. In fact, one new buzzword is GitOps—some even calling it the next stage of evolution for DevOps.

Origins of Git

More than a decade ago , the Linux community faced a challenge because none of the existing revision control tools met their need for a distributed version control tool. And from that need Git was born. In the words of the creator of Git, Linus Torvalds, “The main issues with SCMs is the politics around who can make changes.” Ever since its creation, Git has radically changed the notion of software configuration management (SCM), by enabling engineers to work remotely and autonomously. (If you are new to Git and interested in exploring more about Git and its features, Atlassian has a tutorial and GitHub has a list of resources.)

Git and More

While Git as the core product is fundamentally the same whichever flavor your prefer—GitHub, GitLab or Bitbucket—the wrapper around it determines the important non-functional characteristics such as operatability, security, usability and maintainability. Our quest for an enterprise-scale Git focused on the wrapper and not the core product itself, as much has been written, debated and documented on the core product of Git itself already. The value and relevance to enterprises is in the wrapper that these productsplatforms provide.

We looked at the core product and related offerings; as you can imagine, the three tooling vendors have different positioning and taking different directions with the future road map for their respective tools. Bitbucket’s strength lies in the fact that Atlassian is bundling the product with the omnipresent Jira Software and Confluence. GitLab’s road map seems to be taking the product beyond Git and into the idea of a single product for your entire pipeline automation, from concept to cash. GitHub, which is being acquired by Microsoft, is increasingly relying on the strength of integrations with other tools and the workflow capability the company is bundling with it.

(I avoid an overemphasis on the commercial aspect in this blog; however, I’m not ignoring it completely. Part of the reason is due to the fact that enterprise pricing is not entirely transparent and may vary wildly depending on organization’s negotiating capability, the scale of usage, future potential and marketable track records as perceived by the tooling vendors. This blog is also not meant to be a feature by feature comparison, as this is something you can Google and find out yourself very easily.)

The Verdict

Image1; Github Enterprise – Pros and Cons

Image 2: Gitlab Ultimate – Pros and Cons

My Recommendation: It Depends
1. I liked the vision of GitLab as a single-stop tooling platform for continuous integration/continuous delivery (CI/CD), which goes beyond source control. I think the tool is simple and intuitive to use and great for small to midsized setups or even large enterprises that are focused on having smaller and autonomous teams.
2. GitHub seems to be well-suited for large and complex enterprises with significant outsourced and offshore-based technology teams.
I would love to hear your views on what has been your thinking with regards to choosing a Git-based product for your enterprise or teams. Meanwhile, here are some links you might find useful in your decision-making:
1. Why and how GitLab abandoned Microsoft Azure for Google Cloud – https://venturebeat.com/2018/04/06/why-and-how-gitlab-abandoned-microsoft-azure-for-google-cloud/
2. Tooling integrations of GitLab – https://docs.gitlab.com/ee/integration/README.html#doc-nav%20for%20list%20of%20integrations
3. GitHub add-ons are listed at Addons are listed at https://github.com/marketplace
4. Security and scalability info of GitHub is at https://github.com/jonico/github_security_and_scalability_overview
5. Products that integrate with GitHub platform – https://github.com/works-with
6. Pricing of GitLab – https://about.gitlab.com/pricing/
— Aditya Vadaganadam
June 18, 2018
Continuous Delivery is Not Continuous Deployment

Every once in a while, it is worth revisiting a topic that flies under most people’s radar. The differences between continuous delivery and continuous deployment is one of those topics. While those steeped in DevOps are well-aware that software can be delivered daily (or even more frequently), deployment is less frequent, and more dependent upon both development and marketing cycles.

Most CI/CD tools can and do crank out “deployable” apps on a daily basis, or even each time that code is checked in. In theory, each of these builds is a candidate for deployment, and each has had at least some level of testing.

But setting up the environment to host these apps is a separate step. Quite often today, it is a step that is handled by scripts that configure/move everything to build the app environment into a comprehensive system with knowledge of network, data sources, storage, etc.

I’m revisiting this topic because new people often hear “CD” and assume that is the end state. Of course, it is not; the application has to be deployed—to Test and Production at least, and in many enterprises to Staging, also.

While these environments are almost invariably different (Test does not share IP space with Prod, for example), the deployments must mirror each other as much as possible to make Test representative of what Prod will look like.

As with any step in the DevOps toolchain, scripts to achieve continuous deployment are not a great solution. They are certainly a good first step—automating what previously was done by hand—but tools that make the experience easier and more dynamic are available, and should be considered.

Whether an explicit continuous deployment tool or an application release automation (ARA) tool (or an ARA that calls a continuous deployment tool, as we’ve seen some used), something to remove dependence on hand-edited scripts is useful for many reasons, starting with the fact that separate scripts are maintained for Test and Prod and changes must be made in two places when a change is made to an application (such as changing where the DB is located or switching to a dedicated AAA server). This makes scripting fragile. Tools can offer shared deployment and custom deployment sections so that multiple changes can be restricted to those things that truly need to be handled separately.

Deployment is very much a target exercise. The code has been built; deployment is simply giving it a home. Historically, this required agents be strewn about the data center to help target deployments. Increasingly, agentless tools are available that allow the adaptability of continuous deployment while not requiring agents to be installed on every machine that might participate in the deployment activity.

The other major benefit of moving beyond scripts and into a continuous deployment tool is huge: Everything else is version-controlled, which provides built-in rollback. A developer made a mess of fixing that bug, and things are worse? Roll back the update until the issue can be addressed. Tools such as Git make that particularly easy to do. On the deployment side, the same functionality is required. “Release 7.2 has been deemed to violate GDPR” might require moving back to the previous release. But script-based deployments are always forward-looking. It is enough to modify scripts to reflect the current environment, and it is a rare organization that requires parallel development of backout scripts. However, this functionality comes for free with a continuous deployment tool—that which it has built it can undo, because it is versioned automatically.

And the benefit of versioning goes beyond rollbacks: It allows multiple version deployments for AB testing and allows for skipping forward and back as far as necessary to meet current goals.

Is it a panacea? Of course not. Each technology brings new challenges. But continuous deployment solves more problems than it creates. And most of the problems it creates—a desire for greater integration with networking and storage services in non-cloud environments, for example—will be solved in a consistent and repeatable manner by these tools long before any scripting does. Because the tools are designed to create consistent and repeatable results.

While you are out there kicking it, consider implementing continuous deployment tools to tackle the DevOps last mile. They’ll make your life easier, and your environment less fragile.

— Don Macvittie

June 12, 2018
EDAT for ERP Increases Velocity, Ensures Quality
IT organizations today are faced with the challenge of delivering new capabilities at higher velocity to enable digital transformation. While enterprise resource planning (ERP) systems provide a stable integrated foundation and emerging capabilities to meet business needs, their integration and deployment processes were built for a waterfall methodology.

As business agility is paramount to market success, Intel IT works to move to an agile and DevOps methodology to deliver new capabilities at high velocity while maintaining stability at optimal cost. This program built additional capabilities on top of SAP Solution Manager to enable self-service, continuous integration (CI) and continuous delivery (CD). The velocity of self-service has improved by 341 percent while decreasing deployment costs by 45 percent, with no change induced incidents. The quality of integration and deployments was improved, while the complexity of the integrated environment increased. The maturity level for DevOps adoption has been improved and 97 percent of our agile teams have L3 agile maturity with plans to reach L4 within a year.

The features designed to support DevOps included:
- Enabled a complex SAP landscape
- Enabled self-service by bundling needed dependent objects
- Provides adherence to the orchestration of change and release management steps
- Full audit traceability
- Auto rollback capability
- Architected with full DevOps CI/CD
The Agile and DevOps Program Transformation team did an industry scan and conducted detailed vendor evaluations to meet these needs. Most of those tools available introduce redundant change management functions currently provided by SAP Solution Manager. Due to the lack of feature-rich tools to meet the demands of the DevOps culture, Intel created the Enterprise Deployment Automation Tool (EDAT). EDAT solves the problem by providing self-service continuous delivery capabilities to development teams and enabling high velocity while leveraging SAP Solution Manager.

EDAT is a custom-built ABAP Web Dynpro program that integrates Solution Manager with self-service capabilities with workflow automation. The design was reviewed and approved by our internal audit, security, SOX, change management, enterprise and platform review teams. To meet these requirements, strong authentication and tight controls were needed to perform directed steps while restricting access. EDAT automation performs validations of code, approvals and other checks at increased speeds and improved quality.

One feature that makes EDAT unique is the ability to support step releases. Step releases allow transports across multiple changes and systems to deploy simultaneously while providing the ability to pause and resume later. It enables the ability to resume by showing the list of changes in the release and displays the progress completed, which easily allows for co-shared deployment across multiple regions and resources.

Workflow Automation

The picture above depicts the workflow automation implemented in EDAT. Once a user story is created and work is assigned to an iteration, development begins. Unit testing begins in development. Once tasks are released, automated testing is completed in each environment. Before the change can proceed to the next environment, the testing must result in a pass. If there is a failure, the developer is notified so a fix can be created which results in a repeat test until it passes. An automated backup of objects is completed prior to landing to the next environment, so they could be brought back to the previous state should user testing and validation show issues. An automated workflow is orchestrated within each action and code movement, validating and advancing Solution Manager, the service management ticketing system, agile planning tools and automated testing tools. Automated synchronization keeps the environments in sync.

Validation series include both standard and custom validations prior to allowing three methods of deployment

Three methods of deployment are included; One-By-One, Set-Order, and Import-All. One-by-one best suits releases that require many manual steps between transports. Set-order allows for proper sequencing and was designed to allow imports to sequence in serial, while pausing on any landing error. This is often used in releases benefiting from a faster release with on-error stops. Import-all allows transports to land all at once, which best suits releases where the transports do not share objects.

Both standard and custom validations are enabled within EDAT. A structure was set up that allowed for standard validations, which include the various SOX and audit requirements common across all release types. Another structure was set up that allowed for custom validations, which include architecture-specific setup, rule logic and configuration requirements. This dual structure introduced the ability to deploy changes across SAP pipes from within a single screen and deployment rule logic.

EDAT enforces these validations:
- Rule logic based on system and objects.
- SOD/SOX compliance checks.
- Prevention of backdoor addition of unapproved transports.
- Change Manager SAP pipe ownership.
- Release date and the approval date to ensure change management process is adhered.
- Target release systems availability check.
- Lockout, for upgrades and support packs.
Release sequencing is a critical requirement for releases to maintain high quality and ensure a successful release, and is one of the top five reasons why additional resources were needed prior to EDAT. Allowing development teams to order transports and enable the system to import in this order alleviated resource constraints and helped achieve the self-service release capability necessary to increase velocity and improve quality at the same time.

Self-service ability to re-land transports is a key feature that is necessary. Some releases require re-landing a transport to ensure all objects are activated. The only caveat to this feature is to maintain a date check for production landing, to comply with change management.

Landing return codes are shown to the user automatically and full logs are also accessible to the development teams.

Plugin integration enables the ability to allow and support integration across various tools and provide these capabilities:
- Agile planning.
- Auto rollback.
- Pipeline architecture.
- Test automation.
- Live undo.
These capabilities allow EDAT to integrate with other technologies and other platforms through an open API.

EDAT automatically detects objects and platforms which require specific rule logic. Once detected, a summary is displayed to the user so there is an awareness of what will be completed prior to execution. In an upcoming iteration, predictive detection will validate inactive objects in target environments. This will detect issues in advance and initiate fixes automatically.

A single interface with integration across all platforms allows for increased quality, tool reduction and reduced confusion, and provides clear results. This benefit can be seen across multiple service groups and teams. With standard ERP release functionality, users had to use several tools and instances to complete releases. EDAT enabled synchronized and bundled releases from a single tool, where all platforms would import, whether SAP, ABAP or another platform. The ability to implement releases across all platforms is enabling our vision of integrated release across all platforms.

In an upcoming iteration, full test automation will be enabled, where tests will run and are automated. When these automated tests return with a pass, then EDAT will allow them to proceed to the next environment. When the automated tests return with a fail, then EDAT will not allow the code to proceed to the next environment. The development teams will be notified of the failure so a new set of tests can be run after needed fixes. These test results are updated in the SAP Solution Manager change screen for easy retrieval.

One key highlight, there were no change-related incidents and the cadence of releases significantly increased with continuous deployment compared to time-based sweeps. Self-service migrations from Q1 to Q3 2017 increased 341 percent over the same period in 2016.

EDAT made it possible to decrease migration management resources by nearly 45 percent. Migration and release managers are now able to move to higher-value activities. User feedback has been overwhelmingly positive. SAP has since performed a review of the tool and process and acknowledged the change control process has become very agile. The excitement and value realized are positioning the program to continue with improvements and advancing its DevOps’ capabilities.

This article was co-authored by Ramani Subramanian and Jeff Clancy

— Richard Dowdy
May 25, 2018
Functional Programming: Island of Misfit Functional Programming

The age of functional programming is dawning. The hype cycle has heated up, and IT departments are getting both real and perceived benefits out of the idea that a function runs on demand only.

For some, the benefits are very real. Bursty processing, or things that are frequent and quickly resolved, see a benefit from being implemented on functional architectures, and the cost of functional architectures is arguably far better than the cost of cloud instances. I say “arguably” because you can always cram more onto an underused cloud instance, but you pay as you go—even more literally for functional programming, so additions each have an incremental cost. But it is incremental, and you have the power to choose whether functional is right for any given bit of the architecture.

So why the title? Remember The Island of Misfit Toys from the much-beloved Christmas story, “Rudolph the Red Nosed Reindeer”? They were all just a little off. In a world flooded with perfectly good toys, they were left unattended and unwanted.

That is a risk functional runs in the long run, also. The idea that you can pop a function out there, configure the access point (almost ubiquitously a web application firewall entry point, but things change so we’ll stick with the generic “entry point”), and start sending requests is great. The fact that you only pay for that bit of time when your request is actually being serviced is great, too.

But this ideas assumes security is baked in at the functional level. If you are deploying this function anywhere with a public IP, it assumes hardcore security is baked in.

And failure to have that security baked in is the first reason you might have a misfit toy.

By definition, functional programming-based systems are complex systems. They are the ultimate evolution of microservices. They will suffer all of the headaches and heartaches associated with SOA and add on the issues with REST versioning, etc. There will be 50 or 500 functions, each clipping along and doing its thing, composed to make a single application (in a purist world, composed in a functional programming app).

Some of these functions will become outdated and unused. You can’t leave them laying around, waiting to be abused. Indeed, a new form of financial attack might be to look for just such abandoned toys and start calling them repeatedly and rapidly.

Imagine coming in one day and having Finance ask you why your bill at functional provider X tripled in the last month, only to find a function you left hanging around was being called for no good reason?

But the complexity inherent in lots of small bits of code, each doing a specific task, pretty much guarantees there will be cruft left lying around unless you are very careful.

And that, my friends, is (finally, I know) the point of this blog. Our monitoring tools today lean toward watching the performance of the application, but you need more than that before you go all-in on functional programming. You need build tools that can tell you when a given function is no longer called—from anywhere. That is a tall order which none of the previous distributed methodologies has managed to master, but it is more important in a world where you have functions sitting in various public clouds with public-facing IP addresses.

There also needs to be a definitive source code scanner. Those are slowly getting better, but imagine if a library the build system pulls in and uses for 20 or 30 different functional programming microbits becomes compromised? We’ve seen enough instances of ignoring the problem is begging to be a victim, so scanning on build is more important with more systems using external libraries. Or more accurately, with external libraries being pulled into more microservices. There are companies/service providers that will validate your external dependencies and only let you load open source libraries they have certified as safe. I’ve never used one, but they’re worth looking into.

A list of entry points and places they are accessed from is useful for internal-only applications. You can’t do that with a publicly exposed REST API that anyone can call, of course. I’m a developer and a D&D player, so I’ll use that example: Wizards/Hasbro cannot control where this API is being called from. The same is true for any web services APIs that exist to give others access. But internal APIs that are an integral part of internal systems and not called externally? Those you need an automated way to find references to them in internal source and keep a count. Much like garbage collection, you’ll want to take down ones whose count reaches zero.

The thing is that in many ways functional programming reduces operational complexity. It certainly reduces operational overhead. But that complexity is replaced with a different kind of operational complexity; maintenance becomes a bigger issue, and the already occurring breakup of code into smaller chunks increases, bringing cross-reference complexity along with it.

Keep that in mind. You will have orphaned functions, you will have security concerns, you will have changed operational complexity. Plan for these things from day one. They are not at all deal breakers, and the benefits of functional are already clear in certain verticals with more general benefits being discovered as I type. But they’re the new problems that will require answers in this new architecture.

So, build in oversight and tools from the beginning and go see how functional programming could benefit your project. At this time, it isn’t the answer to every problem, but—like most cool new IT things—it is another tool to have at hand when solving problems for the business.

And don’t let your functional programming microservices end up on the Island of Misfit Functional Programming.

— Don Macvittie

May 23, 2018
The Past, Present and Future of DevOps Infrastructure
Infrastructure automation enables organizations to set up operating frameworks that allow smooth functionality with enhanced usage, minimal scope for error and faster deployment of code. The process of infrastructure automation eases development and operations processes by introducing good deployment practices such as computerized testing and versioning.

The need for automation was realized when the demand for deploying error-free applications at a faster rate increased. Additionally, competition from similar businesses forced organizations to look for a way to bridge the gaps between development and operations to deploy applications that would serve their business needs.

The Challenge of Implementing DevOps Practice

The following challenges present a roadblock in bringing out the best results of DevOps practices:
1. The disparate work cultures of the development and operations teams.
2. Non-involvement of DBAs in release cycles.
3. Failure in deployment of error-free codes.
The Emergence of DevOps

The introduction of DevOps practices resolved a lot of problems in terms of increasing business operations efficiency. It created a common operating framework where disparate teams including development and operations teams can work together and deliver applications faster that will, in turn, result in effective business operations and an increase in ROI.

Adopting DevOps involves:
1. DevOps experts outlining of traceable metrics for creating a roadmap by visualizing the desired outcome.
2. Creating a pilot framework that will include the tools of both development and operations teams further integrating these existing tools with the DevOps experts’ robust ecosystem,
3. Analyzing, designing, constructing, automating and implementing the network based on the metrics and resources collated.
DevOps Tools:

The DevOps infrastructure automation process uses several tools that define and drive the success of the implementation process. Here is a short overview of some of the more well-known and widely used tools.
- Jenkins – A Java-based continuous integration (CI) tool contributes to the faster delivery of applications. Jenkins must be aligned with a version control system such as GitHub or SVN. Whenever the code repository receives a new code, a Jenkins server builds and tests the new code and informs the team about results and any changes. Over time, Jenkins has moved from being just a CI tool to being used as an orchestration tool that builds pipelines for application provisioning and deployment.
- Chef – A ruby-based configuration management tool, Chef includes cookbooks that code infrastructure in domain service language (DSL) in combination with a little bit of programming. Chef creates virtual machines and customizes them in accordance with the rules listed in the cookbooks. An agent residing on all the servers that need configuration extracts the cookbooks from the chef master and runs them on the server to get the desired results.
- Puppet – Also, a ruby-based configuration management tool, Puppet was developed with system administrators in mind. A configuration code is written using Puppet DSLs and is wrapped in modules. Puppet performs a similar function to Chef but, unlike Puppet, Chef is a developer-focused tool. It runs a puppet agent on all servers that require configuration and extracts the compiled module from the puppet server and installs required software packages as prescribed by the module.
- Ansible – An agentless configuration management and orchestration tool, Ansible can be used for cloud computing. Ansible has configuration modules called “Playbooks,” which are written in YAML format and are easy to compose in comparison with other configuration management tools.
- Docker – Docker, a process-level virtualization tool, creates isolated environments for applications known as containers. These containers can be transferred to any other system without making any application changes. Docker has evolved as the next step in virtualization. It is beginning to become a favorite of many DevOps practitioners and leading cloud experts.
The Future of DevOps Infrastructure Automation

A lot of organizations are beginning to benefit from the implementation of DevOps infrastructure automation. And with the pace at which the demand for DevOps services is increasing, infrastructure automation has a long way to go.

As per statistics, by bringing DevOps practices into the picture, organizations can benefit by deploying code up to 30X more frequently and reduce the deployment failure to less than 50 percent.

— Veritis
May 22, 2018
Watch Your Language: Behavior Driven Development

There are two basic design methodologies in the software development world: top down and bottom up. In bottom up, the goal is to piece functionality together to “build up” to the desired result. In top down, the goal is to break the problem into smaller and smaller parts until you have implementable pieces.

Both have strengths, and by correlation, both have weaknesses. Analysis and design phases in pre-agile days were almost ubiquitously top down. They would take business requirements, translate them into functional requirements, then break those down and assign the constituent pieces to developers. As the code was implemented, weaknesses in the design would be exposed, and feedback would go back up to the level of functional requirements—or, in really bad cases, to business requirements—and a mini-iteration would take place to resolve the problems.

When agile came along, this was all too slow a process. To be sure, it was (and is) still used and has produced some of the world’s best software, but an agile team can’t sit idle waiting for a long process of analysis. Kan-Ban and similar tools/processes were introduced to address this issue, making the pieces being designed smaller and thus speeding the process somewhat. But smaller pieces implies bottom up to some extent, and the waters got a bit muddied.

Test driven development (TDD) was designed to make sure the code written met the technical requirement by writing a test for the expected result before the actual code. This was one of the most fruitful things to go into/come out of agile development. While it has flaws—the most significant being the human factor—it does make more thought go into code being written and more rigor come out of code produced. The act of writing the test makes the developer think of ways to solve the problem that satisfy the test. But the source of those requirements still needed to come from some form of process.

Enter behavior driven development (BDD). It’s not really development, though the name is catchy. It would be better named behavioral design for TDD, but that’s a mouthful.

In short, the idea is that business owners and analysts speak in terms of the needs of a given role, while developers speak in terms of functionality or the results of a given test set. This creates a gap that causes misunderstandings. The goal of BDD is to get everyone speaking the same language. It tries to extend the spirit of TDD up the design/analysis stack—”shift it left,” in today’s parlance—so that the tests being developed are a hierarchy stretching from acceptance testing down to unit testing, and everyone is speaking the same language.

En vogue today for the language is Gherkin, a simple requirement specification language that writes requirements and test criteria with (relatively) strictly defined sentences. I say “relatively” because, the language is a bit loose from a unit test perspective. But from role-based specification, it is locking down options to make the environment and test cases clear.

I actually got to thinking about all of this while considering SmartBear’s recent acquisition of HipTest. SmartBear has made an investment in making its tools BDD-compatible, and HipTest is largely a BDD platform. The combination of the two should see some movement in the BDD space that will be interesting. I spoke briefly with SmartBear about the acquisition, and it seems the company really is merging the strengths of the two organizations, so it is worth watching over the next few months.

Thus far, the benefits of BDD that jump out are the ability for everyone to talk about an element of the product in common terminology, while the developer can think “down” to the code required to meet the requirement and the business analyst (or owner) can think “up” to determine how to market this cool new feature.

Tools are out there to generate TDD test classes/cases for a variety of languages—notably from my stable, Python and Java. This might be a huge step in the right direction also. Just the other day I was having a bit of a lighthearted conversation with a peer and said, “Get to the point where you can auto-generate tests at build time for us, so IT managers aren’t stuck deciding which lines of code are/aren’t worth maintaining test code for, and then we’ll talk.” Tools and frameworks such as Cucumber and JBehave actually take steps in that direction, though their approaches are different.

We are not currently using BDD on our in-lab test application (Android/web clients, Spring API server) at work, but will have to consider it moving forward as another bit of DevOps tooling and practice we can experiment with. The tools are there, and the process bridges that sticky part of analysis and design where top down meets functional.

After all, the end goal is giving users what they need. If BDD helps make that process smooth and faster, it is a fit for any DevOps environment.

— Don Macvittie

May 15, 2018
Appian World CEO Keynote: Low Code for Sophisticated Applications

Here I am at the Fontainebleau Miami Beach for Appian World 2018, where the theme is “Transform.” The event, which is the biggest one yet with 1,500 registered attendees, is all about low code, and in his keynote address, CEO Matt Calkins emphasized that low code app-building isn’t for unsophisticated, simple apps—we don’t need powerful tools such as Appian for those. Rather, low code is for sophisticated apps to set free the renaissance that will happen with easier app-building.

This is the first Appian World since the company went public last year, and the event has a public-company feel. I didn’t see a lot of people in jeans and t-shirts—most attendees were fairly well-dressed in business-casual, the mark of enterprises.

One of the touch points in Calkins’ keynote was reliance on open standards. Another was to make use of artificial intelligence (AI). I know AI and machine learning (ML) seem to be everywhere. It was interesting the company announced AI integrations with Amazon, Microsoft and Google, but no Apple or IBM Watson. I will ask Calkins about this when I interview him for DevOps TV later today.

Calkins inherently believes that low code will set off a revolution as software continues to disrupt and empower the digital transformation revolution. The logic behind it is simple: “Making applications easier to make because you don’t have to code as much.” But Calkins emphasizes that does not mean you are making unsophisticated applications. The world doesn’t need low-code tools to make unsophisticated applications. It is the higher calling for sophisticated apps that Appian is all about.

Another mantra from the keynote was, “Host anywhere, run anywhere, data anywhere.” The idea here is a true hybrid reality where data can be on-prem or apps hosted on public cloud, on containers or not. In fact, Calkins was very bullish on containers (who isn’t?). Docker integration is already available and he hinted at announcements regarding Kubernetes as well.

Appian seems to have succeeded by becoming the low-code platform. With lots of integration and focus on making application development, management and deployment easier, the company has risen to the top of its market.

Stay tuned for more updates from Appian World today and tomorrow.

— Alan Shimel

April 24, 2018
Catan vs. DevOps Implementation

My family has been playing a fair amount of the board game Settlers of Catan lately and, as happens with me, I got to considering the balancing act of the game versus the balancing act of moving to full-on DevOps. Most orgs have some amount of DevOps running at this point, but most have not made the transition, and I think the similarities have something to do with why. So, let’s consider …

In Catan, you put a settlement at the intersection of three hexagonal tiles. Each tile has a number on it, and those numbers run the quasi-bell curve created by rolling 2d6. There never seems to be a great set of numbers, so you go for the best you can arrange (there are more considerations, such as the type of resources you get for a given hex, but we’ll get to that later). Then you hope that your numbers come up enough to keep you in the game. The game is pretty hot/cold, with the dice seeming to pick a number (regardless of bell curve) and roll it more often in a given game.

The resources part is important. In Catan, each hex represents one of five resources, each of which you need to win the game. But you can trade resources (at a loss) for other resources, so you don’t have to sit on a hex for each type. This leads to some interesting attempts to get around the fact that the dice rolled against you or there were no good resource/number combinations when it was your turn to set up.

The parallels are intriguing, though not perfect. You have resources (people/systems/software) and you have goals (project/product/portfolio) that you are trying to reach. Random things (system failures/hackers/budgeting changes) interrupt you more than is convenient (this last one is equivalent to die rolls in the game, but also another game concept called the robber that I only mention for those who play, since explaining the game would make for a long blog). And you have to be a success.

Choosing to move people to DevOps creates a short-term problem with an eye to long-term gains, since those people moving are both out of their comfort zone and using tools that are new to them. Choosing to dedicate systems to DevOps processes/deployments removes them from the pool of resources available to longstanding systems/processes. It creates a tension of temporarily stretching things so that in the long run you will be better off (much like saving to build a city in Catan when the dice really want you to build roads).

The difference is, of course, obvious. Catan is a game, and if you lose horribly you just reset the board and demand a rematch. You can laugh at the pernicious nature of the dice when they turn on you just as you were about to do something cool. But in DevOps transformation, you just can’t do that. People’s jobs are on the line, and even if you are at a place where job security isn’t an issue, people’s stress levels still will change based on such wild events. If you haven’t experienced this, have your company announce budget cuts that keep you from implementing part of a transformation stage or have your systems compromised that require the attention of staff that were in charge of moving an application portfolio. It isn’t pretty.

The key is to keep the goal in everyone’s mind. The end result of a DevOps transformation should be the ability to get changes done faster with better focus on business needs, more complete testing and better communications. Keep talking this up, as there are always roadblocks to large change in any organization. Those improved communications should help people work together to overcome issues and move on. Even if something such as a data breach brings the effort to a halt for a while, when it resumes, an increased focus on the security of data will only make the transformation stronger.

So figure out how to deploy those resources to the best advantage, keep rolling the dice (so to speak) and keep trying to improve on what the last iteration did. The odyssey will continue unless the org gives up on it; your task is to make the most of each step while you continue to keep those core business systems running.

— Don Macvittie

April 23, 2018
DevOps: Role Clarity is Important

Those who talk a lot about DevOps, particularly those who talk a lot about culture change, sometimes gloss over or “think it is obvious” that some roles change more than others when moving from Dev and Ops to DevOps.

I was having a lighthearted chat with a friend who works for Compuware and realized that, even though several of us have mentioned it before, it bears repeating for those who have not yet dipped their toe into the DevOps pool too deeply.

The driver for DevOps is increased agility, and the belief is that Dev and Ops will merge. What is often left unclear is that they may well merge into one team, but every team member will not become an expert at every role. Pundits seem to think this self-evident. I’m not so certain it is, if you read a lot about DevOps.

By way of example, your Cisco ACI specialist (or ios specialist) will not suddenly become an expert in the intricacies of your development toolset and environment so they can contribute code. Granted, that is an extreme example, but there are two reasons why someone is a specialist in what they do: 1. They know/prefer it, and 2. The company had a need for their specialty. Don’t lose sight of those facts while moving to DevOps.

A better example is DBAs. They’re not going to become network engineers just because they were put on a DevOps team. They’re still going to be DBAs … because you need DBAs. You need network engineers also, but if you need them, you already have them.

Remember that increased communications and increased latitude are the social aspects of DevOps. The DBA is still a DBA, but faster access to queries and those people writing the queries means more optimization of some of the busiest sections of code. If your EMC specialist has access to the entire team, conversations along the lines of, “We’re running low on disk on LUN X, and it will cost us a billion dollars to upgrade, what are our options?” can happen quickly and include views from across the DevOps team (normally across the application or application portfolio).

One of the most thankless jobs in IT is being the single/small shop. Anyone who has been IT for an entire org—particularly one with in-house apps—knows the struggle. My time as a developer hired to work on list serv software but who also inherited the entirety of systems administration (the sys admin quit and changed all the root passwords the day before I started) was not optimal. I learned a ton, which was cool, and I never mastered most of it, which was not.

The goal of DevOps is not to re-create that environment in a team. It is to optimize team interaction, and make things go more smoothly. Make sure you are not on an implementation plan that leads to burnout or lack of accountability. Keep your eye on the real prize: increased responsiveness to business needs.

And as always, keep kicking it. I have yet to walk into an environment that wasn’t unique, or one that didn’t have cool stuff to learn about. And you keep that running.

— Don Macvittie

April 13, 2018
ADDI Addresses the Problem of Dependency Management

Want to know what a bad day looks like? Imagine you’re a developer working your way up the career ladder. One day your manager comes by your cube to tell you that you’ve been assigned to implement a critical, new feature in the company’s premiere application. You’re elated. At last you get to show off your coding mettle.

You spend the next two weeks working day and night to get the feature out. Finally, it’s time for release. You send out a pull request to merge your code into the codebase. Everything should be peachy. You can taste the stardom at hand. The next thing you know, your inbox starts to fill with emails complaining that your new code has broken the build. Turns out that the third-party libraries your code uses are older versions that have long since been deprecated. The code that works fine on your local system is causing havoc in the general codebase. Everything is going haywire. What should have been your hour of glory has turned into an episode of embarrassment and shame.

You Have Entered Dependency Hell

Anybody who has been coding for any amount of time has been in dependency hell. Those of us who programmed in early versions of Windows called it DLL Hell. Java developers have been burned by third-party .jar files that just don’t jibe as code moves down the release cycle. To this day, there is a legion of Python developers out there who are still struggling with the challenges of getting Python 2.7 code to work in newer Python 3 environments. Node developers face ongoing issues transitioning legacy code from ECMA 5 to ECMA 6. Dependency mismatch issues have been a problem throughout the history of software development. Mainframe developers have been blindsided by unknown dependencies since early batch jobs were run under JCL. Dependency mismatches are real and they hurt, sometimes too much.

So, What’s to Be Done?

The software industry has not been asleep at the wheel when it comes to solving the problem of dependency management. In fact, making dependency management easier—degrading it from likes of extreme dental pain to an addressable issue akin to an annoying muscle ache—has been a mostly successful undertaking.

One of the early goals of the .NET framework was to do away with DLL Hell. (DLL Hell is a corruption condition in DLL dependency referencing caused when COM objects declared in the Windows Registry are out of sync with the calling code.) .NET avoids the Registry altogether and references DLLs directly using the file system or the more easy-to-manage Global Assembly Cache (GAC).

Maven, a very popular Apache Project, brings a significant degree of order to dependency management for Java developers. Maven’s Project Object Model (POM) allows developers to declare exactly which version of .jar files their code needs to use. (See Figure 1.)

Figure 1: The Maven Framework for Java development allows projects to declare dependency versions

Babel allows Node developers to move between ECMA 6 and ECMA 5. 2to3 is a program that will convert code written in Python 2 to version 3.

Clearly, most programmers have confronted the issue of dependency management when developing new code. And, the industry has been active trying to lessen the burden of dependency management when introducing new code into an existing codebase. But addressing ways to management code dependency is not confined to PC-based computing; the mainframe community is actively addressing the issue, too.

Which Brings Us to Mainframes

The fact that code quality degrades of over time is not so much due to incompetent programming but rather the demands of continuous revision and the fast fixes put in place to meet the requirements of the moment. Sometimes down-and-dirty code that works today is preferable to more elegant approaches that will take months to implement.

In the mainframe world, there can be decades’ worth of these type of fast fixes in play. Trying to improve code can be a roll of the dice. Usually what gets broken is a dependency well beyond the immediate scope of programming activity. As mentioned earlier, dependency management has always been difficult. That’s the bad news. The good news is that the mainframe community is keenly aware of the problem dependency management and has done something about it. IBM’s solution, System Z’s Application Discovery and Delivery Intelligence (ADDI), is particularly adept at addressing the issue.

ADDI is a big product with a lot of features. One of the more interesting features is the Analysis Browser, which allows a mainframe developer to get a detailed understanding of all the dependencies in a particular application.

Figure 2: IBM Z ADDI Analysis Browser allows developers to visually understand code dependencies and program accordingly.

Having a clear, visual picture of all the dependencies in mainframe application is a very big deal. Mainframe code is by nature onerous. There are a lot of parts that are hidden due to the immensity of the technology. Using human intelligence to try to figure out all the dependencies in force takes too much time. Hence, a tool such as Analysis Browser is indispensable when it comes to addressing the age-old problem of dependency management.

We’re All in This Together

In the past, identifying dependency problems usually resulted in someone getting woken at 3 a.m. to fix a mission-critical problem. Using a proactive tool such as ADDI’s Analysis Browser allows more people to get a lot more sleep. Comprehensive dependency management does not need to be the result of ongoing catastrophes in production; rather, quite the opposite.

The first step to reducing the risks and calamities caused by dependency misalignments in code is to make a commitment to address the problem. The world of software development is littered with teams who just want to get through the current release and are more than happy to repeat the insanity dependency hell all over again in the next one. The excuses vary from, “We don’t have the time to do a solution,” to, “It may not be the best way to do it, but I’ll always have a job.”

Once a commitment is made, then it’s a matter for tools and process. You use the tools to determine risky dependencies. You follow process to improve the release.

Whether you’re in a distributed shop that embraces ephemeral, on-demand provisioning or a more mature mainframe shop working with Big Iron, the fact is that reliable dependency management is critical for the well-being of the enterprise. Yes, while it’s true that the details of coding and code management will vary from an IBM Z to an AWS cloud instance, the underlying thinking that brings good dependency management together is strikingly similar. As just mentioned, it’s a matter of tools and processes.

In terms of implementing ways of dependency management that work, we might be speaking different languages. But, when you take a close look at the basic concepts behind effective dependency management, you’ll find we’re all saying the same thing.

— Bob Reselman

April 12, 2018