devopsy.kinsta.cloud

Author: Contributor

Stop whining about shadow IT and do something about it
Much has been written about shadow IT at the business unit and employee level where the proliferation of cloud applications has all but eliminated IT’s control over what applications are used in their organization. The explosion of public cloud technologies has caused an identical revolt amongst development teams who no longer want to—or need to—wait for IT to spin up the IT infrastructure needed to fuel their development efforts. The reality is public cloud has drastically changed the expectations of developers – even if they aren’t using it.

Gone are the days where development teams are satisfied with putting in a help desk request for a new server and waiting to access it. Developers now expect IT to provide services similar to what they can get from a public cloud provider. And if you can’t provide it? Then you risk them pulling out a credit card and bypassing everything you’ve built. But just as shadow IT can wreak havoc on data security and IT budgets, there’s a whole new set of problems that can arise when shadow IT hits the infrastructure level. So now that the developers’ expectations for immediate resources have been changed irrevocably by public cloud, what can IT do to bridge the gap?

Calling a truce: provide self-service provisioning and management

First and foremost, the obvious one—give developers the immediate access to IT resources that they need to do their jobs. IT needs to leverage its existing investment in virtualization technologies by providing self-service provisioning and virtual machine management capabilities to developers. In other words, get out of the way and give them what they want.

At the minimum, IT should be looking to setup an internal cloud management system which provides IT an on-premises solution with:
- Integration to existing authentication/authorization systems (e.g. Active Directory)
- Quotas to control usage by employee/group
- Chargeback/showback to provide usage tracking and accounting
Giving back control is the best way to ease what can be a tenuous relationship between IT and developers. Developers will be happier because they will be provisioning new servers in minutes rather than days and IT can focus on more interesting challenges.

Abstract away the complexity

Self-service provisioning is only step one. IT needs to get out of the business of provisioning servers and let developers manage their virtual machines themselves. However, that doesn’t mean that IT should leave developers to manage every aspect of their cloud usage on their own. On the contrary, IT’s job should be to create the tools and infrastructure to ensure best practices within your organization and track IT infrastructure resources to avoid unnecessary infrastructure costs.

But how? The first step is using a centralized automation and orchestration technology. I won’t get into the religious battle between the various flavors—Puppet, Chef and Ansible are the three most common that we see and they all have their strengths and weaknesses. However, they all provide the type of framework IT needs to empower self-service provisioning capabilities.

IT should own the base OS images which developers can use for their virtual machines and use an automation framework to provide the means so that developers can easily set up a common set of framework applications (e.g. web servers, databases, monitoring tools, etc.).

IT can also use cloud management tools to effectively track resources. We’ve seen many instances where customers have discovered numerous unused VMs. By tracking resources and enforcing expiration dates on certain resources, IT can deliver infrastructure more cost effectively.

Give users choice

Frankly not all clouds are created equal. And certainly not for all tasks. An internal VMWare virtualization may provide a great cloud—once IT has provided the services mentioned above—for most developer’s needs—but don’t stop there. For example, public clouds, like AWS, Azure, Google, have servers located all around the world. That makes them ideal for web applications which are going to have geographically dispersed users. Most IT shops certainly shouldn’t be trying to duplicate that capability or prevent its usage. Embrace it instead, where appropriate!

That doesn’t mean that IT shouldn’t have visibility and control into that usage. IT should strive to create a system which supports a collection of private and public clouds, has unified chargeback/showback and quotas across ALL of the private and public clouds that IT wants to support.

Abstract Away the Complexity to Support Teams Beyond Developers

These best practices shouldn’t be limited to developers! One of my favorite use cases that I have seen was an IT shop which provided their sales team with the capability to spin up VMs for product demos. They encapsulated the complexity of 10+ product variations and multiple operating system varieties into a single dialog box. When the sales manager wanted to do a demo, they checked a couple of boxes in a dialog box, hit submit and a VM was spun up on their internal VMWare cloud which automatically expired 8 hours later. This was a perfect example of allowing a non-technical user to control of their own virtual machines while IT maintained complete control of the underlying technologies.

Maintain flexibility!

One final word of caution: the world of private and hybrid cloud is changing very rapidly. IT needs to be able to utilize their existing virtualization infrastructure, typically VMware, today while exploring new private cloud options like OpenStack and preparing for the future of containerization and micro-services. It’s important to avoid solutions which are locked into a particular technology—which may leave IT behind in this rapidly changing environment.

There is no doubt that public cloud has led developers to have drastically increased expectations for private IT. By putting in place a cloud management platform backed by DevOps server automation, IT can meet these expectations and stop the growth of shadow IT at the infrastructure level.

About the Author/Jon Mittelhauser

Jon Mittelhauser is the CEO of CloudBolt Software. A 20-year Silicon Valley veteran, Jon Mittelhauser has a proven track record of building highly regarded technology organizations from the ground up and helping to architect pioneering technologies. He is considered one of the founding fathers of the World Wide Web. He was co-author of the first widely-used Web browser, NCSA Mosaic, and was a founding engineer of Netscape Communications Corporation. Mittelhauser was an early investor in Tesla Motors and one of the first Tesla Roadster owner.
July 1, 2015
$The Simple Math of DevOps$

The Simple Math of DevOps
We are living in an information- infused society, much dependent on various technological innovations taking place all around us. In such conditions, as consumers become more connected, they are empowered with more information and choices. They expect quick and consistent information available at all times. Likewise, our business leaders expect their IT teams to build relevant applications and stay ahead of their customers’ needs with an excellent customer experience.

My IBM colleagues and I have been working very closely with major enterprise clients as they begin their DevOps transformational efforts. We’ve found few key points and messages that resonate very well with decision makers who need to set a new course for their teams in the wake of impact from mobile, big data, analytics, cloud, and the Internet of Things technologies.

So what is it about DevOps that is so appealing? First and foremost is the notion that DevOps is all about attaining Speed to Value. Speed is king right now because so many of the business leaders we work with are under the gun to deliver value as quickly as possible. Yet, because of the high cost of technical debt they have come to a state where speed has to be sustainable. It’s no longer easiest to go outside their organizations for the high priority key projects as mobile applications were done recently. They need a holistic approach that builds on their existing capabilities.

Well what does that have to do with math? Here’s the thing, everyone is trying to minimize the time it takes to deliver value and attain feedback.   We can think of the Time to delivery as the following equation.

T_DELIVERY = T_PLAN + T_DESIGN+ T_DEVELOP + T_BUILD + T_DEPLOY + T_TEST + T_FIX+ T_RELEASE + T_EVALUATE

In an optimal world or software factory that is fully automated the T_DELIVERY is optimized by minimizing the time for each of the tasks required to complete the delivery. Most organizations follow a SDLC that is based on this simple math. They estimate the time to do each part and then the total project plan is the sum of the parts. Further, they tend to define work in relatively large chunks or big-up-front style of work definition which inherently increases the time for each task (T_X). As a result, the industry has about a 50% (or worse) failure rate when it comes to completing a project within the estimated time. That causes a lot of hardship because the business can’t count on IT to deliver value to customers in the projected time frames let alone with speed.

Because application delivery is a set of very human oriented tasks, a key factor that will determine the speed to value is TRUST. It’s pretty obvious when you draw out the value stream mapping of how work gets done at a given customer site. As members of software delivery teams lose trust in the validity of the work as it flows through the lifecycle then a large amount of rework and waste is introduced. In mathematical terms our equation becomes:

That is, the Tasks we do in a delivery cycle are impacted by the degree of trust we have in the hand-offs from one to another. If we have zero trust then our T_DELIVERY will be infinite (divide by zero). 100% trust and our T_DELIVERY will be only limited by how fast each task can be performed.

Check this scenario that represents what we’re hearing from our clients in the workshops:

“On a given project team the plan has about a 50/50 chance of being right, and the design is about 85% right, and the developers will get about 90% of the implementation right, and the testers have about 90% of the test cases right, and the release team has about 95% reliability of having the right stuff together to release, then your trust factor is something like

.5 x .85 x .9 x .9 x .95 = 32% Trust”   (and that’s much better than what we usually hear)

Ouch! That’s a 3x multiplier of the time it’s going to take no matter how fast we do all the delivery tasks.   In other words, we’re always limited by our trust factor. That multiplier comes into play with additional tasks we have to add into the equation to counterbalance the lack of trust. Then our equation becomes something like this:

T_DELIVERY = T_PLAN + T_RESCOPE + T_DESIGN+ T_{ARCH REVIEW} + T_DEVELOP + T_{TECH DEBT} + T_BUILD + T_REBUILD + T_DEPLOY + T_REWORK + T_TEST + T_RETEST + T_FIX+ T_REFIX + T_RELEASE + T_ROLLBACK + T_RE-RELEASE + T_EVALUATE

And that’s the issue our customers are facing today. There are a lot of wasteful tasks, checks, and balances that have crept into our delivery process due to lack of trust.

So, how does DevOps help this problem? It’s simple really; the DevOps approach is to attack the TRUST issue head on while simultaneously reduce the task time (T_x) in order to shorten the overall time to delivery. For example, if we apply a DevOps practice of breaking work up into small chunks, getting it out to users early, and getting the feedback, we can immediately impact the overall T_DELIVERY.That’s because we’re positively impacting the numerator and denominator with that practice. That is, by limiting the scope we make it easier to understand, less effort to transform to working code, test, and deploy, sooner to obtain feedback, and more likely to know if we’re on the right track.

Likewise for other practices!

The key is finding the balance of what practices to apply to maximize this equation for a given organization and their current bottlenecks & wastes.

DevOps practices address with three main impact points:

Speed to Value depends upon TRUST

TRUST is enabled by:
1. Clarity: A clear definition of desired outcome(s)
2. Collaboration: Team-wide communication and visibility
3. Consistency: Systematic and repeatable steps
When a team can master these three elements the time to perform each task will shrink and degree of TRUST will increase. DevOps practices impact these trust building elements:

Clarity:
- Break work up into smaller chunks and iterate
- Seek a minimum viable product (MVP)
- Define clear outcomes, ‘sponsor’ users, and playback (Design Thinking)
- Reduce dependencies with loosely coupled architectures (containers, micro services, SOA)
Collaboration:
- Form cross functional teams (end to end)
- Use big visual radiators
- Have a ‘single source of truth’ for all development assets – Requirements, Code, Deployable Assets, Infrastructure as Code, etc
- Be transparent with metrics
- Plan and reprioritize frequently
- Actively seek user feedback
Consistency
- Automate, automate, automate
- Continually improve by replacing manual tasks with automation
Speed to Value will be enabled by eliminating those bottlenecks and waste that have developed due to lack of trust. The key for organizations is to recognize that the current practices are maxed out and not sustainable when trying to move at higher speed. There’s a perfect storm amassing with big data, analytics, cloud, mobile and IOT that will either drive customers to make a DevOps transformation quickly or be a victim of the storm. Try using this simple math to help organizations understand how to move ahead and how our IBM solutions can be applied to attain speed to value.

This year I have had the great experience to work with our DevOps CTO Sanjeev Sharma, our customer thought leaders such as Carmen DeArdo of Nationwide, and our field teams around the world addressing DevOps transformations. Sanjeev, Carmen and I decided to summarize our experiences in a three part blog. This is the first of that three part series.

To learn more about IBM’s DevOps approach, check out www.ibm.com/devops. Also, watch the replay of a recent webcast on Variable Speed DevOps, I had a chance to present on, with Sanjeev Sharma and Carmen DeArdo.

About the Author/ Lee Reid

Lee Reid, Senior IT Specialist, IBM, is a senior solution architect with focus on practical adoption of software engineering tools, software delivery solutions, application life cycle management, and DevOps. He has over 25 years of experience in software engineering, design, programming, testing, development, tools & methodology, team leadership, and strategic planning.
June 22, 2015
Do Regulatory Auditors Hate Continuous Delivery?

In a world where superior customer engagement and other market imperatives depend on having the right code in production sooner rather than later, continuous software delivery is a competitive necessity

But you can’t get code into production unless you properly test it. And you can’t properly test code without appropriate test data.

And that’s where the regulators come in.

Solving for Personally Identifiable Information
Production data is rife with personally identifiable information (PII) and other compliance-sensitive content. That data is typically well-protected by various security measures and access controls in the production environment itself. But putting that data (or, more precisely, a sample set of that data) in a test environment exposes it to significant security and compliance risks.

For one thing, test/QA engineers aren’t account managers or clinicians. So, from a regulatory perspective, there are certain kinds of data they shouldn’t be allowed to see. For another, test/QA environments are typically not secured in the same way as production. Worse yet, test/QA is often performed by contractors — and may even occur offshore. This creates unacceptable security risks and throws up big red flags for regulatory auditors.

To protect PII while enabling thorough and effective testing of new code, companies use synthetic data and other masking/obfuscation techniques. That’s what regulators demand and expect.

Yet another roadblock?
Unfortunately, most companies generate test data on a very ad hoc basis. At some point in each project, someone realizes they’re going to need some test data — and they set about creating it. But without a global mechanism for determining what needs to be masked and actually performing that masking, the undertaking becomes iffy and time-consuming. Not all the data that needs to be masked gets masked. A couple of developers may independently decide that the best thing to do is write a one-off script. And if the script doesn’t work well, the team may just cross their fingers and opt to forego masking altogether.

Hence the title of this blog. Of course regulators don’t actually hate continuous delivery. Most of them probably don’t even know what it is. But their mission — enforcing the diligent protection of sensitive data — can be a serious obstacle to continuous delivery for companies that don’t get their act together when it comes to test data management.

The solution: Continuous compliant delivery
The only way to reconcile the requirements of regulatory auditors with the need of the business for continuous delivery is to adopt a well-managed enterprise-wide mechanism for test data creation. Such a mechanism accelerates every code promotion workflow across every project, while also providing an auditable means of enforcing PII masking/obfuscation policies. Enterprise test data management also reduces devops costs by eliminating the need to re-invent the data-masking wheel on every project.

So, no, regulators don’t hate continuous delivery. But they could — if companies sacrifice compliance for speed. The best way to avoid that sacrifice is to embrace enterprise-class masking discipline for test data.

About the Author/Aruna Ravichandran

Aruna Ravichandran is vice president of DevOps Solution Marketing and Management at CA Technologies, responsible for cross-portfolio solution marketing for DevOps across all product lines. Before joining CA Technologies, Aruna held several management positions in product marketing and engineering at Juniper Networks and HP. Aruna earned an MBA and master’s degree in Computer Engineering from Santa Clara University and holds a bachelor of science degree in computer science from Bangalore Institute of Technology. Ravichandran is on twitter @aruna13 and on Linked in

June 17, 2015
Follow the Unicorn: Transform your Enterprise Applications

For those of us that work on Enterprise Systems such as the mainframe, the claims that the Unicorns⁽¹⁾ make, can be pretty astounding. Dozens and sometimes hundreds of releases a day from big companies like Netflix ⁽²⁾ and Etsy ⁽³⁾ are hard to comprehend. But look a little closer and you can see that the key difference is in the way changes are released to customers, not so much the total volume of changes. A traditional enterprise application may have thousands of changes made over the course of 3 to 6 months, but all of those changes are released at once. The Unicorns are typically able to release a very small number of changes at a time because they can develop, build, test, and deploy these changes very efficiently. Small batch releases to customers has significant value. It enables you to get feedback very quickly, and it is significantly easier to verify because the change is not mixed in with other changes.

So how can you transform your development environment to deliver small batch of changes? I like to tackle the problem like any other computer science problem, no doubt because of my background. Break the large problem into a set of smaller problems and then solve those smaller, less daunting problems:

Step 0: Start with a Proof of Concept

Pick a Proof of Concept (PoC) to begin with that is important, but not mission critical. The PoC can be used as a learning ground and hopefully as a template for subsequent PoC’s. Make sure you have a team that culturally is willing to transform. Often the hardest issues to tackle have nothing to do with technology, but are cultural.

Step 1: Understand your Application

You need to have a good understanding of what your application consists of. Not only does the source code make up the application, but what source files are used, what the middleware configuration might be, what scripts you may have to build and deploy the application, and what data files (e.g. flat files, databases) that you use are also important. Some of these components may be shared with other applications, so you need to understand those dependencies as well. Once you have a good handle on the application, you can then automate the application build and deployment process so that all members of the team can work efficiently.

Step 2: Understand your test cases

If you can build and deploy the application quickly but it still takes days or weeks to test, then testing prevents you from delivering to your clients. An important shift for testing needs to be from testing the entire application to testing the change. You need to focus on running the right tests to ensure the change is correct and causes no regressions. Tests that do not exercise the code change are irrelevant and can be skipped. For many existing applications, the test suites may consist of hundreds if not thousands of test cases. Many of those test cases may be duplicates or testing areas of the code that are not being changed.

Step 3: Understand your processes

How fast can you get your updates to your customer? If your bug fixes and enhancements have to sit on the shelf for days, weeks or months before the customer gets them, they will not be satisfied. Streamline not only the development processes but the operations processes in your organization.

To know more about the topic with the help of a real life case, watch the replay of a recent webcast which IBM in partnership with devopsy.kinsta.cloud hosted with Canada Mortgage Housing Corporation’s Jeff Blackadar on how breaking down IT silos helped speed up mainframe development for them.

Also, do let me know your thoughts on the blog and on the topic.

¹ the born on the web technology teams

² http ://www.slideshare.net/jedberg/devops-at-netflix-reinvent

³ http ://www.slideshare.net/beamrider9/continuous-deployment-at-etsy-a-tale-of-two-approaches

About the Author/ Mike Fulton

A IBM Distinguished Engineer, CTO Enterprise DevOps at IBM, Mike is a software development professional specializing in compilers, debuggers, profilers and virtual machines. At IBM, Mike works towards understanding and influencing trends and directions in Enterprise IT delivering new technologies into the DevOps Portfolio. He also overlooks integration of the DevOps Portfolio into IBM and Enterprise Software Solutions. Mike is an IBM Master Inventor with several patents to his credit.

He is an avid blogger and regularly writes on his blog site – makingdevelopersliveseasier.wordpress.com

June 10, 2015
Webinar: Build, test and deploy mobile apps with back-end mainframe services securely in a hybrid cloud

You may already have a good understanding of the hybrid cloud and its benefits. But how do you get the value of the last three decades of investment in your backend infrastructure into the hands of your end users faster?

In this webcast we will demonstrate how you can take advantage of the latest IBM technologies to quickly and easily incorporate your backend services into your mobile applications running on IBM Bluemix. We will also highlight the latest capabilities of IBM DevOps Services (IDS) and IBM Bluemix that support the end-to-end mobile development lifecycle.

Join Kristal York, Executive Mobile Strategist, PointSource an IBM Business Partner and Rosalind Radcliffe, Distinguished Engineer, Chief Architect for CLM and DevOps as they take you on an in-depth tour of the IBM DevOps Services and Bluemix, an open-standard, cloud-based platform for building, managing and running mobile apps that connect with your back-end services securely.

Recorded Thursday, June 25, 2015

VIDEO

SLIDES

[seoslides embed_id=”2e765888697b” script_src=”https://devopsy.kinsta.cloud/embed-script/build-test-and-deploy-mobile-apps-with-back-end-mainframe-services-securely-in-a-hybrid-cloud/7999/” overview_src=”https://devopsy.kinsta.cloud/slides/build-test-and-deploy-mobile-apps-with-back-end-mainframe-services-securely-in-a-hybrid-cloud/” site_src=”https://devopsy.kinsta.cloud” site_title=”devopsy.kinsta.cloud” title=”Build, test and deploy mobile apps with back-end mainframe services securely in a hybrid cloud” /]

Your Host:

Alan Shimel, Editor-in-Chief devopsy.kinsta.cloud, An often-cited personality in the security and technology community and a sought-after speaker at industry and government events, Alan has helped build several successful technology companies by combining a strong business background with a deep knowledge of technology.

About Panelists:

Kristal York, Executive Mobile Strategist, PointSource an IBM Business Partner

Kristal joined PointSource in 2012 and has led teams in the financial, retail and insurance verticals. She excels at fostering a collaborative environment and leading others to solve complex business problems. Kristal’s prior experience as a business analyst has helped as a mobile strategist to define and analyze our customer’s mobile moments. She translates those mobile moments into a mobile strategy across the organization’s business processes and systems. Mobile demands an iterative approach so Kristal helps analyze strategic planning, usability, research, as well as consistent updating. She uses her business analytics skills to help define adoption rates and sentiment analysis once the app is launched. Previously, Kristal was at Cartus for 12 years, most recently Senior Business Analyst, Business Analytics and Mobile Product Owner.

Rosalind Radcliffe, Distinguished Engineer, Chief Architect for CLM and DevOps, Member of the Academy of Technology

A Distinguished Engineer within the IBM organization, Rosalind is Chief Architect for CLM and DevOps. She is responsible for driving the DevOps for multi-platform architecture; this includes System z and Power system.

June 2, 2015