devopsy.kinsta.cloud

Author: David Geer

  • How machine learning could head off security quality failure

    How machine learning could head off security quality failure

    With so many players in the mix from operations to developers, from PMs to technical leads, to Q&A, most of their activities are automated. Systems record all their email, Wiki, and chat conversations and other notations about these activities. This gives the devops community the opportunity to learn from that data in order to improve future projects.

    “We have access to a massive amount of data that can in total define what the process is that’s going on and define it in-flight in real-time,” says Aaron Cois, Researcher, CERT Division, Software Engineering Institute, Carnegie Mellon University. Through exploratory research, Cois is learning that by using machine learning tools, enterprises can analyze security activities in the development process based on data from issue trackers and commit logs.

    Valleys in Topic Density Point to Software Development That is Missing the Mark

    Natural Language Processing, a subset of machine learning, uses systems to process text in unstructured data and to perform topic detection, determining whether or not a portion of the text belongs to a given topic category. “In our work, we’ve taken text from issue trackers and used machine learning to classify each issue as security-related or not security-related,” says Cois.

    This has practical applications for detecting the course of a project and steering it in the right direction. If the PM expects the developers to raise the quality of security for an iteration of the software package, he can monitor the chatter among developers about the issues they are adding to the issue tracker or he can monitor the comments on the code.

    “By using machine learning to classify issue tracking or source code commit messages as security-related, we can see a reasonable measure of the amount of work that’s being done to ensure the security of the software system,” says Cois.

    If the PM can determine based on a shortage of security chatter that in actuality not much security work is going on, he can ask why, what is the challenge? “Is the security stuff more difficult? Did we do our due diligence to allow our developers to adequately address security?” asks Cois. Then the PM and the teams can work together to remove any roadblocks before any more scope creep or veering off course occurs.

    Machine Learning for Physical Modeling and Trends Analysis

    If the industry can build statistical models of what a healthy software project looks like as it progresses, then it should be able to determine whether a given project is on track and what risks it may face along the way.

    Issue tracking data, commit logs, wikis, chat server logs, build/deployment logs—every type of data that can lead to a full picture of project activities—would form the basis of these models, according to Cois.

    Through Natural Language Processing, machine learning can help determine from this data what the rates of creation and resolution of issues are and what the rates of commits and overall progress are for the software project.

    Machine learning can help PMs look at how their project’s rates in these areas compare with rates of similar projects, which developed certain problems or turned out in certain ways. If there is a strong correlation between rates of projects that had known outcomes and rates in your project, then you might want to use that information to make corrections early on, in cases where the outcomes of the other projects were undesirable.

    Over time, the models become stronger and their predictive value increases, leading to some strong correlations between current projects and the models collected. With the increasing density of the data available in models as they grow, even the weakest numbers in early models can become stronger, giving the model strong correlative value in areas where strong correlations did not appear at first.

    “If I have collected both activity and outcome data on thousands of projects, and a large number of them are highly similar to the current project, I’ll have much more confidence in my predictions about the potential outcomes of the current project given the behaviors I am observing,” says Cois.

  • Mixcloud Makeover: Faster Development, More Confidence

    Mixcloud Makeover: Faster Development, More Confidence

    “Mixcloud provides audio hosting and distribution for millions of podcasters and DJs,” says Matthew Clayton, CTO & Co-Founder, Mixcloud. Mixcloud’s developers program the site in python (Django) and javascript, and develop their native mobile apps in Java and Objective-C. The Mixcloud team grew its mammoth Django monolith application over a five year period. Mixcloud’s primary server infrastructure supports this application, which makes up most of its code base.

    The Legacy Development Workflow & Goals For The Future

    The Mixcloud team used to code its application on a branch, deploy the branch, check for stability, merge it into the master branch and redeploy it. “We used to branch the code on git, then push this to github so that all the team could see the branches developing in parallel,” says Clayton. To check code stability, the Mixcloud team used to simply watch to see whether the site crashed on a single server. “We originally didn’t have any tests, so this was really the only way we could see whether there was a problem,” says Clayton.

    The old overall development methodology also produced challenges every time Mixcloud hired new developers. “Adding developers who didn’t understand the entire stack massively increased the likelihood of bugs and the risk of downtime,” says Clayton.

    When Mixcloud went looking for a DevOps / continuous delivery approach, it had in mind to increase the quality of the code its developers produced and to increase the team’s confidence in the code they shipped to production while overall having faster development.

    New Workflow, Bumps In The Road

    Today, Mixcloud hosts and reviews its code using github.com. The company uses Slack for internal communications. The team uses getsentry.com for exception handling and Asana for product feature and bug tracking. “We use Codeship for running Linting (python/JS/Templates), JS unit tests, Python tests, Selenium integration tests, and database integration tests,” says Clayton.

    Python/JS/Templates enable code linting by leveraging tools such as pylint, jslint, and Django-extensions to perform static analysis and to lint the appropriate code.

    But the path to DevOps enlightenment and improvements is never ending. It took some time for Clayton and his team to understand how all the working components affect each other as they integrate together. “We still have not included database migrations in our release process. This is still a manual process,” says Clayton. Mixcloud will eventually add this too to its list of automated endeavors.

    Navigating The DevOps / Continuous Delivery Maze

    At the very start of a round trip through the new development process, Mixcloud developers will open up a pull request on github.com and assign it to another team member. This action automatically sets a complete build in motion and deploys a staging infrastructure. “The assigned developer will now review the pull request using github, and also the running code. Once this review process is complete, the branch merges with the master,” says Clayton.

    This will next set off a deployment in production. When the team is ready, they will run a single command to activate the new code for each and every user. “The whole test and deploy process now takes about three minutes. This enables a rapid iteration,” says Clayton.

    Benefits & Surprises

    Two of the biggest benefits for Mixcloud were adding code linting and pull request reviews. “These helped the team become more aware of changes across the entire code base, and eradicated entire classes of bugs for what is essentially a fixed, one-off engineering cost,” says Clayton.

    When compared with the legacy development methodology that could introduce new bugs every time a new developer came on board, the Mixcloud team now only introduces bugs of a magnitude that could create website outages and downtime in the rarest of cases.

    But the major unpredicted upside was how the automation and the reduction in manual testing added to the productivity and development speed. “Surprisingly, there have been massive improvements to interaction speed among team members. We are seeing productivity and developer happiness that are significantly better than before the change,” says Clayton. The ability for the development process to refactor huge amounts of code with confidence has also made great strides forward.

  • Rugged DevOps: From Idea to Automation to Impact

    Rugged DevOps: From Idea to Automation to Impact

    The Idea Behind Rugged DevOps

    josh cormanWhereas DevOps is IT at ludicrous speed, Rugged DevOps is a shift left to find a way to inject security into DevOps from the beginning, says Joshua Corman, CTO, Sonatype and Rugged DevOps founder.

    One way to do that is for Rugged DevOps to establish security parameters and requirements, meet those requirements as the project progresses, and establish and enforce the automated application of new and existing pen tests on software as development teams deliver it to the internal staging environment, in order to confirm software quality.

    Automating Pen Tests? Running the Guantlt Is One Way

    Guantlt is a project and a platform for doing just that: developing tools to hook into pen tests and run them on internally-staged code. The Gauntlt project works to automate security testing using established tools, to disperse standard errors, and to fail code based on test results and software requirements. “The goal is to provide an automated way to attack your software because, what we need in DevOps is to find ways to automatically enforce security quality,” says Aaron Cois, Researcher, CERT Division, Software Engineering Institute, Carnegie Mellon University.

    The industry is developing frameworks such as Gauntlt to support autonomously running pen tests on every newly staged release candidate in order to avoid involving or requiring any manual effort. These frameworks enable an automated system to run unattended tests up to several hundreds of times a day to keep step with as many potential new releases each day. We can’t expect limited human capital to run that many tests in succession. “We want coders to write that automated test once and then go off and do something creative that we haven’t already done,” says Cois.

    All this is necessary to meet security quality standards while maintaining the shortest time to deliver each new release candidate with the confidence that development could release it to production at any time. “We want to fire off Rugged tests against that internal environment. Having that parity environment is what enables the development of more comprehensive and more interesting testing from a security perspective,” says Cois.

    Teams must run pen tests on running code in an environment that mirrors production. That’s the only way to know whether the software will stand up to attack. Teams must be able to add new tests as new threats appear.

    To automate a response to pen test results, coders write conditional ‘if-this-then-that’ rules into Continuous Integration to respond to conditions such that, for example, when any unit test fails, the system fails to build the software. “A lot of organizations start out by saying, well, we’re going to start writing security-focused unit tests and security-focused integration tests, getting our QA department involved and our Integrated Security Department involved early in that process, having them rating those tests and enforcing tests in Continuous Integration,” says Cois.

    Impact: Rugged DevOps in Action

    “Heartbleed and Shellshock were high-profile vulnerabilities that had been present in BASH and Open SSL for a long time and we didn’t know about them. We had to find ways to make sure we weren’t vulnerable to those,” says Cois. Using Rugged DevOps, the security team member can create tests for such new threats and require the tests using CI. As CD quickly fixes bugs, so Rugged DevOps can quickly defend against new threats.

    One hope for Rugged DevOps is that it will bring truly secure coding even to COTS software as DevOps eventually envelopes all software development. DevOps should enable developers to continue to develop COTS software cheaply while adding the inherent security that would prevent a lot of the successful attacks on software that trouble industry today.

    “I don’t know that we will reach that stage quickly since it certainly requires an investment,” says Cois. There first has to be that continued uptake of DevOps approaches and those changes in processes, culture, and mindset. Rugged DevOps has to accompany that or the eventuality of cheap and secure COTS software might never materialize.

    Concluding Remarks

    “People like DevOps because the concrete hasn’t dried yet and people are copying high performers. At the same time, programmers are assembling 90-percent more code from existing open source code,” says Corman. This is good news where those high performers and those who follow in their footsteps fail faster, fix errors sooner, and carefully consider security at every step, as this will magnify secure code. But developers have to build using the higher quality code those high performers are creating, says Corman.

  • Adopt DevOps Practices Organically, But Don’t “DO DevOps” (or is DevOps bullshit?)

    Adopt DevOps Practices Organically, But Don’t “DO DevOps” (or is DevOps bullshit?)

    In a post on his blog , programmer Josh Johnson points out that “radical claims of great cultural shifts, especially in technology” are usually “bullshit”. “Over time technology changes, but true radicalism is rare. Most often, a reinvention or revisiting of past ideas forms the basis for such claims. This DevOps thing was no different,” says the post from Josh Johnson, Software Engineer at Adzerk.

    In 15 years as a technology journalist, I’ve heard vendors use labels and marketing speak to sell solutions. You can’t judge the efficacy of a solution by its label any more than you can “tell a book by its cover.” But is there something to these devops practices that we should take seriously? Should we make the effort to separate the wheat from the chaff, the faster, better software builds from the buzzwords and BS? I think so. And when I read Josh’s post, I came to believe that he thinks so, too. Josh shared his thoughts with me. Here are the fruits of that exchange.

    First, no one should be doing devops. It’s not an action, it’s not a title, it’s a blanket term for approaches that bridge the gap between traditional operations and development groups, says Johnson. Second, having devops teams is wrong. “Sectioning off a group of people and giving them special projects and expecting devops skills doesn’t fix the divide, it creates new divides,” says Johnson. This leaves other developers and ops people, not to mention management, out of that inner circle. “The value of devops is in the cross-functional aspects of bringing people together from different disciplines,” says Johnson. True devops should include everyone.

    A healthy, organic transition to devops practices should encourage people who specialize in operations or development to apply their specialties to the fullest while learning what the other side does. “If I’m the security expert,” says Johnson, “I’m going to monitor CVE feeds and recommend policy and practice. On a cross-functional team, I will also perform code reviews, participate directly as a peer in planning, and help the QA specialist by turning them on to fuzz testing. They’ll teach me how to use Selenium. I’m still the security expert.” In this scenario, the security expert grows and benefits and so does everyone whose work he touches.

    Hurdles

    The company will make compromises. The enterprise is asking a lot of its people and will have to pay to get it. The turnover in human resources can slow change to a crawl. The organization will probably have to outsource at least some tools and talents. “You may have to use GitHub instead of hosting your own git server,” says Johnson.

    The more the devops metamorphosis progresses, the more the organization can lose central control of critical components. All things being equal, the bigger the enterprise, the harder it will be to let go. Initial costs and security concerns when getting started, together with outsourcing, can drive project scope through the roof. And that’s not the worst of it.

    “You might lose some great people. There are some people who really need to focus exclusively on their specialty. There are managers who don’t thrive outside of silos. There are people who will not be able to rise to the occasion and work well with their new teammates,” says Johnson.

    Don’t Force It

    To achieve a thriving instantiation of devops while maintaining your sanity, you have to guide everyone to simultaneously want these practices. Start by poking holes in silos and letting members of different teams peek through. “Take a team of bright, collaborative people, give them a goal (say, reduce downtime in deployments by half) and empower them to make it happen, and it will happen. DevOps will magically spring forth,” says Johnson.

    This organic approach is far afield of simply adding a devops team in hopes that it will save money by reducing staff. “Four DevOps engineers are not directly equivalent to two systems engineers and two software engineers,” says Johnson. If that’s the case, it’s worse yet to believe that you can replace four specialists with three experimental devops positions and maintain quality output at a savings of one FTE. Instead of saving money, this move may increase costs due to compressed time to burn out, expensive errors, and an enveloping drop in moral. “That will blow up in your face,” says Johnson.

  • Sofico Adds DevOps Approaches, Updates CD, Part Two

    Sofico Adds DevOps Approaches, Updates CD, Part Two

    We recently geared up with Sofico’s existing CD process and the challenges it presented together with its search for a devops solution. Now read the conclusion as Sofico earns satisfaction through its own devops approaches.

    Sofico Adopts DevOps Methods / Tools

    Today, Sofico uses the Bamboo CI server to perform builds, deployments, and tests. Bamboo automates builds of the backend server, web interfaces, command line interfaces, and a Windows line. Sofico uses Bamboo to deploy its application builds. Sofico runs tests on software using tools such as Selenium for browser testing and functional tests.

    Sofico runs Dynatrace monitoring together with the functional tests to gather performance data. “This gives us correlations between the functional outcomes, the performance outcomes, and the expected outcomes of the testing,” says Jan Swaelens, Software Architect and Member of Solution Management, Sofico.

    “We actually created a custom Bamboo plug-in for that together with the people at Dynatrace in order to visualize the outputs of that correlation inside Bamboo,” says Swaelens. Sofico also uses JMeter to launch tests on servers and Dynatrace monitors those as well for performance metrics.

    Sofico uses application tools to manage its Agile inner workings and to run reports on that. Sofico uses Stash to manage its code repositories where its developers do feature branching for new features. “Developers use the Dynatrace server to monitor their performance metrics for the features they’re building. It enables proactive monitoring of what they create, even before they commit the change to central testing,” says Swaelens.

    So the developer does his regular testing and he also enables the agents that do the monitoring to identify errors. Sofico also does load testing using increasing loads for performance metrics, and performs tests for various use cases.

    Results

    Using DevOps to expedite builds, tests, and deployments significantly decreases the number of times that something will go into production and need to come back for a fix. “We actually manage to stop software with errors before it gets to our customer. This is very critical because in our sector when, for example, we make a major mistake in billing software and the customer actually has that in their production environment, their customers are impacted,” says Swaelens. This is very expensive for the customer and for Sofico because then they have to fix the issue in as little as a day, which is not always possible.

    The bumps in the road that Sofico experienced were in rolling out devops and adjusting to the new way of working. “We found that the impact on systems and processes was even larger than we were expecting. We restructured our old company at every layer from management down to the developers and testers,” says Swaelens.

    Sofico redesigned every process so everyone could work in devops teams. All of Sofico’s existing methods and processes were analyzed to determine how the company functioned, why it worked that way, and to make sure the end result was still providing the same or better results.

    The changes and improvements that devops made were far-reaching. Rather than making devops come to Sofico, Sofico had to gravitate towards devops, change its business culture, and break down all its silos. “We went all in to get to the next level and do more business globally. Because we’re not just one company, we had to spread devops across all our sub-companies around the world,” says Swaelens.

    Sofico can draw a straight line between adopting devops and the speed with which it can now grow globally to increase customer uptake of its offerings. The software quality and timeliness of delivery are where they should be. Sofico’s people are happier, too.

    Sofico has relocated its teams together into the same space. Consultants, developers, testers, and operations staff are sitting physically together in the same areas. People are working together and talking with each other to derive the best solutions and to improve efficiency. “We actually have cases of people blossoming because they are seeing whole new parts of the business and working in new ways that we could never have imagined,” says Swaelens. Where Sofico’s people once saw only their piece of the development puzzle, they now see the entire process. They know the full value of their work.

  • Linux Containers & Security Implications

    Linux Containers & Security Implications

    Does the word “container” intimate containment, suggesting that containers are inherently secure? If it does, any such assumption of security may be the broadest Docker vulnerability to date.

    aaron cois“One of the biggest threats I see with Docker is its positioning and the implied security in the language. The reality is that these containers don’t contain anything,” says Aaron Cois, Researcher, CERT Division, Software Engineering Institute, Carnegie Mellon University. Yet, that is the implication.

    Just as those who thought Linux or VMs were secure enough on their own were mistaken, so those who believe that containers put a lid on security will be sorely disappointed. Today, Linux environments require network, OS / host security, Internet security, and web application security measures similar to those used with other platforms. Tools like security auditing / PEN testing, firewalls / WAFs, anvi-virus and anti-malware tools, DLP, IDS/IPS, remediation tools, and really the gamut of security measures similar to what Microsoft environments require are increasingly needed to defend data in Linux environments. “Likewise, operations can give developers tools to log into the VSphere console to create and change VMs while limiting their privileges,” says Cois.

    And so, containers also require appropriate security measures. “Developers and non-admin operations staff don’t need to log into the host command line to work, and no one in security wants them to,” says Cois. But today’s Docker workflow not only permits but requires it.

    The Root of the Problem…

     

    –THIS STORY HAS BEEN MOVED TO CONTAINER JOURNAL–

  • Sofico Adds DevOps Approaches, Updates CD

    Sofico Adds DevOps Approaches, Updates CD

    Yes, you can have continuous delivery without DevOps. To understand why you wouldn’t want to, follow the journey of Sofico from its CD of olden days to its CD rooted in DevOps today.

    A Very Brief History of Sofico

    Its founders started Sofico 27-years ago as a small family company. Today Sofico is a multinational organization with approximately 170 employees. Sofico creates software for automotive financing, leasing, and fleet and mobility management. “We create software for companies that manage all aspects of leasing a car to, say, an employee for example, in case you get a car from your employer,” says Jan Swaelens, Software Architect & Member of Solution Management, Sofico.

    Sofico offers its customers websites where they can configure and quote the car they want from their boss. Sofico also creates back office software that processes orders and contracts and supports billing and accounting functions. “We offer a complete solution, which you could compare to a large ERP system like SAP, but specifically for the leasing sector,” says Swaelens.

    Sofico CD, Sans DevOps

    Sofico started out using CD / CI approaches about seven years ago, first migrating its old systems to do CI for software delivery. “We were using really ancient things like batch scripts to do our software builds and Windows timers to schedule them,” says Swaelens. Sofico took a lot of manual steps in deploying software.

    Sofico used Waterfall development methodologies, moving step-by-step through the development process in small chunks. The process was not interactive. “We had separate departments for the analysts, for the designers, with developers working solely with other developers, testers working only with testers, and business consultants working only with other business consultants,” says Swaelens. Sofico moved the work from one silo to the next, with several silos literally isolated by differing geographical locations.

    The Challenges of Manual CD

    This configuration of continuous development presented several challenges. “We had problems getting software done on a reasonable time scale. It took quite a while to get software built in the first place. Deployment was another matter,” says Swaelens. The quality assurance processes were manual, which created issues for large projects that required a lot of changes in short time periods. The CD process led to many functional and performance errors. “We had some serious technical regressions that caused malfunctions in the products,” says Swaelens.

    Staff in one group would return work to its originating group. Development would return work to the consultants who had created bad specifications. Operations would return work to development because it didn’t work right. “The customer bore the cost of these mistakes because his issue or feature either wasn’t done correctly or wasn’t done on time,” says Swaelens.

    These homely CD processes also affected Sofico in its growth into other countries. Sofico had to grow fast to gain momentum and succeed at its new locations. It was difficult for the company to pull all the knowledge necessary to work with those processes together fast enough to keep pace with its large projects to make sure those took off so that its new locations could thrive.

    Sofico In Search Of Something Better

    Automating stuff is one thing; getting people to work together efficiently is something else, according to Swaelens. Sofico was looking for a way to improve how its people worked as an organization. It was important to make customers happy, and to make employees happy. Working in those silos reduced people’s enjoyment of their work because they were stuck in one kind of job doing only one thing. “We hoped that switching to DevOps would also give them more rewarding jobs,” says Swaelens.

    Sofico also wanted to improve software quality by automating an increasing number and types of relevant tests such as testing for software performance. Delivering software functionality was not enough for Sofico. They wanted to guarantee internal and external software product performance to their customer. “In our market, that is very important. It directly impacts their end customer in a very visible way,” says Swaelens.

    Sofico sought faster go-to-market time, too. When under the old processes the customer called Sofico asking for a new feature set, Sofico had to reach out to each and every silo more than once with communications back and forth whenever there was not sufficient clarity about project requirements. “The advantage of what we were trying to achieve was that it would reduce the time required when crossing those silos just to get the right answer and to secure the prospect of project delivery,” says Swaelens.

    To learn what Sofico did to achieve DevOps nirvana, come back and read part two of “Sofico Adds DevOps Approaches, Updates CD”.

  • IBM InterConnects DevOps & Systems of Record/Systems of Engagement

    IBM InterConnects DevOps & Systems of Record/Systems of Engagement

    There are many planes and points-of-view from which to examine how DevOps changes development. One way to look at it is to cut an application down the (more or less) middle, dividing it between its system of engagement and its system of record. Briefly, the system of engagement interfaces with the customer while the system of record engages with the backend systems that contain the data stores that the application leverages for the benefit of the customer. These two systems must interoperate smoothly and intuitively with each other internally, and with the systems and people they interact with external to the app.

    DevOps affects the entire workflow as development, testing, and operations work together to better build their apps. “This is a big part of what IBM InterConnect  is about,” says Mike Fulton, IBM DevOps for Enterprise Systems CTO, and blogger, https://makingdeveloperslivesbetter.wordpress.com/. DevOps really defines how enterprises pull systems of record and systems of engagement together more cohesively while enabling people from different components of the development-operations / CD process to communicate to and learn from each other more quickly and efficiently, speeding the arrival of deployable software. DevOps connects workflows into this learning process even as workflows enable efficiency improvements. IBM tools surrounding development workflows participate in that process.

    IBM Tools

    “IBM provides a series of tools to help you move through each unique state of the development lifecycle,” says Fulton. IBM’s Rational Team Concert (RTC, http://www-03.ibm.com/software/products/en/rtc) is an ALM targeted to DevOps team work for teams of several developers, testers, and operations professionals working with application building workflows. IBM’s UrbanCode Deploy (UCD, http://www-03.ibm.com/software/products/en/ucdep) helps teams deploy apps onto different systems and to a variety of test tools.

    “You can build pieces in IBM Worklight Studio to use for building a system of engagement,” says Fulton. IBM tools make it easier for teams taking a DevOps approach to create code more efficiently and precisely, reaching a higher standard of quality, while getting it out to the customer for quicker feedback sooner so the teams can figure out how they need to further innovate the app for the better.

    “Whether you’re using IBM’s tools or anyone’s, the notion of having well-defined processes for building apps is important,” says Fulton. IBM’s RTC can track workflow for the pieces going through the system to get developers to work on the code and to check the workflow so that the system hands code over to testers while making sure there are no regressions. This intertwines with the build process in UCD so builds start seamlessly with the most appropriate timing.

    When DevOps teams have good, clear communications and processes for how to build an app, they can optimize for speed while making the process and the app better. This is a result of learning while using workflow tools. “If you don’t have these processes in place, you will struggle with how to build apps quickly and efficiently,” says Fulton.

    Forward-Looking Questions

    IBM InterConnect attendees looking forward to Mike Fulton’s talks should ask themselves the following questions in preparation to absorb and thoroughly benefit from these sessions. “One of the talks on enterprise devops focuses on the systems of record side. One of the questions you should ask yourself is, ‘how can I improve my process so that I can become more agile and be able to respond more quickly to customers’ demands?’” says Fulton. And that is potentially a multitude of questions since the answers will vary widely from customer to customer.

    “Come into these sessions and IBM InterConnect as a whole ready to talk about specifics. Come with the thought of where and how you think you can improve the entire process you use to build your apps at your business,” says Fulton. Attendees can discuss their answers and more questions with Fulton and with each other, talk about their experiences as these relate to others experience with IBM UrbanCode and with other tools.

    “One key is to have an understanding of what you have in your application development environment, where it is today, in order to know where you want to go with it,” says Fulton.

    Get ready to enjoy a rewarding IBM InterConnect.

  • Reaching Millennials with new Mainframes and agile technology

    Reaching Millennials with new Mainframes and agile technology

    “I’ve been with IBM for 25-years and been with the mainframe. We hear either that the mainframe is dead or that it will be retired in one or two years,” says Mike Fulton, IBM DevOps for Enterprise Systems CTO, and blogger at https://makingdeveloperslivesbetter.wordpress.com/. But IBM and Fulton have built bridges from the largely new and improved mainframe to the new IT professional, making access easier and more inviting.

    IBM has made it easier for Universities teaching about the mainframe to build and use very high quality education programs in order to educate students about the mainframe and about application development for the mainframe.

    “One of the neat things University folks have done is to create free MOOCs so that students and others globally can learn the mainframe and log on to a real mainframe from the Internet to develop their skills,” says Fulton.

    Reaching the Millennials

    In Fulton’s session about reaching Millennials with the mainframe, he starts with the realization that they address the mainframe differently than he would. Where everything used to be a green screen on a dumb terminal, now it is web based.

    “I see people facing challenges in how to adapt to the workforce of people coming in who have this different approach to work and life. The key thing is to focus on the people expertise and on training the millennials on the business side and the technology side related to apps and not so much on the kinds of tools used to access the mainframe,” says Fulton.

    People whose work and lifestyles are mobile with remote working can work from the Internet and access the same applications on the mainframe to accomplish the same business tasks. DevOps has led to improvements allowing people to move on to these new tools including web tools for systems of record and web tools for systems of engagement.

    IBM is reaching out to the millennials. “We partner with Universities for this,” says Fulton. Customers with mainframes and massive apps should look beyond next quarter and into the future to how to train the new batch of people taking over on how to use these web tools to accomplish mainframe tasks.

    Security

    Securing the data is one of the big challenges for Millennials and anyone working with information systems today. People can engage Millennials with how secure the mainframe is. “Fundamentally if you are building any app of interest today, security has to be at the top of the list and it is important if you are a millennial or anyone else because you can easily destroy your business with one security slip up,” says Fulton.

    IBM bakes security into the mainframe so it is natively secure without additional security configurations. “Additional configurations give you the opportunity to make mistakes,” says Fulton. But an app designed to work on Java using sockets on the z/OS will automatically be locked down and secure because it will use TLS, which IBM uses by default, explains Fulton. The customer doesn’t have to code all this manually and IBM knows that it is fundamental regardless of what technology you use that you must be aware of security vulnerabilities in order to address them, says Fulton.

    The mainframe is an obvious choice for security because IBM has been baking security into it for 50-years now, says Fulton. “It’s easier to guarantee security on the mainframe where it is native than it is on systems where people are still adding security in after the fact,” exclaims Fulton.

    The mainframe and its security are based on a proprietary, closed-system approach. You would have to be one of the few people who know even one of the internal subsystems, then you would have to be able to get into the mainframe and use that knowledge to get from one system to another, which requires another type of proprietary expertise.

    The System z memory architecture keeps any one application from gaining any access to the data or processes of any other application. IBM’s user authentication and authorization software also keeps individual applications from accessing other areas of memory. The mainframe further offers more encryption than other platforms. When it comes to security, the mainframe has a strong feature set that makes it attractive.

  • IBM InterConnects Intuitive Themes to DevOps Challenges

    IBM InterConnects Intuitive Themes to DevOps Challenges

    IBM DevOps Evangelist, Eric Minick will leverage his slots at IBM InterConnect 2015 to help you surmount continuous delivery and automated deployment DevOps challenges, three of which include: balancing a continuous delivery pipeline; deployments in hybrid cloud scenarios; and, fitting in trickier deployments.

    devopsy.kinsta.cloud spoke with Eric about these challenges and the central themes that will bubble to the top as he speaks to each one. If you attend Eric’s sessions, see if you can connect the dots between these challenges and themes.

    The CD Pain Point Domino Effect, Cloud Deployment Conundrums, and Deployment Complexities

    The first challenge Eric will address is balancing the continuous delivery pipeline. When maturing that pipeline, does development, IT ops, and the business really fix continuous delivery pain points or simply redistribute them? With every attempt to fix a weak link in the continuous delivery chain, whether that link is builds, tests, or deployments, each newly strengthened point in the pipeline reveals another that is resultantly more vulnerable.

    “If we have slow manual builds, for example, and we fix that with a continuous integration tool, I guarantee you that you will then have a problem where your deployment process is pained because you’ve given yourself a lot more stuff to deploy,” says Eric Minick, IBM DevOps Evangelist.

    The second challenge is deployments in hybrid cloud scenarios. As much as DevOps benefits from the cloud, the cloud can also add complexity to deployments. In hybrid cloud scenarios, the organization must ask, if we’re updating infrastructure in our deployments, how does that change when we’re in multiple clouds?

    “What if some of the stuff we want to put into a test environment is traditional apps that we would normally build in-house and some of it is platform-as-a-service bits that need to change?” asks Minick.

    Finally, it can be difficult to deploy trickier elements such as database bits together with more straightforward aspects such as static content even though the organization has tested and passed them in the same pre-production, staging environment.

    Recurring Themes

    There are two central themes that run through these discussions. The first is that organizations should deploy or promote components to production at the same level of granularity that they test them at. There are benefits to deploying these elements together. The detriments of promoting them separately include broken software.

    If the organization is testing a set of interconnected web applications that have dependencies from one application to the next and they all pass, then the enterprise should promote all that to production because they tested it together and it functions well together.

    “In the same way, if you’re testing a single sub-system and it passes, you should promote that into a production environment just like the one you tested it in,” says Minick.

    If the development team deploys one of several components that work well together into an older production environment that it did not test that piece of software in, it will likely produce errors at the very least.

    The second theme is that organizations can align their decisions about what to automate and what not to automate based on the recognition that the strength of people is creative problem solving while the strength of computers is that they follow directions well.

    “So anything that requires fancy decision-making is probably best left to people to handle where as anything that is simply a list of instructions to repeat is probably something the organization should automate,” says Minick.

    Each of these themes carries particular benefits for DevOps. The benefit of deploying and promoting components together at the same level of granularity that you test them at is that you avoid the risk of something breaking in production because that comprises an untested environment for that component or sub-system. So if the organization tested a grouping of components and they worked well together, the enterprise should deploy them all together.

    “In such cases, deploying separately results in errors and downtime and all sorts of ugliness and bad behavior. To avoid that, you would have to try to track the relationships between every code change in every sub-system and how these might impact each other,” says Minick, “which is a hugely expensive and very painful thing to do.” That’s why it makes sense to test things together that are going to production together.

    The benefit of the organization separating what it should automate from what it shouldn’t is that anything the company can hand over to computers, those computers will do it more quickly and cheaply and on more machines simultaneously. “Handing over tedious tasks to machines keeps the developer’s job more interesting too,” says Minick. And automation means the enterprise’s capacity to deliver changes to the marketplace increases.