Author: Scott Ferguson

  • SoulCycle Flexes Its Muscles with DevOps and CI/CD

    SoulCycle Flexes Its Muscles with DevOps and CI/CD

    When SoulCycle wanted to improve application development and deployment, including improving services for customers across its 80-plus indoor cycling centers in the United States and Canada, the company turned to Mark Sost, a veteran developer and engineer who worked at several media companies, including Hearst.

    Over the last two-plus years, Sost, who now holds the title of senior director of engineering at SoulCycle, has built-up the company’s engineering and DevOps teams, increasing the headcount from eight or nine developers to about 40 employees now.

    The first major project was revamping a legacy, LAMP-based stack used to support applications to focus more on a modern platform that could take advantage of DevOps methodologies, as well as continuous integration and continuous delivery practices (CI/CD).

    Sost and his team dubbed this new platform Podium. However, Podium is more than a development platform, Sost explained.

    “Podium became the ethos and the methodology, as well as the architecture and application stack for re-platforming Soul’s technology stack in total,” Sost said in a telephone interview. “We moved from that monolithic to more of a service-oriented architecture. As such, we started to break out functionality on multiple applications.”

    The re-engineering also meant investing in new infrastructure, as well as other cloud-based technologies. This includes a multi-cloud approach.

    While most of the legacy applications have now been moved to Amazon Web Services, SoulCycle also uses Google Cloud Platform for Podium and all new developments. Sost also takes advantage of Kubernetes and Docker for container orchestrations, as well as investments in other managed services.

    Other tools in the DevOps arsenal include Jenkins to run unit tests; DataDog and StackDriver for monitoring and analytics for workflows; combing Travis CI, Cloud Builder and GCR for CI; and technology from startup Harness, which provides CD-as-a-service. Sost called Harness the “last-mile” tool needed for application deployment.

    “For Harness, that’s CD for us,” Sost said. “That’s continuous delivery to our various Podium environments. We’re leveraging it for delivery to all our Podium environments. Everywhere from UAC (user account control) during the code review cycle to QA after merge, to production, load test and everywhere in-between.”

    This revamped approach to application development now means that the SoulCycle engineering team has created approximately 30 different apps over the last 24 months, this includes internal apps used within the company environment, and ultimately to customers using the facilities and equipment.

    Additionally, Podium helps create a more unified architecture to help ensure that the entire team—whether it’s DevOps, QA or the production team—can paint a 360-degree view of the problem that needs to be solved and what the implications are for the business once the solution is put into place.

    This is what Sost and his team call “Podium principles.”

    “When we say that we’re building things with Podium principles in mind, we’re talking about the fact that everything is version-controlled,” Sost said. “We have full test coverage. We have everything in terms of Kubernetes manifests to infrastructure automation written down and version control checks it as part of source code. That applications are conforming to more of that 12-factor methodology and that we’re really focused really heavily on developer empowerment.

    “I think for us, that really is a shift that Podium has introduced; a move from writing the code as an application developer and then throwing it over a wall to an operations person—a QA person—to test it and deliver it somewhere,” Sost added. “Instead, we provide a developer the tools and the patterns of principles to go about controlling more of their application development life cycle all the way through to delivery.”

    The results have helped improve the overall SoulCycle experience for customers, which includes:

    • Reducing deployment time by approximately 80 percent.
    • Instead of one hour, deployments now take one developer 10 to 15 minutes.
    • QA verification has been reduced by approximately 80 percent.
    • It now takes three QA engineers one hour to verify, compared to three to six hours previously.

    At the same time, Sost believes that the Podium approach gives developers more control over their projects and lets them focus on writing good code that creates apps people want to use.

    “It’s more, ‘Now, I know what’s involved with getting my application to people,’” Sost said. “Because at the end of the day, engineers are just concerned with writing things that people want to use. … This is how we’re going to give you more control of the code and applications that you’re writing, so that you can see meaningful results quicker, without having to wait for someone.”

    What’s next for Sost and his team at SoulCycle? The answer: Machine learning and a bit of automation to the DevOps process.

    Here, SoulCycle is using Harness tools to log more data and infrastructure monitoring into the platform, which allows the algorithm to learn more about the various development projects. This can make the code better and eliminate errors that come with constant development and deployment.

    “We’re letting it get smarter over time around how it can detect anomalies and understand whether our code was successful because there are going to be times where things slip through unit testing or slip through integration tests and there are even going to be times where they slip through load tests, where things like 99 percent are just not going to cut it,” Sost said.

    — Scott Ferguson

  • Python Benefiting From Increase in DevOps Use

    Python Benefiting From Increase in DevOps Use

    DevOps and Python have developed a mutually beneficial relationship.

    In a recent survey released by the Python Software Foundation and software development company JetBrains, researchers found that more and more developers are using the program language for an array of versatile projects, including DevOps and machine learning.

    In fact, Python developers told the researchers that DevOps is increasingly part of their professional development work.

    “In 2018 we had significantly more respondents specifying they’re involved in DevOps (an increase of 8 percent compared to 2017). In terms of Python users using Python as their secondary language, DevOps has overtaken web development,” according to the “2018 Python Developers study,” which is based on responses from 18,000 developers in 150 countries.

    Over the last two years, Python has grown more important to developers thanks to its flexibility, especially when it comes to web development. It’s no surprise that as other projects have gained popularity in the enterprise—DevOps and machine learning—developers have started adopting the language to other disciplines, especially as businesses invest more in application development.

    In TIOBE’s February 2019 index, Python now ranks third behind favorites such as Java and C, edging out C++ for its current position.

    Digging into the numbers, the survey finds that 43 percent of developers are using Python for DevOps projects, as well as system administration and writing automation scripts. That’s an increase from the 35 percent who reported the same types of uses for the programming language in 2017.

    The report notes that these numbers reflect a combination of Python used as either the primary or secondary language for various projects. As the report notes: “DevOps/system administration/writing automation scripts has moved into first place among Python users using it as a secondary language.”

    Machine learning is another up-and-coming category, with 38 percent of developers using Python for these projects compared to 31 percent last year.

    Overall, data analysis remains the No. 1 use case for Python developers, with 58 percent reporting that it’s their main focus when it comes to development.

    In second place is web development, according to the report.

    In terms of the top cloud platforms for Python developers, Amazon Web Service is the top choice, with 36 percent of respondents reporting that they use AWS. From there, it’s Google Cloud Platform (29 percent) Heroku (26 percent), DigitalOcean (23 percent) and Microsoft Azure (16 percent).

    About a third of those surveyed responded that they don’t use any cloud platforms.

    When asked about running code in a cloud production environment, 47 percent responded that they use virtual machines, although 40 percent of Python developers reported that they prefer containers. Another 28 percent use platform as a service (PaaS) and 21 percent are taking the leap to serverless computing.

    Additionally, when developing for the cloud, about 35 percent of those in the survey report that they use Docker containers.

    This increase use of Python within DevOps, as well as other connected areas such as serverless computing, does not surprise Anshu Agarwal, the CEO and co-founder of Nimbella, a cloud-agnostic, serverless cloud platform for developers. She noted the language’s use within a wide range of projects, specifically machine learning and AI, are driving this change.

    “Python developers are driving serverless requirements in new and unique ways, primarily because of the interest in data analysis, ML and AI,” Agarwal wrote in an email. “These data analysis and ML/AI-centric serverless use cases are different than request/response style serverless functions which run for much shorter durations. Python functions in serverless may have much longer durations (10s of minutes or hours), and consume a lot more memory. For such functions a different kind of serverless framework needs to be adopted that can handle long running workloads.”

    When it comes to continuous integration systems, about a third of respondents don’t use any. However, when they do, Jenkins is the most popular at 25 percent, followed by Gitlab CI and Travis CI, with 18 percent each. For configuration management, 20 percent reported using Ansible, while 9 percent used custom tools. Puppet, Salt and Chef also appeared in the list.

    Finally, about two-thirds of Python developers (69 percent) use Linux as their main operating system of choice for development. Microsoft Windows comes in a distant second at 47 percent and Apple’s macOS in third at 32 percent, according to the study.

    — Scott Ferguson

  • ‘DevOps Engineer’ Title Lands in Top 5 of Dice Salary Survey

    ‘DevOps Engineer’ Title Lands in Top 5 of Dice Salary Survey

    The position of DevOps Engineer is gaining some serious respect, and with that comes cash.

    In Dice’s recently released “2019 Tech Salary Report,” DevOps Engineer now ranks in the Top 5 of all tech salaries, with an average of $111,683. The survey is based on the responses of 10,780 technology professionals conducted between Oct. 22 and Dec. 13, 2018.

    It’s the first time that the title of DevOps Engineer has appeared in the Top 5, according to Dice. The one caveat is that the position might not be as popular as the annual salary suggests; rather, more professionals now identify with this position.

    What it does mean is that the DevOps Engineer title is now ranked with more established tech titles, including senior leadership positions CEO, CIO, Vice President and Director, with an average salary of $142,063; System Architect, at $129,952; Tech Management, including Strategist and Architect, at $127,121; and Product Manager, at $114,174.

    Overall, the average salary for tech workers remained relatively flat year over year. In 2018, the average salary totaled $93,244, a less than 1 percent increase from 2017, when the average salary stood at $92,712, according to Dice.

    In an email, Dice CTO Klavs Miller noted that the increasing use and adoption of various cloud-based technologies is pushing enterprises to invest more in DevOps engineers to take advantage of that compute power to develop new apps.

    “Many DevOps Engineer job requirements include a heavy focus on the creation of infrastructure as code, essentially transforming operations into the management of codebases that create infrastructure (in cloud services or native cloud infrastructure rather than servers and hardware),” Miller wrote. “We are also seeing and experiencing first-hand that more and more organizations are moving toward having combined development and operations teams around a domain, rather than splitting along traditional functional lines.”

    In his email, Miller noted that while there might be few tech pros that use the “DevOps Engineer” title, it doesn’t mean that there’s an actual lack of engineers with the skills to help enterprises with their DevOps projects.

    Instead, it’s a matter of looking for talent that might already be there.

    “There is probably a shortage of talent that label themselves as ‘DevOps Engineers,’ but employers should look at the skills required by those roles instead,” Miller noted.

    “There’s a growing demand for a skill that is still evolving with market trends,” he added. “As an example, DevOps is not nearly as common as its more common sibling, software engineers, but it has become more popular each year according to our salary survey and among employers who are sourcing Dice for these positions.”

    The types of enterprises looking for DevOps talent are varied. Right now, insurance company Geico and hardware store giant Lowe’s are each looking for specific DevOps titles, according to Dice.

    If private business isn’t your area, the Louisiana Economic Development agency is currently advertising for an entry-level DevOps position.

    In addition to specific tech titles, the Dice report shows that several types of programming languages that are now associated with the DevOps discipline are commanding significant salaries in the market.

    For instance, those with expertise in Google’s Golang programming language can command salaries of up to $132,827, according to the Dice survey.

    A little further down the line, and keeping with Miller’s emphasis on cloud-based technologies driving much of the DevOps conversation, those with knowledge of Amazon’s DynamoDB and Redshift can command salaries of more than $125,000 on average, the study found.

    Additionally, platform-as-a-service (PaaS) skills can net a developer an annual salary of more than $122,000.

    In his email, Miller noted that it’s difficult to draw a direct line between needs for certain programming languages and the growing popularity of DevOps. The growing reliance on cloud and the types of digital transformation projects enterprises are engaged with now show how the market has shifted.

    “There are some assumptions that we can’t confirm, but DevOps shares programming languages with other roles that are also in demand, as employers adopt cloud-based technology and open source becomes more pervasive in companies’ tech stacks,” Miller added.

    — Scott Ferguson

  • Study: CISOs Need to Take Charge of DevOps Security

    Study: CISOs Need to Take Charge of DevOps Security

    A new report from CyberArk looks at the complexity that goes into securing emerging DevOps environments, and why CISOs need to take charge to protect them.

    As digital transformation is pushing more enterprises to adopt DevOps to address their needs for better software delivered faster and more frequently, a huge issue lingers over this strategic business shift: security.

    While businesses pay a good deal of lip service to cybersecurity, the rush to churn out new software and update those applications means that security is often skipped in the name of speed and innovation. At the same time, InfoSec is seen as a cumbersome barrier to what the DevOps team is trying to achieve.

    Despite these attitudes, it’s imperative that security is part of DevOps conversations, and it’s the company CISO who needs to lead that charge.

    In a report released last month, CyberArk polled 1,000 CISOs across the globe to assess how to make DevOps more secure and to ensure that the cybersecurity team is talking to the CIO and other executives overseeing the DevOps process to solidify good cybersecurity.

    The report, “Protecting Privileged Access in DevOps and Cloud Environments,” actually skips the more trendy term of DevSecOps in favor of a more holistic approach that bakes the security component into the very beginning of the application development cycle.

    Brian Kelly, the head of Conjur Engineering at CyberArk, believes that the DevSecOps term limits the ability of CISOs to offer ways to secure application development at the beginning of the process and ensure that updates embrace good cybersecurity practices.

    “Some organizations embrace it, while others refuse to use it,” Kelly wrote in an email about the debate over the DevSecOps term. “We do not use the term because it implies that security is a ‘gate’ or ‘phase’ that can be done after development is finished—but before the apps go operational. Security isn’t just a bolt on to DevOps, and this mentality can lead to a lot of broader issues for operation and development teams.”

    This means ensuring security is a part of every DevOps conversation, including making it part of hiring, training, road map creation, feature design, architecture, coding standards, code review practices, continuous integration/continuous delivery (CI/CD), operations, monitoring and maintenance.

    The CyberArk report makes clear that many of the tools needed to make DevOps successful within the business are a tempting target for cybercriminals looking to burrow deeper into the corporate network. For example, cloud access keys—whether for AWS, Microsoft Azure or Google Cloud Platform—can open a wide range of company data for threat actors to steal.

    The Uber data breach of 2016 started when the company’s AWS access keys were compromised.

    Other targets include CI/CD tools for configuration management, including Puppet, Chef and Ansible, as well as tools for running automated tests and builds and this can include Jenkins and Bamboo.

    Another red flag for security teams and CISOs is GitHub, since developers tend to have multiple accounts, and can sometimes mix code from personal and corporate projects. A cyberattack using brute force methods can sometimes leave code exposed. Since many developers cook some secrets into the code, such as database credentials, this can expose company and customer data.

    “[GitHub] hosts most of the world’s code for modern projects (and almost all open source projects) so any security mistake they make will have wide-reaching consequences,” Kelly wrote. “Up until now, they’ve been able to sidestep many issues by just being a hosting provider for repositories and leaving the responsibility to their customers for what they push up there. But they’ve since added more features for collaboration and CI, which naturally make their users want more of a safety net: automated scanning of third-party libraries for CVEs or watching for things like AWS keys being checked in, for example.”

    With all of these difficulties, what can the IT and DevOps teams do to increase security? How can the CISO help make security a central part of the app development process? Kelly offered some practical steps enterprises and their leadership can take:

    • Engage security teams early in the development cycle. This helps identify security issues early in the process.
    • Use a security-policy-as-code approach to have the Dev, Sec and Ops teams communicate efficiently and unambiguously. This means that new applications or microservices would declare their security policy requirements within code. It gets checked into source control and is as much a part of the application’s code as its internal implementation.
    • Decomposing large applications into smaller microservices, each with their own security policy.
    • Reducing the “batch size” of changes as much as possible, enabling the security team to work on a smooth, predictable flow of small items.
    • Getting end-to-end early to mitigate the risk the ongoing upgrades of components and modules.

    At day’s end, while CISOs need to lead the security charge when it comes to application development and delivery. At the same time, DevOps isn’t a fad, and will remain part of the enterprise for a long time. Security leaders need to do some embracing of the trend, too.

    “If CISOs want to remain relevant, they will need to see DevOps as a model that they have to align with, not against,” Kelly wrote. “Recent history shows us that large, org-wide DevOps initiatives rarely get shut down once they start, so a successful CISO will build bridges to the Dev and Ops groups and work with them to apply visibility controls and security oversight in ways that don’t cripple their speed. A failing strategy would be for the CISO to resist that sea change and instead try to be a gatekeeper and hold up inevitable progress. The only progress they’ll likely see will be in the speed of their replacement getting hired.”

    — Scott Ferguson