devopsy.kinsta.cloud

Tag: APIs

  • Cisco Looks to Bridge Gap Between Networking and DevOps

    Cisco Looks to Bridge Gap Between Networking and DevOps

    Networking in the enterprise has always stood apart. It might take two minutes or less to spin up a virtual machine, but it can take two weeks to set up the network connections. To help bridge some of the gaps between networking and the rest of the DevOps team, Cisco Systems this week unveiled a series of intelligent switches that promise to do just that.

    Via a Cisco intent-driven networking initiative, it soon will be possible to describe a business outcome that will be automatically enabled at the networking level, says Kiran Ghodgaonkar, senior marketing manager for Cisco. To achieve that goal, Cisco is moving to unify networking on an end-to-end basis, which Ghodgaonkar says primarily will require building a common controller that can be implemented at both the edge of the network and in the data center.

    The first concrete step in achieving that goal is a set of Catalyst 9000 switches running an IOS XE operating system, which Cisco has imbued with machine learning algorithms. Over time, those algorithms will be deployed across both legacy networking infrastructures when appropriate, as well as any new networking and security equipment Cisco brings to market in the future. Once in place, IOS XE will make it easier to implement policies in software that spans everything from prioritizing traffic to implementing security polices.

    This latest Cisco initiative follows the rollout a Tetration analytics platform based on an instance of Apache Spark and Cisco’s acquisition of AppDynamics, provider of an application performance management service delivered via the cloud. Ghodgaonkar says Cisco is aiming to combine analytics and machine learning algorithms to simplify network management in a way that furthers DevOps integration. Right now, however, the biggest impedance to achieving that goal, says Ghodgaonkar, is the fact that 80 percent to 90 percent of networking tasks still require manual intervention. Because of that issue alone, many organizations have experienced limited success in terms of integrating server, storage and networking management, despite the rise of software-defined networking (SDN) in the data center. In fact, to get past that issue, many organizations make investments in pre-integrated systems such as the Cisco Unified Computing System (UCS) platform. But the number of organizations willing to make that kind in forklift investment in IT infrastructure is not all that high when compared to the size of the overall IT market.

    The biggest DevOps issue IT organizations now face, says Ghodgaonkar, is that as various elements of the IT organization invest in automation, silos of automation are created. Cisco says it has developed open application programming interfaces (API) will provide interoperability between its technologies and the rest of the IT environment to help break down those silos.

    It’s still a little unclear these days just how much networking and IT operations can get automated. The good news is that, given the state of network management inside most organizations, things can only improve. The real issue (as any existing DevOps team already knows) may not be so much the technology as it is the culture differences spanning the compute, storage and networking teams.

    — Mike Vizard

  • The Time Is Now for an IT Operations Revolution

    The Time Is Now for an IT Operations Revolution

    There’s a level of frustration starting to build inside IT operations teams that’s about to boil over. Lucas Carlson, vice president of strategy for Automic, a unit of CA Technologies that provides an IT automation platform, says developers are bullying IT operations teams by programmatically invoking application programming interfaces (APIs) exposed by public cloud service providers. As the number of application workloads running in those clouds increases, IT operations professionals are losing control.

    To ensure their continued relevance, IT operations teams need to leverage automation to inject levels of agility into local data center environments, he says. That would make it much less likely developers would feel the need to programmatically invoke IT infrastructure resources.

    Carlson says achieving that level of agility will require IT operations teams to rely more on machine- and deep-learning algorithms to automate low-level tasks. Instead of thinking in terms of managing silos of compute, storage and networking resources, Carlson contends IT operations professionals need to reinvent themselves.

    The artificial intelligence (AI) models based on advanced algorithms must be trained continuously. IT operation professionals could become the trainers of those AI models. In fact, Carlson says IT operations teams are approaching a critical moment in history: They can either wait for AI models to eventually replace them or they can leverage them to take back control of IT infrastructure.

    However, developers tend to make decisions based on what they prefer versus what’s best for the organization. Before too long, multiple duplicate operations management frameworks are in place. Rather than incur all those management headaches and associated technical debt, Carlson says organizations need a single framework that can more adroitly accommodate the multiple roles within the DevOps team. It’s already long past the time when IT teams should have modernized their operations, he says.

    The relationship between developers and IT operations teams has always been contentious. The more belligerent members of the developer community are even making a case for “NoOps.” In that context, all the IT dollars spent on operations are moved into developing applications because the management of IT infrastructure is completely automated. Given the amount of legacy infrastructure that isn’t likely changing anytime soon, achieving that level of automation inside the enterprise won’t happen for a while. But there’s no doubt that IT operations needs to get more agile. Because of that issue, investment in software-defined data center (SDDC) technologies is growing rapidly.

    Instead of resisting that trend, Carlson says teams need to demand that internal IT operations be brought into the 21st century. It’s true that SDDCs infused with AI models will automate most of the lower-level tasks associated with DevOps. But those members of that IT operations teams who become the masters of building and training those AI models will be among the most highly sought-after individuals in IT.

    — Mike Vizard

  • Reality Check: Bringing Continuous Delivery Dreams to Life

    Reality Check: Bringing Continuous Delivery Dreams to Life

    Every developer shares the same dream: to write a piece of code or app that changes everything.

    Whether it makes your company do things more efficiently, attracts hordes of new customers, simply makes existing clients happier or maybe even changes the world a little bit, that code is what we strive for. Some of us don’t merely harbor that dream; many of us actually achieve that moment of brilliance. For those who have, it’s remembered as a moment of pure joy.

    So, where does this magical process begin? First, you talk to your manager and, if you’re lucky, she or he agrees that it is a fantastic idea and starts to sell it internally. Sometimes the idea is so good that you get the go-ahead on the same day and even get a small team to work on it!

    Then, soon thereafter, reality kicks in.

    You need data, inevitably from applications running somewhere in the cloud and/or some other data from on-premises data stores. You need to interface with APIs, some well-documented, some not. You need some user experience work done, and the standards your company has adopted don’t really match your vision.

    You also need to interface with many of the standard services your company has already implemented because, after all, your app doesn’t run in isolation—you work with existing billing systems, a CRM system, a fulfillment system … and all of them run on servers that can be anywhere. Soon you find yourself fighting for test data, systems to test with and applications and APIs to interface with. Before you know it, you’re banging your head against the wall.

    Fortunately, your teammates are flexible, creative people. And because your internal systems seem to work against you, you go around them. You download a trial version of some testing software; you install a copy of some internal systems on a virtualized test server (if you can find one) and start working with these.

    Since your app needs to be snazzy and snappy, you download some open-source software to help you test the performance on mobile devices. After three sprints, you’re now ready to move your app to an environment where you can start performing real tests!

    Most times this is the moment when the hardest reality kicks in. The copies of the systems you tested with were not up to date. Data has changed. APIs have changed. The real system responds three times slower than the copy that sat on the server right next to you. The data in the cloud has changed. The performance testing tool you used is not available in the official testing environment. … The list goes on.

    This doesn’t even include the difficulties of moving all the components of your new app to different environments. Trust me, this is not a fictional nightmare scenario; it is reality and I see it happen all the time.

    Now, what if you had the test-data generators necessary to create the data you need and a server that ran virtualized services, mimicking all the (up-to-date) external systems you need including the APIs to talk to them—services that would allow you to say, “What if this application responds three times slower when I request data?” or, “What happens if this system suddenly goes down?” What if, when you’re done building, you could simply migrate the app and all the components that go with it to another (test/QA) system with one click?

    Everybody wants to support people with good ideas and turn these ideas into working software without the frustration that very often distracts and demotivates people. To do so, you need to create the infrastructure that supports this. Is it difficult? Not really, the tools to automate services, create test data (and test scenarios) and migrate complete applications from A to Z are available.

    However, just like a great app, this too starts with a dream—the dream of allowing people with great ideas to work on these ideas without the overhead of an infrastructure that allows them to focus on the coding and not on the infrastructural issues; the dream to create a software factory where the creation of the app takes 90 percent of the time and the infrastructural burden only 10 percent, not the other way around.

    If you’re serious about developing software and want to create an environment where the infrastructure allows people to develop and release software faster instead of working against them, find the tools that support it and jump in, because they exist today.

    Enacting this change now and maybe the next time someone comes to you with a great idea you can be confident that they will have all the support needed to get there efficiently.

    — Marcel den Hartog

  • 4 Top Tips for Developing a New Feature

    4 Top Tips for Developing a New Feature

    As a backend developer at CA BlazeMeter, I recently had the chance to work on a big feature for our web app. The feature required a massive change on our backend code, a big change on our frontend and multiple migrations. You know what I’m talking about—it was one of those projects that developers fear.

    But, I gained a lot of valuable experience from the project. Not only did we deliver a kick-ass feature, I also learned the importance of teamwork and patience when developing something new. Armed with those two values, and the right tools and technologies, you can deliver a new feature quickly and efficiently.

    Here are a few more things that I learned that should prove helpful whether you’re developing a major feature or one that only requires a minor code change.

    Tip 1: Plan to understand the development requirements

    The process of development starts way before you start coding, with planning. After the product manager decides what needs to be developed, R&D needs to understand the cost: how many days (or weeks) of development the feature will take, how many people will be needed and which APIs are required.

    To do that, the product manager needs to conduct at least three “grooming” sessions with backend, frontend and QA leaders. The grooming sessions start with the product manager presenting the product flow and requirements.

    Since no one wants to read a lot of text (except perhaps the awesome readers of this article), the product manager should prepare mockups. They don’t have to be designed or colorful, but they do need to show the team what the feature looks like in his or her vision. During our grooming sessions, we used Balsamiq, which lets you create screen sketches for how the product is supposed to look, feel and work.

    Editing screen sketches using Balsamiq

    After the initial presentation, the sessions should include a lot of open discussion—asking questions, checking for answers and examining compliance, etc. —until everyone is satisfied and understands what the feature requires from them and from others.

    This is the time and place for frontend, backend and QA to give their opinion about the schedule and to alert and flag any potential problems in development or extreme use cases. It’s the role of the product team to answer questions and change the flow and requirements until there are no R&D rejections.

    Afterward, the frontend and backend leaders need to create a feature development plan, sitting down to determine what data they need to transform the product vision into a real thing. Or in other words, which APIs are required for each flow or button, how long each step is supposed to take, including buffers, and the order of development. APIs are the agreed “contract” between the development teams. After determining the contract terms, each team can proceed individually.

    I recommend you write your APIs in Swagger, which turns YAML files into detailed HTML pages with all the APIs.

    Swagger Editor

    This plan needs to be transparent to everyone working on the project. In fact, it’s even better if each project participant can add and describe scenarios. Tools such as TestLink and Atlassian do just that. Then, when the code is ready, developers can go over the list and see everything is covered.

    We will need to stick to this schedule later on, especially when new demands unrelated to the feature come in, such as fixing bugs. Sticking to this plan is the best way to distinguish the wheat from the chaff of the development process.

    Start writing automation ASAP

    After planning, we want everyone to start working, and not only the backend. To do that, the backend can prepare mocks, either through outsourcing services such as JSON servers, which show inputs and outputs, or by writing dummy code.

    I mentioned earlier that we used Swagger to write APIs. We also used it to generate a server in NodeJS/Python, like this:

    Generating a JSON Server through Swagger

    From this moment on, frontend can begin writing and building screens and QA can start writing API automated testing, by using tools suh as JMeter. At this point, frontend and QA have the response codes and response bodies; they know which features to put where; and which APIs they need and how to use them.

    QA also know which data they’re missing, such as field names and field data. Sure, the backend code isn’t ready yet and all of the automated exams will return “red” instead of “green,” but the main issue is making sure everyone is as prepared as possible.

    This includes load and performance testing. You can use open-source tools such as JMeter, which is enhanced by CA BlazeMeter. By uploading our test script into, configuring the number of virtual users and ramp-up time, determining geo-locations to test from and analyzing KPIs to determine trends and the health of our system, we can ensure our product is delivered to our customers without crashing.

    This is also true about sending code to QA: Developers should test their code themselves and ensure that when they commit their code, it works 99.9 percent of the time.

    By starting automation as soon as possible, you can minimize future bottlenecks in the development process.

    Ensure the team communicates and works together

    Like I said before, teamwork is the most basic requirement when creating a new feature. This is because new questions arise during development and because requirements change. Therefore, everyone in the team needs to be in sync all the time, from the product manager and the designer to the developers and QA.

    Open communication lines are crucial because everyone is working on the same product at the same time, but each developer or QA should be working on a different part of the product and on different functionalities. Communication ensures each person’s work is compatible with the rest of the team’s work. This saves time, prevents frustration and redundancy and ensures maximum professionalism.

    To make sure everyone knew what was happening, we used Slack and opened a dedicated channel for the creation of this feature. We also had daily (short!) standups where we discussed what we have and what is missing.

    Opening a Slack channel

    Communication also includes team and assignment management. JIRA, our primary issue tracking tool, offers Kanban-style, Scrum and fully customizable boards, which show the progress of each development task, including the developer assigned to it, and his or her exact status or phase.

    This helps you understand what other people are working on, and just as importantly, where people are stuck. If you can, get everyone to work in an open space and put a large screen in the middle of it. This lets you stay in sync at all times.

    We used JIRA to open and manage bugs as well. This was helpful when multiple QAs found the same bug. It’s important to carefully choose which tickets are opened and put them under one epic, so everyone stays focused.

     

    JIRA Dashboard

    Another important aspect of communication is ensuring everyone uses the same terminology, as sometimes people might use the same term to describe different things or different terms to describe the same things. To prevent mixups that could waste your time, make sure everyone is on the same page. For example, do you say “system admin” or “<name (ex. BlazeMeter)> admin”?

    Work on the same development environments

    It’s important that everyone works on identical development environments and even on the same version of the tools and libraries. Otherwise, it could cause an (in)compatibility mess, including a lot of unnecessary back and forth and even might cause bugs to pop up. This environment needs to be as similar as possible to production, but only on a smaller scale.

    Conclusion

    Now look at the feature that came out of all that work here. I’m so grateful that I had the opportunity to work on it with an awesome team.

    I hope these best practices help you the next time you develop a new feature. Don’t forget to keep going agile after development and delivery by running tests in a continuous environment such as Jenkins for every commit.

    CA BlazeMeter works with Jenkins, ensuring you can load test in the cloud and analyze your data while shifting left. You can also use different APM tools, such as New Relic, AppDynamics and CA APM to alert you about errors before QA, which saves time.

    — Dor Atias

  • How Google’s Acquisition of Apigee Impacts Developers

    How Google’s Acquisition of Apigee Impacts Developers

    Google revealed a few weeks ago that it is acquiring Apigee for approximately $625 million. The deal demonstrates both the increasing demand for APIs (application programming interfaces) and Google’s desire to remain on the cutting edge of providing a platform for APIs. The purchase also underscores some important trends for developers.

    If nothing else, Google’s purchase of Apigee validates the importance of APIs for online commerce, as well as for businesses and developers in general. I had an opportunity to get some insight and feedback from Steve Davis, CTO of Four51, a provider of B2B ecommerce solutions, regarding the Apigee news.

    As DevOps and microservices continue to gain momentum, APIs play a very pivotal role. According to Davis, the API is the product for Four51, which makes it the foundation for the company’s DevOps strategies. “Most microservices are API-based as well, so the role is simply that they are the heart of these strategies. The concepts of DevOps relate to continuous delivery and all of the stages from development to deployment and operation. APIs allow developers the tools and visibility they need to work in this environment of delivery.”

    I asked Davis if he believes that it was a smart decision by Google to acquire Apigee at this time. “It was a wise move by Google to jump in, as the API economy is accelerating faster than other areas,” Davis answered. “The corporate customer appetite is increasing for APIs as companies continue their digital transformation, and API management is a critical discipline for security, scalabilty and reliability.”

    Davis pointed out that Google’s primary competitors—Microsoft and Amazon—have been out there with AWS and Azure cloud services for a while, and they’re doing quite well. He suggested that Google has some catching up to do, and that Apigee—as a best-of-breed independent platform—made a lot of sense for Google to target.

    For developers, Google’s acquisition of Apigee elevates Google as a cloud-hosting option—putting it on a more even footing with Microsoft and Amazon. The presence of an API management services enables developers to offload that tedious work and focus on more important things—such as continuing to innovate and bring value to their products.

    Davis also shared some thoughts on what the Apigee acquisition means for Four51. “Though we aren’t directly impacted by this acquisition, it does validate the path we’ve taken with our API-first eCommerce platform,” he said. “Many of the same enterprises looking to tap into the API economy are doing so to improve security, scalability and efficiency in their e-commerce and order management with their customers. We’re a big believer in the benefits of cloud infrastructure, as we recently moved hosting of our OrderCloud.io platform to Azure, and we’re utilizing the services there.”

    DevOps and microservices are all about streamlining productivity and operating as efficiently as possible. APIs play a major role in that effort, because they allow much of the heavy lifting on the back end to be automated. The acquisition of Apigee makes Google more competitive with its primary rivals, and highlights just how important APIs are—and will be for the foreseeable future.

    — Tony Bradley

  • The Internet of APIs

    The Internet of APIs

    The Internet moves at the speed of light. Not only that, but the terminology used when describing The Internet seems to change with the tides. One of the newest phrases to start making the rounds is “The Internet of APIs.” In an attempt to understand exactly what it is, I went looking for a definition and was unable to find anything at all. As such, it’s time to put a definition out there.

    There are two important words in the phrase: “Internet” and “API”. Let’s start with the last word first and work backwards. What exactly is an API? Luckily, the definition of API is pretty settled. For our purposes, I like the definition given to us by API Evangelist:

    An API, or Application Programming Interface, at its most fundamental level, is how software applications talk to each other using the Internet. An API allows you to open up data and other digital resources, to public developers, businesses, or even between departments and locations within a company. APIs are increasingly the way in which companies exchange data, services and complex resources, both internally, externally with partners, and openly with the public.

    And while we’re at it, let’s look at the popular definition of “Internet” given to us by Webopedia:

    The Internet is a global network connecting millions of computers. More than 100 countries are linked into exchanges of data, news and opinions.

    So we’re narrowing down on a definition of The Internet of APIs. But we’re not quite there yet. Let’s look back at recent history to see if we can clarify things a little bit. The modern-day web API has been around since Salesforce introduced it’s API at the turn of the century. In the year 2000, they introduced an XML-based API that kicked off the web services API era. And while there were a few other web services APIs that came out afterwards (including eBay and AWS); but things didn’t really start getting serious until Flickr launched their API in 2004. The social network revolution, sparked by companies like Friendster, MySpace and Twitter, brought about a real need to securely exchange information between multiple applications. So when Facebook and Twitter announced their own APIs in 2006, that was when the usage of APIs grew to include identity management in addition to the exchange of data it had become known for.

    And then came smartphones. APIs exploded in popularity once the smartphone industry took over the continent thanks to Palm and Blackberry; then came iOS and Android. Why the popularity explosion? Well, applications needed to be made for these new (popular) form factors. However, these new form factors had very specific requirements around data usage, drastically different screen sizes than application developers were traditionally used to, along with previously unseen device capabilities. Add to that the fact that mobile applications needed to figure out a way to share data with each other securely and a perfect storm of needs presented itself.

    While all of this was going on, the wireless capabilities available to the consumer also changed. In the year 1999, one year before Salesforce released their web services API, 802.11a and 802.11b hit the market. WiFi was still a growing technology, but the introduction of 802.11a and 802.11b changed everything. The prior iteration of 802.11 had a transfer rate of 2 Mbps; while a allowed for short range speed of 54 Mbps and b allowed for longer range speed of 11 Mbps. The cost of b was also a lot lower than a.

    Also in 1999, the first full Internet service available on a phone was launched in Japan by NTT Docomo. This was during the days of 2G networks. In the US, around the same time, cell phones became affordable to everyday consumers. It was a time of incredible growth in the wireless industry and over the following years, cellular networks were stretched to their limits.

    This is the environment that led to the growth of the web service API. There were a few things that were extremely important to the upcoming boom:

    1. A speed boost in wireless networking was required; and
    2. Wireless networking hardware needed to get into the hands of more people; and
    3. The cost of wireless networking hardware needed to drop; and
    4. Cell phones needed to get in the hands of more consumers; and
    5. Internet service needed to hit the cell phone.

    Add all of that to the growth of social networking and identity management via API, and a perfect storm of circumstances presented themselves. More and more APIs were built utilizing a person’s identity for personalization and security.

    Thus the Internet of APIs was born.

    So, based off of the existing definitions as well as recent history, what exactly is “The Internet of APIs”? The Internet of APIs is:

    A global network of structured data, digital resources and commands accessible via a growing interface that allows applications and devices to exchange information using any and all communications protocols available.

    Since the creation of the ARPANET (arguably, the Internet’s predecessor) packet-switching networks have been utilized as an efficient and reliable way to exchange data. So what’s different about the Internet of APIs since at its core it’s about exchanging data? The main difference lies in the fact that the data provided by the Internet of APIs is data with structure. It’s data that is being shifted around and meant to be consumed in ways that may not possibly have even been foreseen by the creator(s) of the data. The Internet of APIs allows data that may have been designed for one thing to be used for something completely different; thus facilitating the connection of two completely different applications in ways sometimes unimagined.

    The Internet of APIs is the underlying foundation to the Internet of Things. Without the capability for connected devices to personalize their usage according to the individual, as well as share data, the Internet of Things could not exist. Without the improvements in specifications in both wireless and cellular hardware, the Internet of Things could not exist.

    The Internet of Things is powered by the Internet of APIs; and as the Internet of APIs grows in scope and capability, the same will happen to the Internet of Things. Without regularly updated data and personalization, a hardware device is dumb. A thermostat is just a thermostat without the underlying API. But with the API, a thermostat becomes a Nest. Without an API, a light bulb is just a light bulb. But with the API, a light bulb becomes a Philips Hue light bulb. Without the Internet of APIs, a person walks in to their home, walks in to their living room, flicks a light switch and turns their heat up. With the Internet of APIs, a person walks in to a warm, well lit home because their thermostat turned on five minutes before they got home and their lights turned on one minute before they walked through the door.

    Data and personalization: that is the power of the Internet of APIs.

    About the Author/Kurt Collins

    Kurt-Collins-devopsdotcomKurt Collins is Director of Evangelism at built.io (link). Before that, he started his engineering career as an intern at Silicon Graphics (SGI) where he worked on debugging different parts of their UNIX-flavored operating system. Two years ago, Kurt co-founded The Hidden Genius Project along with eight other individuals after they all saw the screaming need to increase the amount of Black men in the technology industry. Mentoring young people interested in the technology industry is a personal passion of his. Today, he continues his work with tech startups and non-profits alike.