devopsy.kinsta.cloud

Tag: agile

REP SPOTLIGHT: Omnikron

DevOps Institute: Tell us a little about your organization.

Omnikron: With over 30 years of industry experience, we have honed IT and project management training techniques to ensure that course material caters to our clients’ needs. Our training is not just lectures from a book. We offer world-class courses, assessments and simulations in a variety of formats and at various paces. As the workplace has changed in the 21st century, so has the training approach here at Omnikron.

DevOps Institute: What other topics do you teach or consult on?

Omnikron: Omnikron’s most popular training includes ITIL, PMP, TOGAF9, Change Management, DevOps, Security & Security Awareness and Cloud.

DevOps Institute: What type of interest in DevOps are you seeing in your region?

Omnikron: There is a lot of talk about DevOps and our classes are slowly but surely increasing in size. Hopefully by the end of the year, we will have a packed house for our classes.

DevOps Institute: How do you think that DevOps contributes to the business’ value stream?

Omnikron: We have honed in on ITIL for years and DevOps is such an easy transition for most of our students. Not only are the students enjoying the class, but they are able to use their past experience to share stories in class as well as apply what they have learned outside the class.

DevOps Institute: What topic area in DevOps excites you the most?

Omnikron: Organization consideration and practices seems to be the favorite among our students. Plenty of dialogue has been given around this particular topic.

DevOps Institute: Why should enterprises and individuals consider formal DevOps training?

Omnikron: Because DevOps is designed to help our student continually improve their services to their internal and external customers. Can’t get any better than that!

July 5, 2016
Scania: Adopting DevOps for Auto Production

When you think of large vehicles, chances are you think heavy-duty front and rear axles, suspensions and cab parts. When you think of large engines, chances are you think crankshafts, cylinder heads, pistons and injectors. When you think of the brains behind a company that produces all this, you should think software and the processes that create it.

Swedish automotive company Scania was building its software intelligence using large, legacy development approaches, coding entire software packages as a single project. The organization’s teams were couched in silos. Releases were highly infrequent.

Everything changed when Scania scaled and conquered the partitions and divisions that separated teams and adopted agile practices and DevOps.

Scania and its Software

Scania, a large global maker of heavy transport buses and trucks and marine and industrial engines, runs many software departments. Its Connected Services department creates web-based portals, APIs and embedded software that transmits live and historical data transmitted to and from drivers and vehicles for logistics including positioning, tracking, service planning, geofencing and messaging.

Scania’s move toward DevOps accelerated in the 2012-2013 time frame, when the company combined developers and testers into feature teams and molded its first DevOps practices. The company had a host of development issues before that.

“Before 2012, we were organized around traditional big software projects, which slowed progress and hindered communication. We had long lived branches and a product release cycle of about three releases a year. Silos existed between Dev, Ops, Test and Marketing,” said Anders Lundsgard, technical architect at Scania Connected Services. By 2013, he added, feature teams were adopting processed based on a strict adherence to Scrum.

Scania’s Dev and Ops teams were housed in different locations and working separate goals that were not aligned or cohesive. These teams suffered from confused communications and increasing stress as development worked only on new features while operations tried to use the new software in a stable way in production.

Pivotal Changes

Everything started coming together when Scania formed feature teams that progressively shouldered more of the responsibility for forwarding code to all environments and making it work. “There was no involvement in this from operations or infrastructure as they supported tooling and automation for on-premise infrastructure and shifted software to a cloud platform,” Lundsgard said.

Scania ultimately added Perforce Helix, a software configuration management tool. As long as software engineers know that the focal point of their work is checking in their code, the Perforce product can be the single point of storage of any one software component for all projects in the deployment pipelines. This way there are no multiple, correct current versions of any piece of code that could cause confusion. “Perforce has a number of capabilities that help engineers collocate software for reviews and even shelve files as they pass between developers,” says Lundsgard.

Scania uses Perforce Helix in combination with its continuous integration (CI) environment and processes to improve quality beyond what manual testing could ever accomplish. “The goal is that software engineers check code into Perforce Helix at least daily,” Lundsgard said.

Scania DevOps Processes Today

Scania now versions every change and automates builds, tests and deployments, with CI as a central development practice. The company no longer uses feature branches, with everyone adding code directly to the mainline. “We use a microservice-oriented architecture in order that each feature team can manage and own certain services,” Lundsgard said.

Different teams approach development with varying degrees of DevOps maturity. Some teams deploy software continuously, sometimes many times per day. Other teams benefit from using some manual testing and may not deploy software for a few weeks. “The tooling has enabled all teams to have a fully automated process for pushing their code into all environments if they choose—automatically or via a click of a button,” Lundsgard said.

Remaining Challenges

Scania next plans to fully automate every component of infrastructure from the network to the servers. “Some of our infrastructure and operations people have started to adopt infrastructure as code and they version that code in Perforce Helix as well,” Lundsgard said.

June 28, 2016
State of DevOps Report: Reaping Results

The newly released 2016 State of DevOps Report concludes that DevOps is “no longer a mere fad or buzzword,” but rather an established set of practices and cultural patterns. Yet, a closer look at this year’s findings reveals progress is far more substantial than any matters of perception.

The report was conducted and analyzed by the experts of DevOps Research and Assessment (DORA) and sponsored by CA Technologies. Among its findings is proof that high-performing organizations that have embraced agile and DevOps methodologies deeply are outperforming massively those that have not. And the gap is widening.

The State of DevOps Report also shows leading DevOps practitioners are appreciating benefits that span nearly every area of their business. If I know my colleague Gene Kim, he will assert this point exuberantly on our upcoming State of DevOps Report Webcast.

It’s noteworthy that high performers are surpassing incredible milestones related to core elements of the applications life cycle. Examples include deploying code 200 times more frequently than low performers, driving 2,555 times faster lead times, 24 times faster recovery times and producing three times lower change failure rates. This is evidence that DevOps is having the desired outcome on applications delivery, software quality and organizational velocity. It also shows organizations truly committed to wholesale transformation of culture and process are actively succeeding.

At the same time, the State of DevOps Report shows employees working at organizations enjoying this success are more satisfied with their jobs and more closely engaged with their companies. Chalk this up to the halo effect of operating in a fundamentally more sensible, innovative fashion. Cultural change is undoubtedly the greatest hurdle to DevOps advancement. Here’s proof that if you can navigate the process, your workers will be happier and more productive.

Using Employee Net Promoter Score (eNPS) as a barometer, the report finds that employees in high-performing organizations were 2.2 times more likely to recommend their organization as a great place to work. They also were 1.8 times more likely to recommend their team as a great working environment. Previous studies have found that such an air of positivity can be directly correlated with improved business outcomes, researchers noted.

Shifting the manner in which you build applications to better adapt to changing customer expectations is a necessity. That’s what began driving this movement years ago. That said, proof it also makes it easier to keep valued software engineering and IT operations employees should not be overlooked.

In addition to positive conclusions on the impact of continuous delivery and improvement of test data management, the report offers some important metrics related to return on investment. This is nothing short of landmark, as for decades we have struggled to make a hard case for IT investment beyond implied competitive advantage. It actually was impressive that last year’s report highlighted such a “predictive relationship between IT performance and organizational performance.”

One such new measurement of note is that high-performing teams occupy the least amount of time responding to unplanned work and rework—efforts widely recognized to be results of flawed processes. On a related note, DevOps leaders spend 49 percent of their time on new work‚ surely, in many cases, the next “must-have” feature which impacts competitive standing and, ultimately, the bottom line.

This report is fantastic. It reaffirms powerfully that so many of the promises made about DevOps over the last few years are neither hype, nor overstated. The gap between the leaders and laggards is growing. Industry landscapes and business models are being redefined.

This is critical transformational work that every organization must adopt to survive.

Are you on board? Download the full State of DevOps Report here.

June 23, 2016
REP SPOTLIGHT: TaUB Solutions
DevOps Institute: Tell us about your organization.

TaUB Solutions: TaUB Solutions is a consulting-cum-training organization based in Bangalore India. Our vision is to take the customers beyond the existing state of maturity to envisioned business outcomes.

We serve a lot of customers in southeast Asian region by delivering consulting assignments on ITSM, IT governance, project and program management and business relationship management. This also includes delivering business simulations on the respective domains and make our customers gain benefit from learning by doing.

DOI: What other topics do you teach or consult?

TaUB Solutions: ITSM, IT governance, agile, DevOps, business relationship management, cybersecurity, project and program management and ISO standards (20000,27001, 22301)

DOI: What type of interest in DevOps are you seeing in your region?

TaUB Solutions: At the moment, we have successfully delivered around 11 batches of DevOps Foundation and Certified Agile Service Manager courses. We appeal to our customers because of the practitioner focus that we bring to the table and take them through case studies to understand the end-to-end value chain. Our current run rate indicates a strong pipeline of interest in southeast Asian countries who want to apply the DevOps principles and philosophy in their workplace. We also see great traction from Indian corporates to plunge in to the DevOps journey to serve their respective customers better.

DOI: How do you think DevOps contributes to the business’ value stream?

TaUB Solutions: One of the things that appeal most to me is the ability to articulate the end-to-end value chain. Business today does not care about which framework or methodology you adopt for the customers. All that matters is to deliver an end-to-end value chain and get a common understanding among different functions and leverage the best across the enterprise. I see tremendous value of leveraging agile, ITSM and lean principles with culture and tool chain to demonstrate key business value.

DOI: What topic area in DevOps excites you the most?

TaUB Solutions: There are three things:
- Culture change
- Power of automation using tool chain
- Business KPIs and Common Metrics across the organization
DOI: Why should enterprises and individuals consider formal DevOps Training?

TaUB Solutions: Today, DevOps is still at its infant stage and has not yet been codified. So people have their own perspectives, thoughts and misconceptions. So in order to be certain with the path they traverse, it is increasingly important to step in the right direction. This is where a formal DevOps training from DOI can help people to understand fundamentals and DevOps trends from practitioners.
June 22, 2016
MTTR: The Magic Metric
How do you improve with one metric quality, cost, performance, employee and, most importantly, customer satisfaction? I think MTTR is one to consider, but with a DevOps twist (agile+lean+ITSM). The thing I love about DevOps is that it allows for people to think differently. DevOps is the movement driving organizations to find new ways of collaborating, communicating, cooperating and improving the way they use technology to create and support services.

While many agree that MTT stands for “Mean Time To,” there is no common agreement on the R. I like using 5 Rs, as they cover the life cycle of idea to realization (in the hands of the customers in a way they like).

So what are the 5 Rs? Well, let’s work backward (shift-left thinking):
- Review: What happened? How can we stop this in the future? How can we shorten the cycle?
- Return: Do customers agree that all is well? How satisfied are they with not only our performance but the fact the event occurred?
- Restore: The time and effort to get servers, network, applications, data, passwords back as they should be.
- Repair: The time and effort to resolve the issue.
- Respond: The time to notice, decide if we can fix or get others involved, and their reactions to begin getting things sorted.
MTTR is a cycle of events. Breaking them down and understanding the value of each step enables you to reduce time, effort, waste and cost. How often do things break and why? Are they self-inflicted because we test poorly? How can you apply automation or self-healing? How good is your monitoring and alerting and response mechanisms? How great is your documentation? Do you rely on a certain person to fix things or can anyone? Knowing the answers to these questions can make issues become less frequent—or, at least, reduce the effort to address them when they do occur, as things will break.

Knowing these answers also helps clarify priority of issues. Working back from the customer drives the flow of work to highlight faults. Why treat all issues as P1 all month? For example, payroll is only a P1 when you are getting paid, but the rest of the month it can be a P2. Working with customers will help ensure you are fixing the important things, not something someone wrote in a service level agreement that is rarely updated.

MTTR is also a fabulous learning measure. DevOps is all about collaboration and that begins when people talk to each other, repair things or introduce a better test or monitor or alert to capture issues and customers are minimally impacted. Let your service desk own MTTR and see how issues get resolved better, faster and more safely.

MTTR helps drive movement to virtual or cloud. MTTR can also help you improve your A/B use of infrastructure or services. Have an issue? We’ll flip a switch.

But the overall goal of any metric is to drive actions, decisions and improvement. MTTR is such a measure. Be brave, throw away your SLAs, get people involved for issues and use MTTR to make things better across the life cycle. Then celebrate the success of your journey!

Let us know. Do you use MTTR? Can you measure each step and perform a retrospective?

About the Author / Daniel Breston

Daniel Breston is the Chief of DevOps Transformation for Ranger4. Daniel has 30+years leading IT Service, Operations, Applications, Data Center and service partners in management of technology enabling business processes and functions. He has also spent 15 years of consulting and coaching leadership teams across the EU and UK in developing their people, processes and use of technology to deliver technology services better, faster, safer. Daniel has qualifications in IT Service Management (ITIL, COBIT,etc.), Lean and DevOPS.
June 17, 2016
3 Rules for Successful SAFe Adoption

Startups and enterprises alike are redefining—some say disrupting—markets, industries and customer experiences by delivering innovative ideas at a rapid pace and responding to feedback. Often they fail, but they fail quickly and adjust. You know the ones: Spotify, Etsy, Netflix, Airbnb. How do they do it? What can we learn from them that will work in our own organizations? What is different? In a nutshell, all of these successful organizations innovate completely, not only in terms of what they do, but also how they do it. And the common thread is the application of lean and agile practices to improve their culture and deliver innovation to customers faster than the rest of us.

Today’s business imperative is to be the disruptor and more and more organizations are turning to enterprise scaled agile as a set of practices to help lead transformations to achieve that goal. The Scaled Agile Framework (SAFe) is the industry-leading enterprise scaled agile framework with a proven track record in thousands of organizations for several reasons, not the least of which is the incredible breadth of definition and collateral so that you just don’t have to make this stuff up yourself. Nevertheless, there is not an one-size-fits-all framework and SAFe is no exception. But there are three key concepts prescribed by SAFe that, if you do nothing else, will help you be successful in your evolution toward becoming that disruptor.

Go SAFe or Go Home

1. Apply Lean Principles at an Enterprise level
You’ve likely heard the term “MVP,” or minimum viable product. This is about investing “just enough” to deliver a capability—a solution, product or application—and get feedback. It is about avoiding the pitfalls of over-engineering. This is perfect embodiment of lean thinking. By focusing on investing only as much as necessary to get feedback quickly (and doing this continuously) you not only reduce or eliminate wasted resources but also speed time to feedback. Lean is also about the notion that “capacity rules”—in other words, only invest as you have capacity to follow through. Otherwise, what’s the point?

2. Apply Economic Thinking
Traditional methodologies typically focus on priority to rank at all levels, from business initiatives and projects down to team-level work that delivers changes into products and applications. In the agile world, story points (or complexity) also might be used. So what’s the problem with this? First, it is only a single dimension. What about business or customer value? What about cost of delivery or reducing risks or enabling market opportunities? Is timing a factor? All of these aspects should be considered when ranking because, at the end of the day, organizations are looking for the “biggest bang for the buck.” By applying economic thinking, you are maximizing the business’s ability to deliver the most value at the lowest cost in a timely manner.

3. Focus on Delivery of Value
I’ve often heard that revenue generation is the primary goal when organizations look to speed delivery cycle times. In fact, the goal should be delighting the customer; revenue is a bonus, a side effect of happy customers who use your stuff and pay for it! A focus on improving feature delivery is too narrow. What you really want to do is improve time to value. That means articulating what value means, associating the notion of value with features that are delivered, assessing that value and then tracking so you have insight into your organization’s ability to deliver value and get better at it.

SAFe embraces all of these concepts through artifacts and activities anchored in lean and agile practices. Try it, you might be surprised at how impactful this can be in improving your ability to innovate and be the disruptor instead of being disrupted.

Read how a typical enterprise dipped its toe in SAFe water to drive delivery of a critical project, and be sure to visit the SAFe landing page to learn more.

About the Author/ Amy Silberbauer

Amy is a Solution Architect charged with the definition and delivery of the Enterprise Scaled Agile and IBM DevOps Plan solutions and strategy for the IBM Watson IoT line of business. She is a recognized subject matter expert on scaled agile and software development lifecycle solutions, including enterprise modernization, SOA and collaborative DevOps. She has been with IBM for 29 years and has two decades of software development experience as an engineer, architect and manager. She is a certified SAFe® Program Consultant experienced in leading and consulting on multiple internal and external SAFe (Scaled Agile Framework®) transformations.

Connect with Amy on LinkedIn.

June 15, 2016
What is a DevOps ‘Best Practice’?
In IT, we love our frameworks. Whether agile, ITIL, lean, cobit, six sigma or others, our thirst for “best practice” guidance seems to be unquenchable.

The concept of a “best practice” is an enigma. Who decides if a practice is actually the best? And best for whom? While an “adopt and adapt” mentality is advocated in most frameworks, there is still a tendency to benchmark efforts against and conform to published best practices. Whether promoting “best in breed,” “world-class,” or “best practices,” many organizations swing these terms around as some form of competitive advantage.

Is it really? Shouldn’t business outcomes be the real competitive advantage and measure whether IT’s practices are really the “best” to meet customer requirements?

The DevOps Institute was founded in 2015 as a global learning community for emerging DevOps practices. Since our launch, we have intentionally avoided any reference to DevOps “best practices.”

We believe that DevOps practices are constantly emerging and, in many cases, being proven in the wild.

Through research, interviews, conferences and the experience of our Regents and partners, DOI’s goal is to identify, capture and share emerging DevOps practices through knowledge, education and certification models that best serve the learning community and enterprise IT. We feel it is premature to certify an individual as an “Expert” or “Master” when the expertise and mastery needed for successful DevOps is not clear and may not be appropriate in a single person.

DevOps is fairly young. It does not have a single body of knowledge. There is disagreement on whether it is a philosophy, movement or framework. Despite that, DevOps is crossing the chasm from innovators to the early majority faster than any other IT framework. Why? Perhaps it is because DevOps speaks to many of the cultural and technical challenges that IT has been amassing for years.

So instead of focusing on best practices, let’s agree on DevOps’ core principles:
- Go faster
- Shorten feedback loops
- Experiment and learn
- Cultural transformation
- Deliver business and customer value constantly and consistently
No doubt, some practices have clearly risen as being critical to the realization of DevOps’ core principles:
- Agile software development
- Continuous integration
- Continuous delivery pipelines
- Automated and continuous testing
- Proactive monitoring
- Improved communication and collaboration
Still, over the past year, DOI has seen additional practices gain interest
- DevSecOps/rugged DevOps
- ChatOps
- Agile service management
- Lean
- Immersion practices (Garage, Lofts, Dojos)
- DevOps teams
What will emerge next year? Who knows—but that’s the exciting part of DevOps. As time passes, there is less contradiction and more supplementation. Each emerging practice seems to complement and fine tune what came before.

Even without a standard definition, DevOps is already encouraging organizations to examine their proprietary practices, identify gaps, assess their automation and, most importantly, engage in collaborative discussions. Transformations have taken seed without defined best practices. That’s awesome! Let’s not quell the momentum by attaching a static set of best practices.

Will DevOps’ best practices eventually be agreed? Maybe. DevOps touches almost every aspect of IT management—people, practices and automation. That’s a tall order for a set best practices. Certainly some will try to publish their version of a “definitive” DevOps body of knowledge. Even “The DevOps Handbook,” an important book publishing later this year, was built as a collaborative effort between many of the early founders of the DevOps movement. Their perspective and insight will be awesome but not finite.

I understand the frustration by the lack of a single body of knowledge and prescribed best practices. In past frameworks, the body of knowledge was both a enabling tool and constraining limitation. Bodies of knowledge are difficult to keep current. For DevOps, let’s embrace the concepts of spirit of sharing, collaboration and continuous innovation by promoting an inclusive and collective body of knowledge of emerging practices.

If we settle on a set of DevOps best practices, are we actually inhibiting the very values that we are trying to instill? I hope not.
May 27, 2016
Enterprise Cassandra Deployments
Successful Cassandra deployment in the cloud takes a certain amount of skill and knowledge. In this blog I will share best practices for a successful Cassandra deployment in the cloud such as Amazon Web Services (AWS) and Google Cloud Platform (GCP). I will also discuss how you can protect and recover the data in your Cassandra cluster when deployed in the public cloud.

Cassandra Deployments in the Public Cloud

There are two main ways to have successful enterprise Cassandra deployments in AWS:
- Use a ‘managed’ Cassandra deployment. This is a reasonable approach; the only downside is the inability to customize data placement beyond what is typically provided by the vendor.
- Manage the cluster yourself, if you want to have full control over your Cassandra deployment and customize it to scale and perform according your business needs.
Please refer documentation from Datastax or Apache Cassandra for detailed best practices for Cassandra deployments. The following best practices are reiterated here to simplify deployment of a scalable Cassandra cluster in AWS.

Storage

Both AWS and GCP offer two types of local storage options for their compute instances. For AWS, the options are Elastic Block Storage (EBS) and Ephemeral Storage. In GCP, the options are Persistent Disks and Local SSD. Both options have their own trade-offs. Pick the one that best suits your deployment.

Persistent Storage

EBS volumes have performance characteristics similar to network storage such as a NAS or SAN. Because storage from EBS volumes is multitenant in nature, it is not suitable for distributed, log structured databases such as Cassandra. If you want to ensure reliable performance through EBS volumes, use “EBS optimized instances” with “provisioned IOPS.” Since the IO traffic to EBS volumes goes over the instance network, EBS optimized will provide reliable performance, but does not guarantee any increased performance. To get the most performance out of EBS volumes, use instances with multiple disks and separate the volumes used for “Data” and “Commitlog.”

If you have to use EBS Volumes, go with “EBS Optimized Instances,” with “Provisioned IOPS” and split data and Commit Log on different volumes.

Ephemeral Storage

Ephemeral storage is a great option to use for downstream Cassandra environments such as test/dev and staging. Since ephemeral volumes are typically SSDs, there is no need to separate the data and commitlog into separate drives. To optimize performance, choose EC2 instances with multiple ephemeral storage drives and stripe them together into one logical volume. Send data and commitlog to the logical volume. Remember that when you stop the EC2 instance, the data in Ephemeral storage will be deleted.

Stripe all Ephemeral drives into a single volume to get maximum performance.

Data Distribution

Cassandra uses Snitch to determine the topology of the cluster. The Snitch is also used to determine the data center and rack for each node in the cluster. Replication strategy is used to determine the placement of data across the different nodes in the cluster.

AWS
1. EC2Snitch works well for single region deployments.
1. Ec2MultiRegionSnitch works well for cluster distributed across multiple AWS regions. Please note, Multi-DC, Active-Active configuration makes your cluster resilient to region failures and for multi-dc cluster, use “local quorum” to improve performance.
Google Cloud Platform (GCP)
1. Use GoogleCloudSnitch for Cassandra deployments in GCP.
1. GoogleCloudSnitch treats regions as Cassandra data centers and availability zones as racks.
Other Features

Both AWS and GCP offer a series of operational capabilities that seem to make it easier to maintain high uptime on Apache Cassandra cluster. These capabilities have to be chosen carefully depending your deployment needs.
1. Load Balancing: Network-level load balancing service is offered by both AWS (Elastic Load Balancer) and GCP. It is not recommended to put a load balancer in front of Cassandra. It is better to use a client connector that can handle this, consider open source utilities such Presto and Astyanax
1. Autoscaling: Both AWS and GCP offer Autoscaling facility for compute instances. However, although this functionality looks very enticing, putting all C* nodes in an autoscaling group will not produce the desired result. Instead, create a separate autoscaling group of ‘1 node’ for every node in your cluster. This will ensure that anytime a node goes down, it is automatically brought back up. This will ensure the cluster data placement is not disturbed when nodes go down and nodes coming up do not cause excessive data movement.
Scalable Backups and Reliable Point-in-Time Recovery

Backing up your data is a must-have for enterprise organizations. The impact of data loss is so high that no enterprise wants to risk relying alone on replication. Protecting data in a public cloud environment may seem daunting, but there are several options available that can make this easier compared to an on-premise deployment. A simple option may look like one of the native snapshots:

Storage Snapshots

I have seen few customers use snapshots of EBS volumes as a way to protect the data in their cluster against logical and/or operational errors. This solution will work but with some severe limitations. On top of the overhead of maintaining the scripts to take snapshots, there is a large overhead in storage costs. Taking a snapshot of volumes does not allow for incremental backups of your data. This causes an exponential bloat in the amount of data stored in snapshots, leading to high cost of the backup solution.

Amazon S3 or Google Cloud Storage

Using an S3 bucket as the secondary storage enables enterprises to use low-cost, highthroughput storage, while reducing overall operating costs for storing backup data. Here are a few recommendations when using S3:
1. S3 Region: Amazon S3 buckets are region-specific. A bucket created in one region will provide better performance characteristics to other infrastructure in the same region. Subsequently, it is important to ensure the S3 bucket used as the secondary storage is in the same region as the production database being protected and the Datos IO cluster.
1. Storage Class and Lifecycle Policy: Amazon S3 allows users to create lifecycle policies on buckets. These policies dictate the performance and response time characteristics of objects in the bucket. STANDARD_1A is more suitable for longer lived and less frequently access data, and is not suitable as secondary storage. Additionally, do not turn on any lifecycle policies on the buckets used for the secondary storage. All buckets should be in the standard storage class.
Conclusion

As enterprises leverage the agility provided by public cloud environments, it’s important to understand the various facets of Cassandra deployments to ensure the efficient use of your money and time.
May 23, 2016
Data Protection for Operational Reporting Platforms
Making real-time business decisions based on data from social, mobile and cloud platforms is a requirement in today’s world. That’s why an ecosystem of next-generation operational reporting platforms such as Spark, Apache Cassandra, Kafka, Docker, Mesos, Marathon and more have come to life. Enterprises are using these technologies to re-architect their data storage and processing frameworks to achieve lower costs, scalability and faster response times. However, as enterprises onboard business-critical applications on this next-generation technology stack, they need to prepare for failures or errors that may disrupt customer-facing applications.

One of the key benefits that operational reporting platforms provide is real-time visibility, which allows businesses to make strategic decisions based on actionable insights. However, the new applications generate a large amount of data and archaic operational reporting platforms (those based on traditional data warehouses, data marts and ETL logic) are unable to catch up and, as a result:
- Business decisions are based on stale data
- Fragmented data increases the risk of errors
- Multiple copies of data increase storage costs
These issues have driven many progressive, data-centric enterprises to re-architect their operational reporting platforms. In this new architecture, Kafka serves as a persistent message bus that ingests data from relational stores and provides failure resiliency and message buffering. Spark or Spark Streaming may be used for transformation, aggregation and other lightweight stream processing. These processing nodes may be deployed in Docker containers. Finally, Apache Cassandra serves as a distributed data storage layer that provides linear scalability and high-availability. Kafka also enables the ability to add multiple data consumers such as Hadoop/HBase through Flafka.

However, as enterprises adopt operational reporting platforms, they want to ensure that bad data feeds and human error do not result in data loss. Apache Cassandra natively replicates data but that does not provide the capability to go back in time to recover data efficiently and at scale.

As enterprises onboard their critical applications to achieve development agility, scalability and lower operating costs, they must also explore data protection solutions that are specifically dedicated for scale-out databases.
May 18, 2016
Faster, Smarter DevOps

Whether you call it DevOps or not, if you are concerned about releasing more code faster and at a higher quality, the resulting software delivery chain and process will look and smell like DevOps. But for existing development teams, no matter what the velocity objective is, getting from here to there is not something that can be done without a plan.

Moving your release cadence from months to weeks is not just about learning agile practices and getting some automation tools. It involves people, tooling and a transition plan. I will discuss some of the benefits and approaches to getting there.

Waterfall to agile, agile to continuous integration, continuous integration to continuous deployment. Whatever your processes are, the theme is the same: Find a way to get code to users faster without sacrificing quality. But speed and quality are sometimes at opposition to each other. Going faster means things can break faster, and when we only think about DevOps as releases, it’s easy to fall into this trap.

Established development shops cannot just jump from one flow to another. Unless you start out net new, the goal is to introduce new processes without delaying releases for three months or more to do transition in lump. This often is done using a pincer approach that addresses both bottom-up tactics and top-down oversight and culture at the same time.

However, because adopting DevOps tools is so easy, the trend is to focus on tactics only and adopt from the bottom up without consideration of the entire pipeline. The outcome is release automation tools that dictate your delivery chain for you, and not the other way around.

Structured Automation

DevOps requires automation. But what is often not considered is automation that sustains and fits into the entire delivery chain. Governance, artifacts organization and inventory, metrics and security must be considered. If an organization establishes a vetting process for all new automation and how it fits into the pipeline’s orchestration, then new automation will support what exists today and in the future.

For example, many organizations driving down the DevOps path have encountered challenges when trying to incorporate practices from security or governance teams. Historically, these teams have resided outside of the Dev and Ops echo chamber and their processes were asynchronously aligned to the work being done. The challenge for many organizations is to determine the best ways to bring the people, processes and technology supporting these initiatives into the fold without slowing things down. The best organizations are finding new ways to automate policies from security and governance teams by shifting away from artisanal, asynchronous approaches to synchronous processes earlier in the life cycle.

Let’s take a look at an example of application security. A number of technology vendors in the application security arena are touting “automation” as key value point for their solutions to better fit them into a DevOps tool chain. In some instances, automation means that machines are now fully responsible for monitoring, analyzing and fixing security vulnerabilities for those applications at wire-speed. In other instances, automation implies facilitation of human-centric workflows that might represent hours or days of asynchronous analysis not fit for continuous operations. In both cases, the technologies may accomplish similar ends, but their approaches could be dramatically different.

Also, one solution might be built to support asynchronous investigations by a security professional, while the other might provide synchronous support to a developer at the design and build stages of the software development life cycle. Establishing a vetting process can help determine if the automation levels required by a team or process can truly be delivered before investments are made. It is also worth noting that layers of obscurity frequently exist within words such as “automation,” “integration,” “configuration,” and “continuous.”

There are other areas that also tend to get overlooked in building a DevOps environment. The complete story can be found here.

May 12, 2016