In the first post in this series, we talked about how our old event system worked and some of the lessons we learned from operating it. In the second post, we covered the design of our new event delivery system, and why we choose Cloud Pub/Sub as the transport mechanism for all events. In this third and final post, we will explain how we intend to consume all the published events with Dataflow, and what we have discovered about the performance of this approach so far.
Whenever a user performs an action in the Spotify client—such as listening to a song or searching for an artist—a small piece of information, an event, is sent to our servers. Event delivery, the process of making sure that all events gets transported safely from clients all over the world to our central processing system, is an interesting problem. In this series of blog posts, we are going to look at some of the work we have done in this area. More specifically, we are going to look at the architecture of our new event delivery system, and tell you why we choose to base our new system on Google Cloud managed services.
In the first post in this series, we talked about how our old event system worked and some of the lessons we learned from operating it. In this second post, we’ll cover the design of our new event delivery system, and why we choose Cloud Pub/Sub as the transport mechanism for all events.
In this talk, Google will cover its pursue of a fair and meaningful Cloud benchmarking framework, PerfKit Benchmarker, from one of its performance engineers’ perspective. The talk will cover the challenges and pitfalls the team faced in defining what matters, in addition to common customer challenges, and share how they were tackled. It will also cover sampling challenges, processing, and storage of 3K samples/second, and the challenge to mine and visualize the data in a meaningful way.
This talk will present a high level view of Network efforts at Google including datacenters, cloud, and content delivery. It will then move into more detail on the issue of content delivery and how Google builds and operates its CDN infrastructure.
In this whitepaper, you will find more detail on Google’s key hierarchy and root of trust, as well as information on the granularity of encryption in specific GCP services for data at rest (this document does not cover encryption in transit). For all Google products, we strive to keep customer data highly protected, and to be as transparent as possible about how we secure it.
The content contained herein is correct as of August 2016, and represents the status quo as of the time it was written. Google Cloud Platform’s security policies and systems may change going forward, as we continually improve protection for our customers.
In working to keep cloud computing users’ data safe, we observe many threats—malware on the client, attacks on ssl, vulnerabilities in web applications, rogue insiders, espionage—but authentication related issues stand out amongst the biggest. When trying to help hundreds of millions of people from an unbelievable variety of endpoints, attitudes, and skill levels, what can possibly displace plain old passwords? No single thing, nothing overnight, and nothing perfect. A combination of risk-based checks, second-factor options, privacy-enhanced client certificates, and different forms of delegation is starting to find adoption towards making a discernible difference.
Offering strong data protection to cloud users while enabling rich applications is a challenging task. Researchers explore a new cloud platform architecture called Data Protection as a Service, which dramatically reduces the per-application development effort required to offer data protection, while still allowing rapid development and maintenance.