Persistent Streams: The Internet With Ephemeral Storage

Abstract

Communication in the internet today is transient by default. Because of this, whenever an application needs to store data even for a moment, its provider needs to develop an application-specific solution, which is often done using client-server models. This is costly at scale, and generally requires users to concede control over the data they generate to allow application providers to generate revenue from the collected data to finance the operation of these servers. This also leads to a lock-in effect, which is prohibitive for new applications entering a market. To solve these issues, we propose persistent streams, an application-agnostic communication protocol that includes ephemeral and persistent storage and is able to handle both discrete as well as continuous (streaming) data. Including storage into the communication path removes the need for application servers completely. Even though the protocol relies on (cloud) servers as transmission and storage proxies, we expect the emergence of new storage technologies like non-volatile main memory to alleviate some issues this introduces. We also show the general applicability of this solution using different kinds of applications as examples. Overall, persistent streams have the potential to greatly reduce the burdens on application providers while also enabling users to exercise increased control over their data.