Thanks to Rachel and Vivek for feedback and review.
Data privacy is not binary. The extent to which a user trusts an application with their private data is a spectrum, and there are many forms that trust can take, each with its own tradeoffs. On the one hand, you have the least trustworthy applications which make no guarantees on user privacy and actively sell or share user data. On the other, you have the most trustworthy applications which literally by design cannot break data privacy, for example by being able to be run completely offline. Generally speaking, the higher you go up the hierarchy to more trustworthy app designs, the more constrained the app developer is in terms of functionality, performance, or overall design space.
Here is the hierarchy from bottom to top:
Level 0: No trust guarantees
The most common category, these apps make no guarantees towards user data privacy. Instead, they may actively sell or share user data.
Level 1: Trust me bro
This type of app promises to not sell or share user data, and makes it a prominent part of the app’s copy or marketing. However, the code is entirely closed source, data is stored on the server, so there is no way to verify what the app is actually doing with user data.
Level 2: Legal/regulatory
The next level of trust comes from the user believing in data rights because of legal or regulatory obligations. This is not a universally well defined category, but for example a user of a app in the EU might reasonably believe their data will be protected under GDPR. HIPAA and SOC2 are other common regulatory and compliance frameworks that govern data privacy. Here the user trusts the legal system to enforce their data privacy, which in most cases is more trustworthy than the developer’s word.
Level 3: Secure hardware
These apps are built so that user data is processed in secure hardware, which in most cases means it is opaque to the application developer. In addition, the user trusts that experts have verified the code running actually enforces this guarantee. Here the user is trusting the company to deploy the secure hardware setup correctly, the verification expert to audit their code, and the manufacturer of the secure hardware itself, i.e. Intel for SGX. Georgios Konstantopoulos has a great article further detailing the 5 levels of trust within secure hardware.
Level 4: Cryptography
This level of trust involves apps that are built using cryptography. For example, a user of an app built on secure multiparty computation can verify that their data is being split into secret shares where they must consent in order for computation to be done on their data. Here, the user can verify that the cryptography scheme is secure and that their client is running the correct code in order to be sure their data is safe.
Level 5: Local-first
The most secure category of applications involve those that don’t even need a server. User data is processed locally and no data leaves the device. Here, the user can be sure that their data is safe by for example turning on airplane mode and running the application. The downside of these apps are that they are limited in design scope, as for example an app which runs completely offline is not able to fetch data from other users over the Internet.
It’s important for developers to balance building increasingly trustworthy applications with the constraints and overhead trust has on the app development process. It would be great if all apps were local-first and guaranteed users full ownership of their data, but this is simply not feasible for many apps users want, both for technical and business reasons. One can hope that the dramatically advancing field of programmable cryptography will offer practical options for developers to build performant, privacy-preserving apps.