Searchable encryption means you can have encryption and queries (from apps etc) still work for authorized users.
This is all started when I was the CTO of a health-tech and the engineers all had access to patient data. They needed DB access (data migrations, managing backups, reading non-sensitive fields) but not access to the sensitive values. 10m+ patient records - it made me nervous.
Row-level Security (RLS) was a nightmare, it was slow, easy to mess up and could be overridden by DBAs anyway. Plus the application connected to the DB using a service account so RLS was useless for end-user access control.
CipherStash solves the following:
* Hide sensitive data from direct database accessors (authorized or not!)
My original use-case. Direct access to the DB doesn't reveal sensitive data (unless the user also has permission to decrypt a specific record).
* Support or even replace application access controls
Decryption happens in the application layer and is based on end-user identity. That means a gap in application defenses is mitigated by the encryption. If the user can't decrypt the value then they can't access the data.
* Access auditing
Every sensitive data access is recorded (and by who) which is very useful for compliance and incident response. SQL query auditing is great for knowing what queries were run but it doesn't tell you what data was actually returned. Its also blind once data actually leaves the DB and can inadvertently leak sensitive data itself. Using key accesses with non-identifying value IDs to audit decryptions makes the auditing more reliable and super granular.
We have customers who use the tech to prove to their users (often large enterprise) that no internal staff can access the data.
* Agent access to data
The encryption shifts access control decisions down to the data layer on a per-value basis. This is incredibly useful for gating access to agents that need access to data. Every decryption requires authentication (e.g via Supabase Auth, Auth0 etc) and combining it with agent identities (creds issued to agents) means you can not only limit what agents can access, you can also audit exactly what _was_ accessed.
The searchable encryption is the enabler: value level encryption is what makes all of this possible. Searchable encryption means you can have encryption and queries (from apps etc) still work for authorized users.
Hope that helps! I'm so in the weeds I don't know if I explain things that well sometimes!
CipherStash - specifically the key service. There is a lot of data for sure but we only record an identifier for each value and (optionally) the user ID. It compresses well.
Explanation:
The identifier is actually for the key that encrypts the value (1 unique key per value).
1. When the value is encrypted for the first time, it gets an ID.
2. When its decrypted, the application requests the key for that ID from the key-server (key-server records that the data was accessed)*
3. When updating, the same data key is used so the ID is persistent*
* Technically the key server doesn't return a key, it generates partial key material that can be used to derive the data key in the app. It can do this at up to 10,000 keys per second.
* You can also tag each value with the table/column name and the row-id to link everything together
The 3 values form the "descriptor" of a value: `table/column/id`.
So the audit log contains an ID for a key which could decrypt a value - does the audit log also contain the encrypted value itself? If not, how do you go from the audit log back to the original value in 1?
No, not by default. You could but as you said, that would be a A LOT of data.
It depends on your setup. If you're using Supabase, one way is to send the logs to Clickhouse and use the Clickhouse partner integration to query the audit logs and join it to the actual data.
Keeping only the ids in the audit log means you need to stitch the data together later. This means you don't accidentally leak data via your audit trail!
SQL query auditing is great for knowing what queries were run but it doesn't tell you what data was actually returned
With what you’re saying - sounds like cipher doesnt tell you what data was actually returned either - it will only tell you if the user could have received a decrypted version of the data right?
Query logging would tell you what queries were executed.
E.g the following would tell you that 10 user records were returned with name and email but not which users.
SELECT name, email FROM users LIMIT 10;
CipherStash can tell you the ID of every row that was returned, like:
If you need to know the actual values instead of just the IDs (say when doing an investigation), you can do a query to join the audit data with the users table.
Assuming audit logs are in a table called audit with a column called record_id and a timestamp:
-- full forensic analysis
SELECT name,email FROM audit
LEFT JOIN users ON users.id = audit.record_id
WHERE timestamp BETWEEN $1 AND $2;
And because the data is encrypted, that query will ALSO be audited. Not even admins escape the auditing :p
All of this is important for "materiality" investigations. If you suspect some data has been accessed inappropriately, knowing how much and exactly which records is very useful - especially if deciding if customer/regulator notification is required.
Comments
Searchable encryption means you can have encryption and queries (from apps etc) still work for authorized users.
This is all started when I was the CTO of a health-tech and the engineers all had access to patient data. They needed DB access (data migrations, managing backups, reading non-sensitive fields) but not access to the sensitive values. 10m+ patient records - it made me nervous.
Row-level Security (RLS) was a nightmare, it was slow, easy to mess up and could be overridden by DBAs anyway. Plus the application connected to the DB using a service account so RLS was useless for end-user access control.
CipherStash solves the following:
* Hide sensitive data from direct database accessors (authorized or not!)
My original use-case. Direct access to the DB doesn't reveal sensitive data (unless the user also has permission to decrypt a specific record).
* Support or even replace application access controls
Decryption happens in the application layer and is based on end-user identity. That means a gap in application defenses is mitigated by the encryption. If the user can't decrypt the value then they can't access the data.
* Access auditing
Every sensitive data access is recorded (and by who) which is very useful for compliance and incident response. SQL query auditing is great for knowing what queries were run but it doesn't tell you what data was actually returned. Its also blind once data actually leaves the DB and can inadvertently leak sensitive data itself. Using key accesses with non-identifying value IDs to audit decryptions makes the auditing more reliable and super granular.
We have customers who use the tech to prove to their users (often large enterprise) that no internal staff can access the data.
* Agent access to data
The encryption shifts access control decisions down to the data layer on a per-value basis. This is incredibly useful for gating access to agents that need access to data. Every decryption requires authentication (e.g via Supabase Auth, Auth0 etc) and combining it with agent identities (creds issued to agents) means you can not only limit what agents can access, you can also audit exactly what _was_ accessed.
The searchable encryption is the enabler: value level encryption is what makes all of this possible. Searchable encryption means you can have encryption and queries (from apps etc) still work for authorized users.
Hope that helps! I'm so in the weeds I don't know if I explain things that well sometimes!
On auditing, is that CipherStash or the application doing the auditing? Seems like your logs would be huge if it contained values of queries
CipherStash - specifically the key service. There is a lot of data for sure but we only record an identifier for each value and (optionally) the user ID. It compresses well.
So an identifier is kept for each column returned by a query?
If the value changes would you be able to get from the id you have audited back to the value that was returned at that point in time?
Not just each column, each value.
The answer to your question is yes.
Explanation: The identifier is actually for the key that encrypts the value (1 unique key per value).
1. When the value is encrypted for the first time, it gets an ID. 2. When its decrypted, the application requests the key for that ID from the key-server (key-server records that the data was accessed)* 3. When updating, the same data key is used so the ID is persistent*
* Technically the key server doesn't return a key, it generates partial key material that can be used to derive the data key in the app. It can do this at up to 10,000 keys per second. * You can also tag each value with the table/column name and the row-id to link everything together
The 3 values form the "descriptor" of a value: `table/column/id`.
So the audit log contains an ID for a key which could decrypt a value - does the audit log also contain the encrypted value itself? If not, how do you go from the audit log back to the original value in 1?
No, not by default. You could but as you said, that would be a A LOT of data.
It depends on your setup. If you're using Supabase, one way is to send the logs to Clickhouse and use the Clickhouse partner integration to query the audit logs and join it to the actual data.
Keeping only the ids in the audit log means you need to stitch the data together later. This means you don't accidentally leak data via your audit trail!
Ok Im having trouble reconciling this comment
With what you’re saying - sounds like cipher doesnt tell you what data was actually returned either - it will only tell you if the user could have received a decrypted version of the data right?
Sorry I was imprecise.
Query logging would tell you what queries were executed. E.g the following would tell you that 10 user records were returned with name and email but not which users.
SELECT name, email FROM users LIMIT 10;
CipherStash can tell you the ID of every row that was returned, like:
users;email,name;1 users;email,name;2 users;email,name;3 etc
If you need to know the actual values instead of just the IDs (say when doing an investigation), you can do a query to join the audit data with the users table.
Assuming audit logs are in a table called audit with a column called record_id and a timestamp:
-- full forensic analysis SELECT name,email FROM audit LEFT JOIN users ON users.id = audit.record_id WHERE timestamp BETWEEN $1 AND $2;
And because the data is encrypted, that query will ALSO be audited. Not even admins escape the auditing :p
All of this is important for "materiality" investigations. If you suspect some data has been accessed inappropriately, knowing how much and exactly which records is very useful - especially if deciding if customer/regulator notification is required.