Any chance you guys measured this speed difference?
No, but we actually can get some comparison, since as I mentioned we host preview apps on Vercel! I did some quick tests right now, and it looks like based on logs, the API request to our Node backend takes:
- From Vercel: ~300ms worst case, ~40ms best case
- From within GCP: ~120ms worst case, ~20ms best case
These numbers are very approximate, but hopefully still insightful! And I'm not sure about the cause of the variance, there are quite a few links in the chain that could be at fault.
Was there a path considered to locally cache api responses on the vercel server or in middleware somewhere?
We do cache the responses using a CDN; adding another layer of caching beyond this wouldn't help much since at that point we need to fetch fresh results anyway.
There's probably a simple way to cache hot backend request/responses to get static-like speeds?
Yes, the CDN cache I mentioned is almost instant in good conditions (~2ms on my internet connection) - the problem is sometimes you do actually need updated data, and having huge variance between the cached response speed and the uncached response speed could be undesirable for users.
Comments
No, but we actually can get some comparison, since as I mentioned we host preview apps on Vercel! I did some quick tests right now, and it looks like based on logs, the API request to our Node backend takes:
- From Vercel: ~300ms worst case, ~40ms best case
- From within GCP: ~120ms worst case, ~20ms best case
These numbers are very approximate, but hopefully still insightful! And I'm not sure about the cause of the variance, there are quite a few links in the chain that could be at fault.
We do cache the responses using a CDN; adding another layer of caching beyond this wouldn't help much since at that point we need to fetch fresh results anyway.
Yes, the CDN cache I mentioned is almost instant in good conditions (~2ms on my internet connection) - the problem is sometimes you do actually need updated data, and having huge variance between the cached response speed and the uncached response speed could be undesirable for users.
thanks for those figures! It makes sense on the cache invalidation given the type of content Casual is serving.
It will be interesting once Vercel offers colo'd data sources and more with their edge rendering, which they just announced: https://vercel.com/blog/regional-execution-for-ultra-low-lat...