Guide to Redis on GCP : Memorystore

Until now I was happy inside my sweet little bubble of GCP where i used BigQuery for providing data warehouse and analytics services and if need be, switch to Cloud SQL for handling transactional services. I was having a happy life, until I came across a use case where time complexity was of utmost importance. Basically, the use case to expose data present in GCP through an API which responds back in under 100 ms. Which is like 1/10th of a second. And, to be frank in the start it sounded insane to me.

I had discussions with a lot of my colleagues regarding this use case and even whether designing such an API was possible or not. After having a lot of these discussions, it was at least clear to me that i will have to leave BigQuery aside for this use case and explore a faster data service which can help me achieve the needed performance. BigQuery is meant for storing large data and through it gives you query performance and everything, it will not give you the response in 100 ms. Also, another thing about BigQuery is that it might take variable amount of times to run the same query depending upon the current slots/compute being utilized inside your project.

Finally, someone suggested me to explore MemoryStore (Redis on GCP) which is a no-sql, in-memory database. And i thought that it made sense, since Redis stores all the data in its cache and is able to…