We're resolving this incident after redeploying and monitoring our redis clusters. We'll update this notice with next steps we're taking to improve our redis cluster and client health and resiliency.

We redeployed our redis clusters (worker and sentinels) and all downstream dependencies.

All 502 errors were resolved at 8:49:24 PDT.

We'll continue to monitor.

We've identified the issue.

Several of our redis clusters which are responsible for caching and message passing did not come up correctly after a GKE scheduled maintenance.

We're rolling out restarts to our redis clusters now.