GKE & Service interruptions: degraded performance and platform instability
Incident Report for Proto
Resolved
GKE issue has been resolved.
However, we are currently experiencing new issues related to the aftermaths of this incident.
We will create a new status page for this implication.
Posted Oct 15, 2023 - 11:20 PDT
Update
We are continuing to monitor for any further issues.
Posted Oct 10, 2023 - 04:06 PDT
Update
We are continuing to monitor for any further issues.
Posted Oct 09, 2023 - 21:25 PDT
Update
We are continuing to monitor for any further issues.
Posted Oct 09, 2023 - 20:40 PDT
Update
GKE issue has returned.
The engineering team is currently monitoring, and resolving the issues as they occur.
During this process, you may experience disconnection/degraded performance.

We deeply apologize of the impact this repetition is causing for our clients.
Posted Oct 09, 2023 - 17:28 PDT
Update
The engineering team is continuing to monitor for GKE related issues.
We appreciate your patience during this time, and once again, apologize for any instabilities.
Posted Oct 09, 2023 - 15:52 PDT
Update
GKE issue is currently occurring repetitively.
The engineering team is currently monitoring, and resolving the issues as they occur.
During this process, you may experience disconnection/degraded performance.

We deeply apologize of the impact this repetition is causing for our clients.
Posted Oct 09, 2023 - 13:52 PDT
Update
Nats is fixed however it is still unstable on GKE, we will continue monitoring the performance of the platform.
Posted Oct 08, 2023 - 00:42 PDT
Update
GKE is still unstable and there are some issue on Nats, we are working on a fix and will continue monitoring.
Posted Oct 08, 2023 - 00:26 PDT
Monitoring
Our services have been fully restored and are now operating normally. We will leave the issue open and actively track the stability of GKE until we are confident that it is completely resolved.
Posted Oct 07, 2023 - 11:15 PDT
Identified
We have determined that the root cause of this issue which is due to a disruption in Google Kubernetes Engine in a few regions. https://status.cloud.google.com/
We are attempting workarounds and contacting Google Cloud for more info. Apologies for GCloud's service disruption affecting Proto.
Posted Oct 07, 2023 - 10:37 PDT
Investigating
We're receiving reports of service interruptions and experienced degraded performance on our platform. We are currently investigating the root cause of this issue.
Posted Oct 07, 2023 - 10:32 PDT
This incident affected: Platform (Chatbots, Livechat, Tickets, Analytics, Hosting, Company).