Platform slowness and instability

Incident Report for Proto

Postmortem

Last week, our platform experienced a disruption due to an interruption in the connection between services, resulting in some pages on the platform being unable to load and delays in bot replies. These issues had a significant impact on platform usability, affecting user interactions and overall functionality. Upon investigation, it was determined that the root cause of the issue stemmed from the service interruption, prompting immediate action to address the underlying connectivity issue.

Our team has implemented a fix to restore the connection between services and alleviate the performance degradation. Additionally, several identified issues were promptly addressed and resolved during the resolution process. Notably, optimizations were made to enhance the responsiveness of bot replies, improving overall platform performance. Moving forward, we are committed to implementing proactive measures to prevent similar incidents and ensure the stability and reliability of our platform for all users. We appreciate your patience and understanding throughout this process and encourage you to reach out to our support team if you encounter any further issues.

Posted Feb 06, 2024 - 05:18 PST

Resolved

Recent fixes have been released throughout the week, several functionality issues have been addressed ensuring a smoother experience for all users. As of now, the platform is fully operational, and you can expect all functionalities to be working as usual.

Posted Feb 06, 2024 - 05:12 PST

Update

We will be working on an upgrade of our server which might cause some instability on the platform. Thank you for your patience as we work to resolve these issues promptly.

Posted Jan 28, 2024 - 22:11 PST

Update

A round of fixes has been rolled out to restore stability to the platform. Currently, we are preparing another release to ensuring long-term stability.

Posted Jan 27, 2024 - 21:57 PST

Update

The Build and Track pages are still experiencing instability. We are continuing to implement a solution for this issue. We plan to roll out these improvements later today.

Posted Jan 26, 2024 - 16:30 PST

Identified

The team has successfully identified the issue, and we are now proceeding with the second round of fixes today.

Posted Jan 25, 2024 - 17:52 PST

Monitoring

The issue has been identified and a fix is being implemented. We will remain in a monitoring state as we implement improvements to our architecture.

Posted Jan 25, 2024 - 08:08 PST

Update

We're addressing performance issues on our platform to ensure optimal functionality. Thank you for your patience as we work to resolve these issues promptly.

Posted Jan 24, 2024 - 07:34 PST

Update

We are continuing to address occasional instability on our platform. We will remain in a monitoring state as we implement improvements to our architecture.

Posted Jan 24, 2024 - 01:48 PST

Update

We're addressing performance issues on our platform to ensure optimal functionality. Thank you for your patience as we work to resolve these issues promptly.

Posted Jan 24, 2024 - 01:47 PST

Investigating

The platform is experiencing slowness and occasional failed requests. We are currently investigating this.

Posted Jan 23, 2024 - 23:02 PST

This incident affected: Platform (Chatbots, Livechat, Tickets, Analytics, Company).