System Performance
Incident Report for ServiceChannel
Postmortem

Date of Incident: 02/25/2019

Time/Date Incident Started: 02/25/2019, 12:05 pm EST

Time/Date Stability Restored: 02/25/2019, 12:21 pm EST

Time/Date Incident Resolved: 02/25/2019, 12:21 pm EST

Users Impacted: Active users

Frequency: Intermittent

Impact: Partial

Incident description:

System performance throughout all systems was impacted by a increase in system response times and intermittent inability to use modules .

Root Cause Analysis:

Investigation via our monitoring systems revealed a larger than normal number of report requests.

Additional research revealed that these requests/reports had suboptimal execution plans and caused transient blocks.

Actions Taken:

We forced recompilation of the reports execution plans.

We will be doing in-depth review of underlying codes to adjust, optimize and stabilize performance.

The system was closely monitored for an additional next several days to ensure stability.

Mitigation Measures:

We are doing in-depth review of underlying codes to adjust, optimize and stabilize performance.

Posted 4 months ago. Feb 28, 2019 - 15:44 EST

Resolved
All services are confirmed running as expected. We consider this incident to be resolved.
Posted 4 months ago. Feb 25, 2019 - 16:32 EST
Investigating
We are monitoring elevated system activity that is contributing to degraded performance for some users. Updates will be sent as soon as possible.
Posted 4 months ago. Feb 25, 2019 - 12:44 EST
This incident affected: Call Center, Fixxbook, Service Automation, and Store Dashboard.