Granicus Issue Notification: Degradation of Stream Quality
Incident Report for Granicus System
Resolved
This issue has been resolved and all systems are operating normally.

Issue Summary & Impact: Degradation of Stream Quality
Product: Media Manager; Streaming Video
Timeline: On Sept. 26, 2019 Granicus teams became aware of an issue impacting the quality of live video streams for some of our govMeetings customers. During our investigation we identified that recent patching to our infrastructure was responsible for the streaming issues some clients were reporting.

With the root cause identified our engineers were able to update the infrastructure responsible for the degraded stream quality. We have been monitoring stream quality from the affected servers since those updates were applied and are confident that the issue has been resolved.

We apologize for any inconvenience this service issue may have caused and thank you for your patience.

If you have any questions or would like more information about this issue, please contact us at Support@granicus.com.

Thank you,
Granicus Customer Support
Posted Oct 09, 2019 - 12:01 MDT
Monitoring
Updated Notice of Disruption in Service for Granicus.

Issue Summary & Impact Update: Our engineers have identified that a recent update to our hosting environment was affecting the quality of video streams for certain segments of our distribution network. They have reverted the portion of the update which caused this degradation in stream quality and are now monitoring traffic to ensure that availability has been restored. We will provide additional updates on the validation of this change throughout the day.

Thank you for your patience as we work to ensure this matter is resolved.

Product: Media Manager, Streaming Video

If you have any questions or would like more information about this issue, please contact us at Support@granicus.com.

Thank you,
Granicus Customer Support
Posted Sep 27, 2019 - 09:35 MDT
Identified
Updated Notice of Disruption in Service for Granicus.

Issue Summary & Impact Update: Our engineers have identified a potential resolution to the stream buffering issue which some clients are experiencing. They are running additional tests to validate this solution. In the meantime, we are continuing to migrate affected client sites in order to mitigate any degradation of stream quality or availability.

Thank you again for your continued patience with this matter. We will provide additional updates as they develop.

Product: Media Manager, Streaming Video

If you have any questions or would like more information about this issue, please contact us at Support@granicus.com.

Thank you,
Granicus Customer Support
Posted Sep 26, 2019 - 15:06 MDT
Update
Updated Notice of Disruption in Service for Granicus.

Issue Summary & Impact Update: Our technical teams have resumed their investigation of this issue and are in the process of developing a longer term plan to mitigate this behavior for any affected clients. We will continue to provide additional updates as that work progresses.

Product: Media Manager, Video Streaming

If you have any questions or would like more information about this issue, please contact us at Support@granicus.com.

Thank you,
Granicus Customer Support
Posted Sep 26, 2019 - 09:20 MDT
Update
Updated Notice of Disruption in Service for Granicus.

Issue Summary & Impact Update: In an effort to reduce the impact this issue is having on video streams, our engineering teams are migrating clients to a group of stream servers which have not exhibited this behavior. Some clients may continue to see some intermittent performance degradation, but we believe this will help improve overall stream quality.
Our technical teams are continuing to investigate the issue and we will provide additional updates as that work progresses. At this time we do not anticipate any additional updates this evening.

Product: Media Manager, Video Streaming

If you have any questions or would like more information about this issue, please contact us at Support@granicus.com.

Thank you,
Granicus Customer Support
Posted Sep 25, 2019 - 15:54 MDT
Update
Updated Notice of Disruption in Service for Granicus.

Issue Summary & Impact Update: At this this time we have not identified any issues within the Granicus platform but our engineers are continuing to investigate this issue as our top priority. We have identified reports of connectivity issues for several major ISPs which may be contributing to disruptions in video streaming. We are reviewing all available options to mitigate potential ISP/connectivity issues, while also continuing the investigation into any potential causes within our own infrastructure. Currently we do not have an ETA for when we expect this issue to be circumvented but we will continue to provide updates as we have additional information.

We apologize for the inconvenience that this issue has caused, and thank you for your continued patience.
Product: Media Manager, Streaming Video

If you have any questions or would like more information about this issue, please contact us at Support@granicus.com.

Thank you,
Granicus Customer Support
Posted Sep 25, 2019 - 15:01 MDT
Investigating
Investigating
Granicus is currently experiencing a client-impacting service issue.

Issue Summary & Impact: Our technical teams are actively investigating reports of degraded performance for streaming video quality within the Media Manager platform.
Product: Media Manager, Video Streaming

If you have any questions or would like more information about this issue, please contact us at Support@granicus.com.

Thank you,
Granicus Customer Support
Posted Sep 25, 2019 - 09:00 MDT
This incident affected: govMeetings - Live Video, Legislative and Agenda (MediaManager, Live Video).