Update - WashU IT is investigating reports of Microsoft Teams slow performance or users being unable to login. We are working with Microsoft to troubleshoot the issue, but currently they do not have a root cause identified.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 23, 2024 - 12:06 CDT
Investigating - WashU IT is investigating reports of Microsoft Teams slow performance or users being unable to login. Initial indications are that this may be an issue on the vendor's service rather than a WashU issue.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 23, 2024 - 09:35 CDT
Update - All services have been restored.

Teams are continuing to monitor service health and working with the vendor to determine root-cause.

Apr 19, 2024 - 13:43 CDT
Monitoring - WashU IT has introduced a possible fix for wireless and VPN issues.

Teams are continuing to monitoring the service. If you continue to experience further issues, please contact us at 314-933-3333.

We appreciate your patience during this incident.

Apr 17, 2024 - 08:00 CDT
Update - WashU IT is investigating sporadic reports of users being unable to connect to VPN and Wifi. Existing connections do not appear to be affected. The environment currently appears stable, but technical teams will monitor and take action if issues reoccur overnight. Additional troubleshooting and investigation will resume on Wednesday morning.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 16, 2024 - 22:47 CDT
Update - WashU IT is investigating sporadic reports of users being unable to connect to VPN and Wifi. Existing connections do not appear to be affected. Technical teams have observed indications that an issue is still occurring intermittently; troubleshooting continues, and a support case was opened with the vendor for additional assistance.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 16, 2024 - 19:46 CDT
Update - WashU IT is investigating sporadic reports of users being unable to connect to VPN and Wifi. Existing connections do not appear to be affected. Technical teams are troubleshooting and have a support case open with the vendor.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 16, 2024 - 18:23 CDT
Investigating - WashU IT is investigating reports of users being unable to connect to VPN and Wifi. Existing connections do not appear to be affected.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 16, 2024 - 17:02 CDT
Update - WashU IT and BJC technical teams are investigating issues with the Avaya One X Agent. We believe the current fix has mitigated user impact at least temporarily. We are monitoring the environment and will evaluate additional corrective actions next week.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 18, 2024 - 18:03 CDT
Update - The change over the weekend does not appear to have improved performance.

An additional change was performed this morning.

Teams will be performing additional testing to evaluate the effectiveness of the change.

Apr 15, 2024 - 11:45 CDT
Investigating - WashU IT and BJC technical teams are investigating issues with the Avaya One X Agent. Technical teams implemented a possible fix but found that the issue was not resolved. An additional change is being scheduled for Sunday 4/14; the next update will come on Monday 4/15 once the change has been evaluated for effectiveness.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 11, 2024 - 16:56 CDT
Identified - WashU IT and BJC technical teams are investigating issues with the Avaya One X Agent. Technical teams have identified a potential fix and are working to validate it with the vendors.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 10, 2024 - 18:06 CDT
Update - WashU IT and BJC technical teams are investigating issues with the Avaya One X Agent. Technical teams have continued to work with the Avaya vendor to test and troubleshoot the issue but have not yet been able to identify root cause. Our next step will be to escalate our support case with Cisco for help analyzing firewall and network configuration, and we will continue to work with Avaya as they continue their analysis of the data we have provided them today.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 09, 2024 - 18:01 CDT
Update - WashU IT and BJC technical teams are investigating issues with the Avaya One X Agent. After some technical discussion, additional troubleshooting is needed before we can continue working on a resolution. Technical teams will reconvene the call on Tuesday morning at 9am.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 08, 2024 - 17:06 CDT
Investigating - WashU IT and BJC technical teams are investigating issues with the Avaya One X Agent. An upgrade was performed in February and appeared stable initially. Since then, an increasing number of remote users on WashU VPN have been reporting issues with audio dropping and similar issues affecting their Avaya service.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 08, 2024 - 14:53 CDT
Update - WashU has provided the vendor with additional information.

The vendor is analyzing this information to determine next-steps.

There is no ETA at this time.

Apr 15, 2024 - 11:38 CDT
Investigating - WashU IT is aware of an issue affecting the automatic discovery processes within ServiceNow following a recent upgrade.

This issue affects a small number of users and is not business-critical.

The vendor is aware of the issue and working to resolve it.

Apr 09, 2024 - 11:29 CDT

About This Site

WashU IT's Service Dashboard gives you real-time updates on system performance issues. If you require assistance, please call our Service Desk at 314-933-3333.

Administrative & Enterprise Systems Operational
Alumni and Development Information System (ADIS) Operational
Research Management System (RMS) Operational
Student Information System (SIS) Operational
Vended Applications ? Operational
Cognos Reporting Operational
Marketplace Operational
Vended Applications ? Operational
Workday ? Operational
SUBS Awards ? Operational
Bear Bucks ? Operational
Back-end Processing (AutoSys) ? Operational
Confluence Operational
Archibus Operational
Mulesoft ? Operational
Clinical Operational
EPIC Operational
Vended Applications Operational
Clinical Websites Operational
Credit Card Systems ? Operational
Communications & Collaboration Major Outage
Email Operational
Office 365 Operational
Microsoft Teams Major Outage
Maestro mass communication system Operational
Zoom Operational
Box File Storage System ? Operational
WUSTL Connect ? Operational
Computing and Printing Operational
E-Fax Operational
PaperCut Printing Operational
WashU Cloud Storage Operational
Azure Hosted Infrastructure Operational
WashU IT Services / Cloud Computing ? Operational
Desktop Computing ? Operational
File Services Operational
IT Infrastructure Components ? Operational
JAMF Apple Device Management System ? Operational
IT Management Operational
Software Licensing Operational
ServiceNow ? Operational
Network & Connectivity Operational
Internet Operational
Internet 2 Operational
Phone Operational
VPN Operational
WiFi Operational
Washington University Research Network (WURN) Operational
Vendor Services (Spectrum, AT&T, etc.) ? Operational
Network Services ? Operational
Research Operational
Research Compute Operational
Storage Operational
Security Operational
Identity Management System (IDM Wustl Key) Operational
Spam Filter Operational
Duo Two-Factor Authentication ? Operational
Digital Guardian Operational
Teaching & Learning Operational
Canvas Operational
Learn@Work Operational
Kaltura ? Operational
Curriculum Management Applications ? Operational
EMS (Event Management System) ? Operational
University Websites Operational
WashU IT Hosted Websites Operational
Externally Hosted Websites Operational
BJC Devices and Services Degraded Performance
BJC Network Connectivity ? Operational
BJC Clinical Applications Operational
BJC Pharmacy POS Machines ? Operational
BJC Phone System Degraded Performance
BJC Handheld Supply Scanners (ATPAR) ? Operational
BJC Administrative Systems ? Operational
BJC Enterprise Client Virtualization (ECV) Operational
Cloud Services Operational
Google Operational
Engineering School Operational
Vendor Service ? Degraded Performance
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Past Incidents
Apr 23, 2024

Unresolved incident: Vendor Service Outage: Microsoft Teams.

Apr 22, 2024

No incidents reported.

Apr 21, 2024

No incidents reported.

Apr 20, 2024

No incidents reported.

Apr 19, 2024

Unresolved incident: Service Outage - High Severity: Intermittent Wifi and VPN Authentication.

Apr 18, 2024

Unresolved incident: Service Degradation - High Severity: Avaya One X Agent.

Apr 17, 2024
Apr 16, 2024
Apr 15, 2024
Resolved - All services have been restored.

Teams will continue to monitor service health

Apr 15, 16:24 CDT
Monitoring - A fix is in progress and some customers are seeing a restoration in service.
Apr 15, 15:25 CDT
Identified - BJC IT has identified a network event that corresponds with the current issue.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions.

We appreciate your patience as we work to restore service.

Apr 15, 14:38 CDT
Investigating - WashU IT is investigating an issue with wireless services in BJC buildings.

Teams are working to resolve the issue.

Please call 314-933-3333 if you have any questions.

Apr 15, 14:00 CDT
Apr 14, 2024
Resolved - WashU IT Engineers have resolved the issues with Splunk.

If you continue to experience further issues, please contact us at 314-933-3333.

We appreciate your patience during this incident.

Apr 14, 19:07 CDT
Update - Washington University IT has identified performance issues with Splunk logging as a result of overutilization. Technical teams continue to work on optimizing the Splunk utilization to restore normal service. While we have seen some improvements, additional work is needed before Splunk services could be considered stabilized.

Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 12, 20:03 CDT
Update - Washington University IT has identified performance issues with Splunk logging as a result of overutilization. Technical teams continue to work on optimizing the Splunk utilization to restore normal service.

Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 12, 17:26 CDT
Update - Washington University IT has identified performance issues with Splunk logging as a result of overutilization. Technical teams are working on deploying a different configuration to ease the load on the Splunk environment.

Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 12, 15:02 CDT
Identified - Washington University IT has identified performance issues with Splunk logging as a result of overutilization. Technical teams are working on implementing a workaround to restore normal operation.

Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 12, 13:09 CDT
Apr 13, 2024

No incidents reported.

Apr 12, 2024
Apr 11, 2024
Resolved - WashU IT and BJC Engineers have resolved the issue affecting the WUSM-secure wireless network on certain BJC access points. If you continue to experience further issues, please contact us at 314-933-3333.

We appreciate your patience during this incident.

Apr 11, 16:36 CDT
Investigating - WashU and BJC IT teams are investigating reports that users are unable to connect to the WUSM-secure wifi network on some BJC wireless access points at multiple locations in CAM and West County. The WUSM-secure SSID is not showing up on the list of available networks in those locations.

Teams are working to resolve the issue. Please call 314-933-3333 if you have any questions. We appreciate your patience as we work to restore service.

Apr 11, 16:22 CDT
Resolved - All services have been restored. If you continue to experience further issues, please contact us at 314-933-3333.

We appreciate your patience during this incident.

Apr 11, 08:28 CDT
Update - BJC has applied the fix, which has restored service for most users.

Teams are continuing to monitor service restoration for a small balance of users, and will be working with customer groups in the morning to validate full service restoration.

Apr 10, 16:58 CDT
Identified - BJC IT has identified the issue and is implementing a fix.

WiFi services in BJC buildings will be unavailable between 3:30 PM and ~4:00 PM as the fix is applied.

Apr 10, 15:28 CDT
Investigating - WashU IT is aware of an issue affecting BJC networks and WUSM wireless at BJC locations

BJC is aware of the issue and working to resolve it.

We appreciate your patience as BJC works to restore service.

Apr 10, 14:46 CDT
Apr 10, 2024
Apr 9, 2024