tag:status.wmtu.fm,2005:/historyWMTU 91.9FM Status - Incident History2024-03-28T14:40:14-04:00WMTU 91.9FMtag:status.wmtu.fm,2005:Incident/99151842022-05-06T06:40:24-04:002022-05-06T06:40:25-04:00Thursday Scheduled Annual Power Outage<p><small>May <var data-var='date'> 6</var>, <var data-var='time'>06:40</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>May <var data-var='date'> 5</var>, <var data-var='time'>17:40</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>May <var data-var='date'> 4</var>, <var data-var='time'>17:03</var> EDT</small><br><strong>Scheduled</strong> - Starting at 5:45pm, we will be shutting off transmission in preparation for Michigan Tech's Annual Power Outage for preventative maintenance. This will last until approximately 6:15am the following morning when we restore FM radio transmissions. Radio can continue to be streamed online at www.wmtu.fm.<br /><br />Please refer to the following link for more details of this outage and the specific building affected: https://www.mtu.edu/facilities/resources/utilityoutages/<br /><br />Please email wmtu-tech with any questions or concerns. Thank you and we apologize for the inconvenience.</p>tag:status.wmtu.fm,2005:Incident/99151682022-05-05T06:40:23-04:002022-05-05T06:40:25-04:00Wednesday Scheduled Annual Power Outage<p><small>May <var data-var='date'> 5</var>, <var data-var='time'>06:40</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>May <var data-var='date'> 4</var>, <var data-var='time'>17:42</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>May <var data-var='date'> 4</var>, <var data-var='time'>17:02</var> EDT</small><br><strong>Scheduled</strong> - Starting at 5:45pm, we will be shutting off transmission in preparation for Michigan Tech's Annual Power Outage for preventative maintenance. This will last until approximately 6:15am the following morning when we restore FM radio transmissions. Radio can continue to be streamed online at www.wmtu.fm.<br /><br />Please refer to the following link for more details of this outage and the specific building affected: https://www.mtu.edu/facilities/resources/utilityoutages/<br /><br />Please email wmtu-tech with any questions or concerns. Thank you and we apologize for the inconvenience.</p>tag:status.wmtu.fm,2005:Incident/64128502021-02-27T05:42:29-05:002021-02-27T05:42:29-05:00Networking Maintenance<p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>05:42</var> EST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>03:42</var> EST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>03:41</var> EST</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.wmtu.fm,2005:Incident/60914942021-01-22T22:45:30-05:002021-01-22T22:45:31-05:00Music Storage Security Updates<p><small>Jan <var data-var='date'>22</var>, <var data-var='time'>22:45</var> EST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jan <var data-var='date'>22</var>, <var data-var='time'>22:15</var> EST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jan <var data-var='date'>22</var>, <var data-var='time'>22:02</var> EST</small><br><strong>Scheduled</strong> - We will be undergoing maintenance to apply updates to our music storage system. Audio might cut out briefly while this occurs. Expected outage is 15 minutes.</p>tag:status.wmtu.fm,2005:Incident/56518312020-11-25T13:21:37-05:002020-11-25T13:21:40-05:00Equipment Replacement<p><small>Nov <var data-var='date'>25</var>, <var data-var='time'>13:21</var> EST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Nov <var data-var='date'>25</var>, <var data-var='time'>13:17</var> EST</small><br><strong>Verifying</strong> - Verification is currently underway for the maintenance items.</p><p><small>Nov <var data-var='date'>25</var>, <var data-var='time'>11:00</var> EST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Nov <var data-var='date'>25</var>, <var data-var='time'>05:52</var> EST</small><br><strong>Scheduled</strong> - Between 11am and 2pm, we will be performing scheduled equipment replacement. During this time period, services may become briefly unavailable.</p>tag:status.wmtu.fm,2005:Incident/51465132020-09-21T22:30:31-04:002020-09-21T22:30:32-04:00Compatibility Upgrade<p><small>Sep <var data-var='date'>21</var>, <var data-var='time'>22:30</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Sep <var data-var='date'>21</var>, <var data-var='time'>21:36</var> EDT</small><br><strong>Verifying</strong> - Verification is currently underway for the maintenance items.</p><p><small>Sep <var data-var='date'>21</var>, <var data-var='time'>20:30</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Sep <var data-var='date'>21</var>, <var data-var='time'>19:38</var> EDT</small><br><strong>Scheduled</strong> - We need to perform some compatibility changes to some core equipment after prior upgrades. During this time, many services will be temporarily down and unavailable.</p>tag:status.wmtu.fm,2005:Incident/50805842020-09-13T07:06:50-04:002020-09-13T07:06:50-04:00Updating VM Host where MusicMaster and WideOrbit reside<p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>07:06</var> EDT</small><br><strong>Completed</strong> - Remote procedure failing, deferring update for physical in-place upgrade</p><p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>06:21</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>06:20</var> EDT</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.wmtu.fm,2005:Incident/42667922020-06-03T11:15:31-04:002020-06-03T11:15:32-04:00MTU Repairs<p><small>Jun <var data-var='date'> 3</var>, <var data-var='time'>11:15</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jun <var data-var='date'> 3</var>, <var data-var='time'>09:15</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jun <var data-var='date'> 3</var>, <var data-var='time'>08:08</var> EDT</small><br><strong>Scheduled</strong> - MTU facilities needs to perform some repairs near where our transmitter is. For safety, the transmitter will be shut off while maintenance is being performed.</p>tag:status.wmtu.fm,2005:Incident/42665122020-06-03T07:30:42-04:002020-06-03T07:30:42-04:00Looking into Stream VM Issues<p><small>Jun <var data-var='date'> 3</var>, <var data-var='time'>07:30</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jun <var data-var='date'> 3</var>, <var data-var='time'>07:24</var> EDT</small><br><strong>Update</strong> - We will be undergoing scheduled maintenance during this time.</p><p><small>Jun <var data-var='date'> 3</var>, <var data-var='time'>07:23</var> EDT</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.wmtu.fm,2005:Incident/39951542020-05-07T07:15:36-04:002020-05-07T07:15:55-04:00GLRC Network Link Upgrade<p><small>May <var data-var='date'> 7</var>, <var data-var='time'>07:15</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>May <var data-var='date'> 7</var>, <var data-var='time'>04:45</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Apr <var data-var='date'>29</var>, <var data-var='time'>16:42</var> EDT</small><br><strong>Scheduled</strong> - Michigan Tech IT will be performing Network Upgrades on Thursday, May 7 from 5am to 7am. In order to facilitate this change, WMTU will undergo a temporary service interruption. Please direct any questions or concerns to wmtu-tech@mtu.edu. Thank you.</p>tag:status.wmtu.fm,2005:Incident/38565932020-04-09T11:20:48-04:002020-04-09T11:20:49-04:00WMTU Networking issue<p><small>Apr <var data-var='date'> 9</var>, <var data-var='time'>11:20</var> EDT</small><br><strong>Resolved</strong> - Thanks to the MTU Telecommunications Team who were able to get onsite and help, WMTU is now operational again.</p><p><small>Apr <var data-var='date'> 9</var>, <var data-var='time'>04:20</var> EDT</small><br><strong>Identified</strong> - Early this morning, a networking issue was detected on our office uplink to MTU. Web services are still fully functional, with some services experiencing degraded performance. Broadcasting is currently down. We are monitoring the situation closely and attempting to get this resolved in a timely fashion with the COVID-19 restrictions. Please direct any questions to wmtu-tech@mtu.edu or wmtu-gm@mtu.edu. We apologize for the inconvenience and will provide updates when they are available. Thank you.</p>tag:status.wmtu.fm,2005:Incident/37245292020-03-20T09:00:31-04:002020-03-20T09:00:31-04:00Network Redundancy<p><small>Mar <var data-var='date'>20</var>, <var data-var='time'>09:00</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Mar <var data-var='date'>20</var>, <var data-var='time'>06:00</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'>19</var>, <var data-var='time'>19:58</var> EDT</small><br><strong>Scheduled</strong> - We will be adding network redundancy to our connection to Michigan Tech from our servers. We do not expect there to be any services affected during this maintenance.</p>tag:status.wmtu.fm,2005:Incident/35373572020-02-08T03:08:13-05:002020-02-08T03:08:13-05:00Maintenance on Database and Atlassian VM's<p><small>Feb <var data-var='date'> 8</var>, <var data-var='time'>03:08</var> EST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'> 8</var>, <var data-var='time'>02:08</var> EST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Feb <var data-var='date'> 8</var>, <var data-var='time'>02:07</var> EST</small><br><strong>Scheduled</strong> - We will be undergoing scheduled maintenance during this time.</p>tag:status.wmtu.fm,2005:Incident/33770952020-01-02T23:45:31-05:002020-01-02T23:45:35-05:00Michigan Tech IT Maintenance (GLRC)<p><small>Jan <var data-var='date'> 2</var>, <var data-var='time'>23:45</var> EST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jan <var data-var='date'> 2</var>, <var data-var='time'>21:45</var> EST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Dec <var data-var='date'>31</var>, <var data-var='time'>16:44</var> EST</small><br><strong>Scheduled</strong> - On Thursday January 2, 2020 at 10 PM Michigan Tech IT will be making a networking change in the GLRC. This will affect communication to our servers causing a brief outage to all WMTU Services.</p>tag:status.wmtu.fm,2005:Incident/32946562019-12-07T01:44:07-05:002019-12-07T01:44:09-05:00Server Upgrades<p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>01:44</var> EST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>00:17</var> EST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Dec <var data-var='date'> 7</var>, <var data-var='time'>00:16</var> EST</small><br><strong>Scheduled</strong> - We are performing some Equipment upgrades. All services will be unavailable temporarily.</p>tag:status.wmtu.fm,2005:Incident/31483792019-10-27T03:10:31-04:002019-10-27T03:10:31-04:00Rack Move<p><small>Oct <var data-var='date'>27</var>, <var data-var='time'>03:10</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Oct <var data-var='date'>27</var>, <var data-var='time'>00:10</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Oct <var data-var='date'>27</var>, <var data-var='time'>00:08</var> EDT</small><br><strong>Scheduled</strong> - We will be changing out our server rack. Services will be offline temporarily.</p>tag:status.wmtu.fm,2005:Incident/28777072019-08-17T17:15:32-04:002019-08-17T17:15:35-04:00Switch Maintenance Pt. 2<p><small>Aug <var data-var='date'>17</var>, <var data-var='time'>17:15</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Aug <var data-var='date'>17</var>, <var data-var='time'>16:15</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Aug <var data-var='date'>17</var>, <var data-var='time'>16:06</var> EDT</small><br><strong>Scheduled</strong> - We didn't get everything completed during our allotted time so we are going to extend the time a bit.</p>tag:status.wmtu.fm,2005:Incident/28734762019-08-17T16:00:34-04:002019-08-17T16:00:37-04:00Switch Maintenance<p><small>Aug <var data-var='date'>17</var>, <var data-var='time'>16:00</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Aug <var data-var='date'>17</var>, <var data-var='time'>14:00</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Aug <var data-var='date'>16</var>, <var data-var='time'>15:48</var> EDT</small><br><strong>Scheduled</strong> - We will be doing some switch maintenance before the start of the MTU 2019-2020 school year starts. This will require restarting our 2 Ubiquiti switches in the datacenter. Since our VM servers use the fiber network for VM storage we will be shutting down our servers during the switch maintenance period.</p>tag:status.wmtu.fm,2005:Incident/23544212019-05-12T20:03:24-04:002019-05-12T20:03:25-04:00Booth PC Maintenance and Upgrades<p><small>May <var data-var='date'>12</var>, <var data-var='time'>20:03</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>May <var data-var='date'> 1</var>, <var data-var='time'>12:18</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>16:30</var> EDT</small><br><strong>Scheduled</strong> - We will be upgrading our Booth PC to Windows 10 and doing upgrades on the booth systems before the end of semester. This will allow us to use our new storage system, and give us more control over the booth systems.</p>tag:status.wmtu.fm,2005:Incident/23268612019-04-13T21:20:36-04:002019-04-13T21:20:37-04:00Storage Maintenance Pt 2<p><small>Apr <var data-var='date'>13</var>, <var data-var='time'>21:20</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Apr <var data-var='date'>13</var>, <var data-var='time'>19:48</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Apr <var data-var='date'>13</var>, <var data-var='time'>19:47</var> EDT</small><br><strong>Scheduled</strong> - We got held up during the first time and did not extend the time.</p>tag:status.wmtu.fm,2005:Incident/23267362019-04-13T19:45:20-04:002019-04-13T19:48:18-04:00Storage Maintenance<p><small>Apr <var data-var='date'>13</var>, <var data-var='time'>19:45</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Apr <var data-var='date'>13</var>, <var data-var='time'>16:45</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Apr <var data-var='date'>13</var>, <var data-var='time'>16:36</var> EDT</small><br><strong>Scheduled</strong> - We will be doing some changes to our storage system which will require all systems using it to be shutdown. This will impact all digital/online services as we will also be working to move music storage over from the old music storage server to the new storage server.</p>tag:status.wmtu.fm,2005:Incident/23125722019-04-07T04:57:01-04:002019-04-07T04:57:01-04:00Physical Equipment Upgrades<p><small>Apr <var data-var='date'> 7</var>, <var data-var='time'>04:57</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Apr <var data-var='date'> 7</var>, <var data-var='time'>04:53</var> EDT</small><br><strong>Verifying</strong> - Verification is currently underway for the maintenance items.</p><p><small>Apr <var data-var='date'> 7</var>, <var data-var='time'>01:31</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Apr <var data-var='date'> 6</var>, <var data-var='time'>23:26</var> EDT</small><br><strong>Scheduled</strong> - We will be installing additional memory, configuring iDRAC services and making network configuration changes. Services will momentarily be affected as different components of the network are being shut down to perform maintenance.</p>tag:status.wmtu.fm,2005:Incident/22682072019-03-23T16:00:40-04:002019-03-23T16:00:43-04:00Data Storage Maintenance<p><small>Mar <var data-var='date'>23</var>, <var data-var='time'>16:00</var> EDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Mar <var data-var='date'>23</var>, <var data-var='time'>14:00</var> EDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'>19</var>, <var data-var='time'>00:42</var> EDT</small><br><strong>Scheduled</strong> - This Saturday, we will be migrating our servers to a more reliable file storage protocol. During this time, FM radio services will remain unaffected, but all web services will be temporarily unavailable. We will ensure each of our systems are brought back online as soon as possible to minimize downtime. We apologize for the inconvenience this may cause.</p>tag:status.wmtu.fm,2005:Incident/21678692019-02-09T22:58:42-05:002019-02-09T22:58:49-05:00Server Issues<p><small>Feb <var data-var='date'> 9</var>, <var data-var='time'>22:58</var> EST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'>27</var>, <var data-var='time'>00:32</var> EST</small><br><strong>Monitoring</strong> - We have successfully migrated all of our virtual machines to our new host. We will be monitoring to make sure everything is still good for a while, but we should be back up and running like normal.</p><p><small>Jan <var data-var='date'>26</var>, <var data-var='time'>01:29</var> EST</small><br><strong>Update</strong> - We have our storage migrated to the new host, and we are working on migrating VMs to the new VM host. We hope to be back up and running fully by tomorrow.</p><p><small>Jan <var data-var='date'>24</var>, <var data-var='time'>17:58</var> EST</small><br><strong>Identified</strong> - We believe the issue is with our main VM host, so we are in the process of transferring our files over to our new VM host and we will bring services back up when the transfer is finished.</p><p><small>Jan <var data-var='date'>24</var>, <var data-var='time'>16:30</var> EST</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Jan <var data-var='date'>24</var>, <var data-var='time'>16:28</var> EST</small><br><strong>Investigating</strong> - We have run into issues with our current VM host being unable to keep our storage mounted. We are not quite sure what the root cause is, and we are currently investigating. While the problems persist most of our services will be offline.</p>tag:status.wmtu.fm,2005:Incident/19551592018-10-18T08:57:39-04:002018-10-18T08:57:41-04:00Storage Server hardware failure<p><small>Oct <var data-var='date'>18</var>, <var data-var='time'>08:57</var> EDT</small><br><strong>Resolved</strong> - We have confirmed all of the hardware is working and our systems are back up.</p><p><small>Oct <var data-var='date'>17</var>, <var data-var='time'>22:28</var> EDT</small><br><strong>Monitoring</strong> - We have the new RAID card in and we are bringing up all of our servers now.</p><p><small>Oct <var data-var='date'>15</var>, <var data-var='time'>11:07</var> EDT</small><br><strong>Identified</strong> - We have narrowed the hardware issues down to a motherboard failure along with a RAID card failure. We have a new motherboard in place that is working without issue, and another RAID card is on the way. We should hopefully have everything back up and running like normal by the weekend.</p><p><small>Oct <var data-var='date'> 7</var>, <var data-var='time'>16:37</var> EDT</small><br><strong>Investigating</strong> - We are investigating issues with our storage server that stores and hosts all the data that is used for our automation and virtual server infrastructure. Until we are able to find the root cause and fix the issue our regular broadcasts and most other services will be down. This does not affect FM.</p>