Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

If you need help subscribing to Outage notifications just email support@helixtelecom.com and we will be happy to setup a subscription for you.

Infrastructure

About

Username
Infrastructure
Joined
Visits
162
Last Active
Roles
Member, Administrator, Moderator

Comments

  • Our system updates are complete.  Any customers who experience difficulty should email support@helixtelecom.com for assistance.
  • Service has been restored.  Any customers experiencing difficulty with their phones should disconnect power from the device, wait a few seconds, reconnect power and wait a few minutes.   If problems persist please call (855) 219-2700 x1 for assistan…
  • Several upstream ISPs have engineers on-site and are working to restore service as soon as possible as the outage is affecting many customers in the region.  The problem was reported to be "unpermitted construction".  Our monitoring systems report s…
  • We are currently experiencing degraded network performance issue due to a fiber connection problem near our primary data center. Our network engineers are actively working with our upstream providers to resolve the issue as quickly as possible.
  • Our staff has received a response and the issue appears to have been resolved.   Any customer still experiencing technical difficulty may need to reboot their device.  If problems persist please visit https://fixmyphonesystem.com and choose one of t…
  • Helix Telecom has isolated the outage to a portion of Century Link's network managed by Level 3 Communications.  Century Link customers, or other carriers who utilize Level 3 services, may be experiencing a loss of connectivity.   Helix Telecom has …
  • Helix Telecom staff completed applying security updates to a number of key systems today.   During this time we found logic problems in the call routing that only occur when both the primary and secondary database clusters are offline.  The problem …
  • The problem was isolated to a Denial of Service event.  Our ISP is investigating the source to help mitigate any potential problems in the future.   Customers who would like more information about what a Denial if Service event is can read this art…
  • Service has been restored but we are not closing this outage to provide our engineer more time to investigate the root cause.  Any customers still experiencing difficulty should disconnect power from their devices, wait a few seconds, reconnect powe…
  • The outage has been identified and resolved.  Any customers still experiencing difficulty should submit a ticket by visiting http://fixmyphonesystem.com.
  • Our primary internet service provider has been able to mitigate the problem but are still investigating the root cause. Customers may still experience connectivity issues but engineers are working on a resolution.
  • The outage has been found to be our primary internet/fiber connection to our facility.  We are working with our upstairs providers to resolve the problem or re-route internet traffic to a secondary ISP.
  • Our engineers made done additional changes and believe the problem has been resolved.  Any customer still experiencing difficulty should send an email to support@helixtelecom.com to make us aware of any dropped calls affecting more than one device s…
  • The change made by our engineers has not resolved the problem.  We are resuming our investigation.
  • Our engineers believe the problem has been resolved.   Any customer still experiencing difficulty should send an email to support@helixtelecom.com to make us aware of any dropped calls affecting more than one device simultaneously.   We are not clos…
  • The cause of this outage has been determined to be a malformed package received by one of our core routers.   The packet contents have been sent upstream to our router vendor and a patch has been applied to resolve the problem.  We are marking the o…
  • Our staff has completed restoration of the Automatic Call Recording server and synchronizing all pending recordings to the new server.   The failure was caused by a corrupted hardware raid controller on the storage server's disk array.  Our monitor…
  • The server responsible for storing all call recordings has failed.  A replacement has been ordered and is scheduled to be installed between May 9th and May 10th.  The process for recovering data from the failed server is ongoing and expected to tak…
  • Our engineers are still working to restore the Automatic Call Recording portal.    Call Recordings continue to be queued.  If an Authorized User on your account needs a specific call recording please submit a ticket including the estimated date, tim…
  • Hello Michaell, Please ask an Authorized User on your account to email this feature request to infrastructure@helixtelecom.com.  Please be sure to include the model of Grandstream phones you are using.  We are happy to update the phone configurati…
  • Helix Telecom staff have completed migration of the Recordings server.   This change has been published to all major ISPs worldwide but may require as long as 48 hours to update.   If you experience difficulty accessing Call Recordings please visit …
  • The call quality issues reported this morning were caused by a malfunctioning router.   Our staff power cycled the router and this resolved the issue.   We are still investigating the root cause of the outage but our staff faced a choice between res…
  • Helix Telecom Monitoring Systems show the issue has been resolved but we are still waiting for confirmation from our staff at the data center before clearing this outage.
  • Service had been restored.  We are investigating the cause of the outage further and we will post more information as soon as it becomes available.  We are sorry for the inconvenience and really appreciate all the kind words from customers who und…
  • We have identified the failed router causing the issue and working to repair the problem.  We should have more information shortly.
  • Our engineers are on-site.  The problem appears to be a failed or hung networking device.  We will post an update when we have additional information.
  • Customers with account numbers beginning with 200 have been restored to the previous DC.   Our staff finished duplicating all the servers to the new location and we will keep these servers ready for the remainder of the day in case the Verizon fiber…
  • Our staff was in the process of moving all of our servers when Verizon notified us the fiber repairs were completed.   They are still testing the repairs and slowly restoring service.  We will need a few more minutes to see if the repairs hold and t…
  • We have moved all customers with account numbers beginning with 200 and we believe those customers are working.   Other customers will follow shortly.   We are anticipating a complete loss of connectivity during the migration. 
  • Verizon is unable to provide us an ETA for a resolution to repair.   For this reason we are in the process of migrating all customers to a new data center.   This process will be invasive and lead to a temporary loss of service.   We will post more …