MSG91 Ups and Downs – Updates

Dear Customer,

When we started MSG91 9 years back, we started with a resolve to wow you with delightful service, and make messaging simple for your business. Over the years we have been able to build your trust enough to have most of you as our regular clients for more than 3 years. For that we are thankful.

As is true with all technical services – there are down times. We face issues that cause disruptions in our availability. Since we have been able to recruit a great team, that is dedicated to deliver delight, they get on to it, as soon as a issue surfaces. However, not all the situations we confront are dependent entirely on us. There are dependencies on server, infrastructure supporters and more. In such disruptions of service as well, we owe it to you, to keep you informed and updated. We want you to know we are doing everything in our capacity to solve every challenge, and make messaging smoother, and more effective for you, everyday.

In this attempt, we have created this page, where we will update every downtime and it’s resolution. We want you to know what we did, and when.

It covers incidences from December, 2018 –

8th Dec, 2018

Disruption start time- 9:30am (IST)

End time- 12:30pm (IST)

Fallback could not be activated and so all SMS in duration either delayed or failed.

The issue
Fault in our development and architecture level – a single point of failure

Action we are taking to prevent this in future

We are changing all single point of failures and slowing our development for time being.

Impact

Today we faced the biggest downtime in the last 5 years which was without fallback. It was our responsibility and we are extremely sorry to have this event.

5th Dec, 2018

Disruption start time- 4:10pm (IST)

End time- 7:50pm (IST)

Fallback was activated at 4:55pm to ensure your SMS delivery is not impacted. You can read more about fallback hereĀ https://help.msg91.com/article/272-what-is-fallback-and-how-it-works

The issue

AWS account got blocked because of some technical issues at AWS, exact issue will be shared by AWS team soon.

Action we are taking to prevent this in future

Building a DNS level fallback and making existing fallback better to reduce the activation time and including encryption in fallback.

Impact

All critical SMS were not delivered for initial 45 minutes and all other kind of SMSs have encountered 3 hours delay or they have not been delivered.

Since we faced this issue for the first time, we could call it bad luck, but we are striving to foresee such issues and deliver better handling mechanisms.

 

 

Views: 33

Leave a Reply

Your email address will not be published. Required fields are marked *