ISPs and Tech Support

I won't name my ISP, but this story unfortunately can apply to most of them.

Last Saturday, my Internet connection was working just fine. I went out for a few hours, came back, and went to look something up. Pages were taking a minute or two to load. I ran a speed test. It took awhile to get loaded, but once it loaded, it was fine. Speeds were good.

My first thought was DNS. So, I switched DNS servers. The problem was still there. I restarted my router. It was still there. So, I thought of latency and started doing some pings. That's when the problem became clear. Pings were low, but I was losing about 1 out of every 10 pings. I tried a few servers, including my ISP's page. They all showed packet loss. A quick test at http://pingtest.net confirmed a packet loss of 8%.

I started thinking my router might be having problems. So, I booted off a Linux live CD, cloned my MAC address to that of the router, and plugged my laptop directly into the ONT in my garage. I got an IP address and repeated the ping test. It showed 10% loss. I pinged my gateway, and it showed loss as well.

So, at this point, I figured I had done my homework. The problem was either my ONT, the ISP's gateway, or something in between. I tried resetting my ONT, but it didn't help. I decided it was time to call tech support. The problem was, I didn't have the number. Luckily, my connection was still somewhat working, and I managed to find it, after about 30 minutes of waiting for pages to load.

As soon as the support agent answered, I had a feeling I was doomed. It was Saturday night, and the call was obviously outsourced. Still, I had some faith in the system. The call went something like this:

Agent: How can I help you?

Me: I'm experiencing 10% packet loss between the ONT and the gateway. This is causing web pages to take several minutes to load, if they even load at all. I've already ruled out my router by connecting my laptop directly up to the ONT. I reset the ONT, but it didn't help. I'm going to need somebody to come out here and fix it.

Agent: I'm sorry you are having problems. I have a few things you can try.

At this point, I know I'm going to have to play along for a few minutes. After all, he doesn't know that I actually know what I'm talking about.

Agent: Could you try rebooting your router. Go to the router, disconnect power for 30 seconds, then reconnect it.

I know this won't work, but I play along anyway. "Ok. I'm doing that." My router was upstairs, and I was downstairs, so I figured about 90 seconds was enough time to have waited to tell him it was done "rebooting."

Agent: Can you try going to [link for the ISP's speed test]?

Me: Sure.

I waited for the page to load. While I'm waiting, I ran a few pings. Packet loss is up to 30%.

Me: It's still a not loading. It's a blank page. This isn't a speed problem, though. I'm seeing packet loss, and it's getting worse.

Agent: How long has it been like this?

Me: It was fine this afternoon. Three hours later, I saw some problems with 10% loss. I called you, and in the last 15 minutes, it went up to 30%.

Agent: Okay. Let me check with a network technician.

After 5 minutes on hold:
Agent: He wants you to connect your router back to the ONT, not your laptop.

Me: I already did that.

Agent: Okay. Try resetting your router. Power it off, hold down the reset button, and bring it back up.

I have plenty of port forwarding rules, firewall rules, custom DNS entries, and static DHCP entries. I'm not about to wipe all those out for something that's not even the router's problem. So:

Me: Okay, but I need to back up my settings first.

Agent: Oh, that would be a good idea.

Again, I figured a couple of minutes is enough time to have "reset" the router.

Me: Okay, it's reset. I'm trying the speed test page, and it's not loading at all now.

Agent: Let me check with the technician.

While on hold, I connected the laptop back up to see how much worse it had gotten. I didn't get an IP address. I rebooted my router. It couldn't get an IP address. I've reach 100% packet loss.

Agent: He's going to reprogram your ONT.

This sounded promising. I was wrong, but I did get my hopes up for a little bit.

Me: Okay. By the way, it's gotten worse. My router won't even get an IP address now. I'm seeing 100% loss and have a completely dead connection.

Agent: Okay. Please hold.

15 minutes later, he came back.

Agent: Okay. He wants you to reset the ONT. Unplug the power to the ONT for 1 minute.

Me: Sure. By the way, can you just connect me to the network technician so I don't have to keep waiting on hold.

Agent: Sorry, we don't have a way to do that.

Me (rolling my eyes at this point and getting frustrated): I'm resetting it. Do I have to disconnect the battery?

I knew the answer, but just had to ask.

Agent: No. Just the AC

BUZZ! Sorry, wrong answer. Unplugging the AC from a device with a battery backup inside it isn't going to do much.

Me: I unplugged it. The lights are still on.

Agent: One second.

On hold again.

Agent: Okay. Sorry. Unplug the battery too.

Me: Okay.

I unplugged the battery, and the ONT went dead. I waited 60 seconds, and plugged everything back in.

Me: Okay. It's coming up. The fail light is flashing, just like it should while it tries to connect....okay....it's still flashing. It should've connected by now...still nothing.

Agent: Okay. Please hold.

......

Agent: Hmm...can you try going to the speed test site?

Me: The ONT's not connecting. I won't be able to.

Agent: Can you try restarting the router?

Me: The ONT's not connecting. The router has nothing to do with it. My router's not even getting a link now. My connection is dead.

Agent: Please wait....

About 15 minutes later

Agent: The technician can't even reach your ONT. We're going to have to send somebody out.

FINALLY!

Me: Thank you.

Agent: Unfortunately, our ticketing system is down, so I can't create the ticket. Somebody will call you.

Me: When will that be?

Agent: I don't know. Our system is down.

Me: Well, it's Saturday night. If your system comes back up as soon as you hang up, will somebody be calling me tonight, or will it have to wait until Monday at the earliest?

Agent: Our system is down.

Me: Yes, I know, but will I get a call, say, tomorrow, if the system comes back up, or do you only call back during regular business hours?

Agent: During regular business hours.

Me: Okay. Thank you.

Agent: Is there anything else I can do for you?

I look at my phone. It's been an hour and 43 minutes since I called.

Me: No. It's been almost two hours as it is, so I don't think there's much else you can do.

Agent: Thank your for choosing [ISP]

So, at that point, after spending over 1 1/2 hours on the phone, it looked like my connection was going to be down awhile. They were going to send somebody out, which was what I asked for 2 minutes into the call.

Sunday morning, I turned on my cell phone. I had a voice mail from the ISP. They were sending somebody out at 2:00 and I needed to call back if that wouldn't work. Had I known they might call, I would have left the phone on. But, at least it was going to get fixed.

I went to the store. At 1:30, I got a call from the dispatcher saying that the technician was on his way. I started heading home. At 1:35, I got a call from the technician saying he was at my house. I told him I was 5 minutes away.

I rushed home, let him into the garage, and explained what I had been seeing. At this point, the ONT was dead. The first thing he did was replace it. It took him five minutes. I came back out in the garage and saw him staring at the ONT. The fail light was flashing. Apparently, the ONT wasn't the problem.

He stood there for a good five minutes, just staring at the pretty blinking light. Then, he started to talk to somebody on his phone. After he hung up, he said he was going to try re-authorizing the ONT and went back to his truck. He came back and said his connection from the truck was down and would have to call it in for somebody to do manually.

Irony.

A few minutes later, he rang the bell. He said the problem wasn't the ONT or in the box in the front yard. It was possibly in the hub, so he was leaving, but wanted me to know he was still working on it.

He finally came back and said it was fixed. It took an hour, but in this case, it was understandable. It was a problem in the central office. Normally, a CO problem would impact dozens of people, and they'd know the problem was there because of the high volume of calls. In this case, it was only me, so it didn't look like a CO problem. I guess I was just lucky.