My back-end API’s are all running on a GoDaddy hosting account. Until yesterday, I was using one of their shared servers and the response times started to bother me.
The median response time for a call was about 760ms, which is bad in API terms. What is worse for me, is the disparity – the quickest results tended to be around 500 to 600ms, but then other calls would take up to 4 or 5 seconds to complete. I have some heartbeat checks which does nothing other than return a response, so these should be super-quick, but they were not.
I had a look around the internet and determined that part of my problem was that I had chosen one of the slowest frameworks out there (Slim v3) to build my API. To prove whether it was the framework or my server, I did two things: created a blank heartbeat which doesn’t use the framework (basically just “echo ‘ok’;”) and then also setup node.js to check the speeds. The results pointed to the server, not the choice of framework. Apart from requests within a few seconds of each other, “cold” requests would always take around 500ms.
Upgrading to business
The decision was made to upgrade to business hosting. Surprisingly little information is available on GoDaddy’s upgrade process other than a bunch of complaints about it not working. I was really unsure about how seamless it would be as I didn’t want to be spending the whole day troubleshooting. I half-did a bunch of backups and paid over the money.
About 20 or 30 minutes later, things were done. A quick check of my API’s showed that everything was running and everything appeared to have just worked…or so I thought.
The main two issues which happened to me were that my MySQL database was really slow after the transfer and my WordPress blog stopped working. The blog was fairly trivial to fix as some permissions had changed during the move (not sure why only on for my WordPress installation as everything else remained intact).
One other side issue which happened is that I didn’t disconnect one of my terminal connections. It became clear after a few small tweaks that I was on the old server, not the new.
Slow database
To be fair, most of yesterday is now just a bit of blur, but I noticed the slow database and tried to fix it. In my own inexperience, I ended up locking a bunch of tables (by running optimize on it repeatedly, renaming a 2GB table etc) and things just completely died on me.
A thorough search of the internet helped to kill of most of the processes which had locked up my tables. Stubbornly, two tables remained and I couldn’t see what was locking it, but generally everything else was resolved and my database was running ok apart from the two tables I couldn’t access.
The last suggestion is to just restart the MySQL service, but I do not have the access to do this, so I decided to phone GoDaddy support. I know people complain about them a lot, but I’ve previously had to deal with them (after they killed my SSH access) and it was a painless affair.
Just shoot me instead
The call wait time was short enough and I was soon answered by someone American sounding who seemed eager to help. I explained that I have two tables which are locked which means I cannot access them. This first got interpreted as me forgetting my password; no, I can access the database, I have two tables which are locked.
The things which frustrated me on the call were:
- The technician clearly managed to access my database as he called out my table names (this becomes important in a minute).
- The technician did not understand what a table lock meant or how to tell that a table was locked. Due to his own lack of knowledge, he kept trying to check with me if what I was saying was really the case or if I’m just another idiot user. Due to this, he had to escalate to his supervisor.
- My total database size is around 2GB and I have 2GB of RAM available to me on the server. They repeatedly tried to insist that I simply had to buy more RAM as this was not sufficient due to my database size. I wasn’t having any of it and just kept on refusing.
- I asked the technician if they were able to just restart the MySQL on that server and he then said that his supervisor just did this. I checked this and he did not restart the service. He kept explaining to me that it was the MySQL that was restarted, not the server. Yes, I was looking at it and I could see that MySQL was running for 18 hours already.
- He then told me that his supervisor suggested that I upgrade to a package with more RAM as my database was inaccessible due to a lack of RAM. I can see the memory usage on my server and only around 300MB was used. He now claimed that he was unable to access my database at all, despite reading me back my own table names earlier. Only two tables were locked, everything else was fine (I checked this in several ways: directly on MySQL, through some API calls and then also checking my logs of external calls with perfectly good responses).
I had to just end the call after 44 minutes because this guy and his supervisor were completely useless. He was clearly out of his depth and instead of owning up to it, he started making up stuff like the fact that your entire database has to fit into the memory available on the server.
While I was on the call, I decided to just rework some of my database any way. The only annoying thing is that I’ve lost around 12 hours worth of data as the two tables in question, are used to store the collected data.
New day
I got up this morning and checked things out and the locks are gone and everything is going swimmingly again. Except for the fact that I redesigned half my database to work around the locked tables. It was a useful exercise any way since I was saving way too much data. (Yes, the one table in question makes up 1.7GB of the 2GB database).
I managed to do queries on it and it is all just fine. In fact…
Rechecking all of my heartbeats and health check, I noticed a few things:
- The fastest responses times after I upgraded my package is now around 300ms – this compares to about 600ms for the fastest responses previously. This is a significant improvement.
- The median response time is now down to around 350ms to 400ms down from 760ms. Given what I was doing on the server yesterday, I expect this to go down, as these measurements are all from the time while I was doing large numbers of database queries.
- Checking this morning, some of my heartbeat responses are coming back below 100ms.
- Most surprising is how fast the response is from Node.js – 22ms for a heartbeat – I might just migrate everything to use Node.js instead as this is the speed I am after.
Executive summary
Here is a summary on my feelings about GoDaddy:
- Shared hosting is ok for absolute starters, but in hindsight, I should have just got started on business hosting.
- GoDaddy is okay in terms of their offering and I feel much better about them after upgrading to the business package.
- The company is being let down badly by their support staff. My call yesterday feels like a major run-in – it is also frustrating that both times I have called their support, they try to upsell to me. Support should be good at support, not sales.