London Bus Pal Blog – Page 3 – Practical tips for mobile application management

GoDaddy hosting upgrade

My back-end API’s are all running on a GoDaddy hosting account. Until yesterday, I was using one of their shared servers and the response times started to bother me.

The median response time for a call was about 760ms, which is bad in API terms. What is worse for me, is the disparity – the quickest results tended to be around 500 to 600ms, but then other calls would take up to 4 or 5 seconds to complete. I have some heartbeat checks which does nothing other than return a response, so these should be super-quick, but they were not.

I had a look around the internet and determined that part of my problem was that I had chosen one of the slowest frameworks out there (Slim v3) to build my API. To prove whether it was the framework or my server, I did two things: created a blank heartbeat which doesn’t use the framework (basically just “echo ‘ok’;”) and then also setup node.js to check the speeds. The results pointed to the server, not the choice of framework. Apart from requests within a few seconds of each other, “cold” requests would always take around 500ms.

Upgrading to business

The decision was made to upgrade to business hosting. Surprisingly little information is available on GoDaddy’s upgrade process other than a bunch of complaints about it not working. I was really unsure about how seamless it would be as I didn’t want to be spending the whole day troubleshooting. I half-did a bunch of backups and paid over the money.

About 20 or 30 minutes later, things were done. A quick check of my API’s showed that everything was running and everything appeared to have just worked…or so I thought.

The main two issues which happened to me were that my MySQL database was really slow after the transfer and my WordPress blog stopped working. The blog was fairly trivial to fix as some permissions had changed during the move (not sure why only on for my WordPress installation as everything else remained intact).

One other side issue which happened is that I didn’t disconnect one of my terminal connections. It became clear after a few small tweaks that I was on the old server, not the new.

Slow database

To be fair, most of yesterday is now just a bit of blur, but I noticed the slow database and tried to fix it. In my own inexperience, I ended up locking a bunch of tables (by running optimize on it repeatedly, renaming a 2GB table etc) and things just completely died on me.

A thorough search of the internet helped to kill of most of the processes which had locked up my tables. Stubbornly, two tables remained and I couldn’t see what was locking it, but generally everything else was resolved and my database was running ok apart from the two tables I couldn’t access.

The last suggestion is to just restart the MySQL service, but I do not have the access to do this, so I decided to phone GoDaddy support. I know people complain about them a lot, but I’ve previously had to deal with them (after they killed my SSH access) and it was a painless affair.

Just shoot me instead

The call wait time was short enough and I was soon answered by someone American sounding who seemed eager to help. I explained that I have two tables which are locked which means I cannot access them. This first got interpreted as me forgetting my password; no, I can access the database, I have two tables which are locked.

The things which frustrated me on the call were:

The technician clearly managed to access my database as he called out my table names (this becomes important in a minute).
The technician did not understand what a table lock meant or how to tell that a table was locked. Due to his own lack of knowledge, he kept trying to check with me if what I was saying was really the case or if I’m just another idiot user. Due to this, he had to escalate to his supervisor.
My total database size is around 2GB and I have 2GB of RAM available to me on the server. They repeatedly tried to insist that I simply had to buy more RAM as this was not sufficient due to my database size. I wasn’t having any of it and just kept on refusing.
I asked the technician if they were able to just restart the MySQL on that server and he then said that his supervisor just did this. I checked this and he did not restart the service. He kept explaining to me that it was the MySQL that was restarted, not the server. Yes, I was looking at it and I could see that MySQL was running for 18 hours already.
He then told me that his supervisor suggested that I upgrade to a package with more RAM as my database was inaccessible due to a lack of RAM. I can see the memory usage on my server and only around 300MB was used. He now claimed that he was unable to access my database at all, despite reading me back my own table names earlier. Only two tables were locked, everything else was fine (I checked this in several ways: directly on MySQL, through some API calls and then also checking my logs of external calls with perfectly good responses).

I had to just end the call after 44 minutes because this guy and his supervisor were completely useless. He was clearly out of his depth and instead of owning up to it, he started making up stuff like the fact that your entire database has to fit into the memory available on the server.

While I was on the call, I decided to just rework some of my database any way. The only annoying thing is that I’ve lost around 12 hours worth of data as the two tables in question, are used to store the collected data.

New day

I got up this morning and checked things out and the locks are gone and everything is going swimmingly again. Except for the fact that I redesigned half my database to work around the locked tables. It was a useful exercise any way since I was saving way too much data. (Yes, the one table in question makes up 1.7GB of the 2GB database).

I managed to do queries on it and it is all just fine. In fact…

Rechecking all of my heartbeats and health check, I noticed a few things:

The fastest responses times after I upgraded my package is now around 300ms – this compares to about 600ms for the fastest responses previously. This is a significant improvement.
The median response time is now down to around 350ms to 400ms down from 760ms. Given what I was doing on the server yesterday, I expect this to go down, as these measurements are all from the time while I was doing large numbers of database queries.
Checking this morning, some of my heartbeat responses are coming back below 100ms.
Most surprising is how fast the response is from Node.js – 22ms for a heartbeat – I might just migrate everything to use Node.js instead as this is the speed I am after.

Executive summary

Here is a summary on my feelings about GoDaddy:

Shared hosting is ok for absolute starters, but in hindsight, I should have just got started on business hosting.
GoDaddy is okay in terms of their offering and I feel much better about them after upgrading to the business package.
The company is being let down badly by their support staff. My call yesterday feels like a major run-in – it is also frustrating that both times I have called their support, they try to upsell to me. Support should be good at support, not sales.

London Bus Pal

It broke!

Post author By admin
Post date 18/05/2019
No Comments on It broke!

I have a post ready to tell you all about the new functionality that I launched for bus information, but during a migration to a better service today, it broke a bit. This doesn’t affect anything except for the additional bus information telling you which routes the buses run on historically. All the prediction stuff continues to be stable.

Once fixed, we will continue.

London Bus Pal

London Bus Pal v4.4.1

Version 4.4.x introduces more bus information into London Bus Pal.

Firstly, I tried to keep the design of all of the functionality to ensure that it is unobtrusive to users not interested in the information, but still easily enough to access for those who are.

Changes which can be observed in the main functionality:

When searching for a specific bus, type-ahead is now available and the app will make suggestions when you start typing either a bus registration number or a fleet number. The type ahead information will also indicate the fleet number and operator.
Rather than just displaying a “no information or not running” message which is unclear, if you search for a bus and it is not in service, the app will show you this message quite clearly now.

New bus information screens:

Once you have found a bus, whether in service or not, you can access the information screens. For in-service buses, an information icon is now displayed in the top right-hand corner and for out of service buses, a clear button is available to show you bus information.
On the bus information screen, there are three tabs available:
- Assembly: this tab gives information about the fleet number and operator. If additional information is available about the bus assembly (such as engine, number of doors, chassis etc), it will be displayed here.
- Usage: this tab will show you which routes the bus has been used on over the last 7 days. Please note that these are estimates based on information interpreted from TfL. These numbers should be used as indicative rather than specific.
- Photos: the photos tab will show you publicly available photos from Flickr for this bus.

This functionality is in a pilot phase at the moment and subject to change. I have some rough plans on what I want to do with this functionality in future, but the purpose of this pilot is to see what the uptake is like and then maybe expand it out a bit further.

At the time of writing this, we have the information available for over 10,000 buses. Only around 750 of those have missing assembly details and I expect to start completing this over the coming months (of course, this will be an ongoing task).

Some plans for the future which I may look at are:

Bus lists of interest (rare workings, new buses, old buses and so on).
Personal bus lists – ability to put buses into your own personal groups; this can then later be used to see where all these buses are.
Route information – based on the information collected at the moment, it becomes possible to see how many buses are working any route at a point in time
Crowd-sourcing vehicle information – all fleet number and operator data is currently managed by myself; this is not sustainable in the long-run.
And many other ideas (some which I just don’t want to mention right now or others which might come from my users).

The roll out of this version is underway at the moment. I am expecting the Android roll out to be complete in a couple of days and iOS might take a little bit longer due to Apple’s requirements (and a technical issue which I experienced trying to create a build).

Tags release notes

Development London Bus Pal

Flutter – the honeymoon is over

Post author By Mulder Digital
Post date 15/04/2019
4 Comments on Flutter – the honeymoon is over

In January, I created a post where I sang Flutter’s praises over Android native. Three months later, does this still hold true?

First, let me give you a brief background. I launched my app back in 2013 on Android and the app has naturally grown from there. Late in 2018, I decided to do a full rewrite of the app in Flutter (with two things in mind: make the app more future-proof and ability to expand to iOS). I soft-launched on iOS in mid-December and went onto Android at the end of December. Despite a few hiccups (such as forgotten or missed features!), it generally went okay.

As of today, only about 6% of my Android user base is still on the old Android native app. I have around 2700 active users a day, so not massive, but it is a well-used app. Almost 4 months of constant hammering by users should give me a good flavour of what it is like to run a Flutter app in production.

What is still good?

Apple (almost for free): Being able to launch my app on Apple is definitely worthwhile. Revenue and use isn’t exactly where I want it to be, but I’m on the platform, so that is 100% further ahead than I was before Flutter. Apart from the nuances in terms of the iOS UX, I haven’t experienced any difference between the two platforms. It is great to do all of my development on Android and it just works on Apple.

Rapid developement: I have managed to add several new features to my app (including some fairly complex new features) which I would never have been able to achieve so quickly in Android native. And I also got this on Apple, almost for free. It may also be a sign of how well I architected my app, but this is also because Flutter and Dart allows this.

Community: The Flutter community is big and growing exponentially. Help is always at hand and I see new tutorials and walkthroughs coming out daily.

What’s not so good?

Dependency hell: I have found myself stuck in some dependency hell. I have one library which hasn’t updated and it causes a bad domino effect. I thought it was easier to manage in Dart, but appears not so. It took a fair amount of effort to unpick bit by bit and find the offending library.

Incomplete Google features: Further to the dependency hell I mention above, Google’s own features are still incomplete. Their maps are not usable yet, so I’ve had to use another package called “flutter_maps”, which is currently causing my dependency hell.

More incomplete Google features: AdMob is not really incomplete, rather just incompatible with the way that Flutter is meant to work. The library hasn’t been updated since February and I have to use awkward workarounds. I’ve had a really good explanation by Andrew Brogdon after I raised the issue on a Flutter Youtube video, but it is still frustrating that we are where we are with this. Nowhere.

Null handling: I’m not sure if it recently changed or if I just didn’t notice before, but I preferred the way that Dart handled nulls in that it won’t fatally blow up and just carry on. I had some really bad issues recently where it would just stop running parts of my code when I accidently tried doing something with a null. No errors, no nothing. It would literally just skip the rest of the function without throwing anything. It took hours to unpick and I would honestly have preferred the app to just crash instead.

Instability: I started using the stable channel of Flutter, so that I could keep up to the latest versions of Flutter, but without too much risk of things going wrong. Unfortunately, a fairly bad regression issue has appeared and it is causing some frustration (mostly just that Samsung decided to suspend my app).

Other weird bugs: This issue has appeared most than 4000 times in my app for more than 350 users. It has proven impossible for me to reproduce, but I can see it happening. I’ve had a number of other weird issues also occur. A more stable framework would not have these sorts of issues.

Do I still prefer Flutter?

Absolutely, I still prefer Flutter. I know that on balance, it appears from the above lists that I have more complaints about Flutter than praise, but this is not accurate. I simply recognise the current flaws with Flutter, but I still believe that the future is bright. Using Flutter, I believe that I can keep on expanding my app and giving my users all of the great experiences they are after.

Tags android development, flutter

Development London Bus Pal

Monetisation – the results

Post author By Mulder Digital
Post date 13/04/2019
No Comments on Monetisation – the results

It’s been a while since I posted here, mostly because I’m busy behind the scenes with huge new features. It will take a while, so I can’t promise or really say anything yet.

Today I had to do a bit of housekeeping and had a look at how sales were doing. It’s has been about 2.5 months since I started my in-app purchases, so I can finally start to see how these are performing in comparison with ad revenue and sales revenue. The breakdown in percentage from 1 February until today is as follows:

Advertising revenue: 95.7%
In-app purchases: 3.2%
Mobile app sales: 1.1%

The thing which is not obvious here, is that there hasn’t been any obvious drop-off in advertising revenue, although, it would be very difficult to tell, since advertising revenue is a little bit volatile any way. Revenue is up by 3% vs the previous 28 days, but it can be down 5% next month for reasons beyond my control.

I see in-app purchases as an additional revenue source, rather than cannibalising my advertising revenue.

Another interesting metric I could look at, is how Android compares to iOS, however, my iOS take up is still fairly low, so it would be an unfair comparison. I simply don’t have enough data points to really say.

Feel free to get in contact if you want to know more – I’m always happy to talk to other developers and share any experiences I may have had. Back to work for me then…

London Bus Pal

London Bus Pal v4.3.1

The next release is here. I fixed a few small issues which I picked up on the journey planner (for example, if you asked for a route where it wasn’t possible to see it on a map, it would mostly just break!).

For the rest of it, I spent a lot of time testing small intricate details of the app to make sure it works well. Through this, I picked up a few small issues which I also fixed.

Journey Planner improvements and bug fixes. 🗺️
Small bugs fixes including one where the routes were not properly sorting in the route list. 🐛
Continuous updates to ensure the app is always as fast as possible and using your phone as efficiently as possible. 📚

Tags release notes

London Bus Pal

London Bus Pal v4.3.0

This version brings the beta version of the new journey planner to you and a few small fixes:

• Journey Planner is here – it is very much in beta phase, but should be very usable! Let me know if you experience any issues. 🗺️
Small improvements to ensure very efficient battery use. 📚
Mostly minor bug fixes and one major one where routes were broken. 🐛
Please don’t forget that you can support me by leaving a rating in the app store 🌟 or by removing ads for a small donation in the settings menu 💷

Tags release notes

Development London Bus Pal

Bad apps will wreck your battery

Post author By Mulder Digital
Post date 02/03/2019
No Comments on Bad apps will wreck your battery

Following my previous post where I did some rudimentary experiments on other bus apps, I decided to leave my phone running for another couple of hours and see the effects on battery usage. This experiment involved opening and viewing specific stop and then simply leaving the app by pressing on the home button. I left the apps, but didn’t kill them. I did this for all the apps in the experiment and then reset all the counters.

And then I just left the phone for a couple of hours.

During the experiment – I left the phone off – in fact, I left the house and just left the phone on a desk. I used the phone for a couple of minutes, but made sure not to go anywhere near any of the bus apps.

The results were interest in that one app had woken up the phone 28 times and enabled the sensors for over 7 minutes during the 2 hours and 24 minutes the experiment was running for. It also transmitted 32.2KB of data during this time – not a significant amount, but that’s almost 50% of the amount that was transmitted by London Bus Pal when it was doing some real work.

Here is a breakdown of the results for this offending app (it is listed as a top app on the Google Play Store):

Duration:	2h 24m 51s
CPU Usage:	47s
Number of times waking device	28
GPS:	7m 27s
MPU6500 Acceleration Sensor	7m 26s
MPL Rotation Vector	7m 27s
Gravity Sensor	7m 27s
YAS537 Magnetic Sensor (compass)	7m 29s

I went back into the app to check why it would need to be on in the background. I noticed that Push Notifications were enabled and that I was automatically subscribed to get notifications for “special promos etc”. I turned off everything and disabled auto-refresh and also turned off analytics data sharing. After having to dismiss an ad (wow these are annoying!), I killed the app and restarted it (to make sure that all of the settings would load properly). I loaded a bus stop and pressed the home button and reset my battery usage monitor.

Time for some tea…

After tea, the results were so bad, that I thought I did something wrong and restarted the whole experiment. This time making sure that I take a couple of minutes before resetting the battery usage monitor to ensure that the app should be asleep.

Duration:	18m 26s
CPU Usage:	1m 25s
Number of times waking device:	3
GPS:	18m 26s
MPU6500 Acceleration Sensor:	31s
Gravity Sensor:	1m 1s
MPL Rotation Sensor:	1m 1s
YAS537 Magnetic Sensor (compass)	58s

The GPS timing was the reason I didn’t take the results from my first test, because I thought it was broken. It turns out the app is broken and holds onto the GPS when it really doesn’t have to.

Now I’m really frustrated

Do users even realise how this app is killing their battery? I looked through the reviews and this app gets a lot of praise (yes I’m slightly jealous) – but it seems that users don’t generally notice. I wouldn’t know without extensive testing which app is it that caused my battery to die so quickly – I usually blame the phone, not the apps. There are users who noticed, but they are far and few between:

Out of interest, this app was “App 2” on my previous list. It was exceptionally bad in terms of data usage and really bad on the battery in real usage testing too.

I’m going to stop writing now, because I don’t like focusing on other people’s work – I want my own to be the best it can possibly be. This was a useful exercise to make me believe in what I am doing, but it equally gets me down that unsuspecting users don’t realise why their phone batteries are draining so fast!

Tags battery drain

London Bus Pal

Kinder to your battery and data

Post author By Mulder Digital
Post date 02/03/2019
1 Comment on Kinder to your battery and data

London Bus Pal now has a dark mode which will help you to save some crucial battery life. This was just a quick experiment, rather than something users have been asking for. London Bus Pal has never been accused for draining battery – and this is for very good reason as I carefully think about every line of code I write to be as good for your phone’s battery as it could be.

This lead me to do some more experiments in terms of battery usage of London Bus Pal and other similar apps – some of the results shocked me to be honest.

The setup

The test I ran was fairly rudimentary. I took the top 8 apps which all return live bus times in London from the Google Play store. I made sure that I installed them and opened them all previously, wherever possible, I followed the same pattern:

If asked, don’t allow location to be tracked
If asked, select non-personalised ads – all apps were the free versions (had ads enabled)
Make sure I tap through the app to get rid of any “one-off” requests or app hints
All tests done in “light” / default mode

Location off means location off

I then had a small script which I manually executed, which involved searching for a stop, viewing a bus, viewing a stop, viewing another bus and so on. I executed the same steps on all the apps.

The first thing which grabbed my attention was the one app which was insisting that I enable my location before it would work. I persisted and kept saying no and eventually it gave in, and I managed to run through to the end of the experiment on that app. It turns out that it didn’t really require location to be enabled – it was just a lazy developer who didn’t want to work around this.

The next thing was the fact that despite location service being disabled and not really required (all the apps produced results in the end), three of the eight apps were still using the accelerometer. The is absolutely pointless and just wastes battery power.

The results

For the first round of tests, London Bus Pal was so low down the ranking of battery use by apps, that I almost thought my test was broken. But it was there, way down on the list. Even in light mode, it was by far, the kindest to my phone’s battery for the same test.

What about location on?

Next up was running the application with location services on. I decided I would do the simplest of tests and that is only allow the app to find my location and then close the app.

I ran through the test again and this time things were a bit less annoying (except for one app which insists on showing full screen ads all the time). This test was a bit more tricky to run because the first app run would possibly have the worst hit if it uses the GPS as efficiently as possible.

London Bus Pal kept coming back as the app, out of the eight, which uses the least amount of battery. In fact, three apps (London Bus Pal and two others) were clearly optimised for battery use whereas none of the others seemed to be, based on the numbers.

Of course, you’ll be asking, show us the numbers, and I have a list of numbers for you below – but it is worth pointing out, that the numbers are relative to each other in the same test, you cannot really compare them too much from one test to the next. It will give you an idea of the ranking and scale of usage of each application during both tests.

The most surprising thing for me was the worst offending app was engaging all of the sensors the whole time the app was open and even had some wakelocks going on (translated – this app is designed to suck the juice out of your battery).

Data usage

While I was doing my first test with location off and doing a set number of steps, I also tracked the data usage of each app. Again, the results truly surprised me – the app which was the worst on battery usage, was also the worst on the data usage (note, these are probably also related). What’s useful to note here, is that ad traffic was included in all of the data usage stats.

App	Battery drain (location off)	Battery drain (location on)	Data usage
App 1	74	79	428.8KB
App 2	81	100	2.3MB
App 3	81	44	818.8KB
App 4	74	53	617.3KB
App 5	100	58	776.5KB
App 6	21	21	87KB
App 7	16	37	87KB
London Bus Pal	14	21	93KB

Conclusion

The purpose of my experiment was to see whether I could officially claim that my app was battery and data optimised. It is something I care about deeply and carefully consider whenever I do anything. Based on the results, I think this is a resounding yes – I can definitely claim that my app has been built to optimise battery use.