Alpha testing has not gone as well as I hoped. Over the last few weeks I have been adding functionality and I thought the system was quite usable. So as a final step before expanding testing, I sat down with my first alpha tester to watch them use it. I expected them to suggest extra features. I had a long list of extra things I could do, but wasn’t sure which were important. The good news was that they only suggested a couple of small things that I already had on my list. The bad news is just watching them use the site made me wince and clearly wasn’t the best experience for them either. Over the 10 minutes of the test I quickly filled a couple of pages with issues: what does this mean, what does that do, why is that there, how do I do that?
I can heartily recommend watching someone removed from development trying to use your system. After a couple of months I am too close to the program. I no longer see what is displayed, I just know what needs to be done. I also know how to move around the site. Seeing the site through someone else’s eyes was illuminating. Watching them have to move the mouse all the way across the screen to do two related actions made me realise I focus exclusively on one bit of functionality at a time. Seeing them wonder if two similar but unfamiliar terms referred to the same concept made me realise I don’t read the text. When they suggested that grey buttons made it look like they we deactivated, I remembered that I meant to spend some time on the design instead of just working on functionality. I have decided that I will not add any new functionality for the next couple of weeks, instead I will focus on fixing up what is already there.