Software testing for startups


This is a guest post by Simon Papineau, founder of the Crowdsourced Testing company and generation 5 supper.

Most entrepreneurs, particularly those involved in web and software companies, are consumed by a sense of urgency. You’ll often hear them say: “time to market is critical. We have to be the first to launch our product.”

And they are right – it’s a competitive world out there and time to market is important, often critical.

When web entrepreneurs hire a team of developers to build their idea, they would like it to be ready it two weeks. As time goes by and the project proves to be more complicated (read: longer) than expected, they increase the pressure on developers in hopes that this will make them deliver a final product faster.

And when patience is running thin, they make sacrifices and cut corners. In fact, there’s a whole “lean” philosophy that’s often grossly summarized with the expression “f**k it, ship it” or the more politically correct version “done is better than perfect.”



This notion became popular in part due to a famous quote by LinkedIn co-founder Reid Hoffman who said: “If you are not embarrassed by the first version of your product, you’ve launched too late.”

The problem is that some entrepreneurs apply this philosophy much too literally and shoot themselves in the foot by launching a product that is obviously unfinished and unpolished.

It is true that early adopters will forgive you if you product contains a few minor bugs. They are looking to gain an advantage on their competitors and generally are tech savvy enough to work with you at solving any remaining issue in the early versions of your product. There is a really good book on the subject called Crossing the Chasm.

But when you are attempting to increase the notoriety of a disruptive product, you are effectively hoping to convert your early adopters into ambassadors; and this will not happen if your product contains more than a few minor bugs.

Shipping fast shouldn’t mean skimping on testing

I launched the first version of the Crowdsourced Testing company’s website in 2012. It was then known as QA on Request.

I wanted to solve a problem that I was experiencing in my daily life. I built a website based on my experience, on my vision and on what I thought made sense for me.

It didn’t take long to realize that none of the clients I was targeting could figure out how to use it. I had built a website for myself. I was the only person in the world who knew how to use it.

Had I spent 250$ over at I would’ve instantly known that something was wrong. But I didn’t take the time to conduct usability tests. I was too eager to launch.

Usability testing is something that too few startups do. Learn from my mistake. Conduct usability tests before you launch, you won’t regret it.

The other type of testing that is often neglected is proper quality assurance (QA) testing.

Proper quality assurance testing

Very few startups employ quality assurance testers because of their sporadic testing needs and because they perceive QA testing to be an unnecessary burden.

Actually, it would be right to say that very few startups actually perform any kind of tests before releasing their product because they simply want to get into the hands of their potential customers as fast as possible.

Before you discard the idea of taking the time to thoroughly test your product before releasing it, I urge you to reflect on the teachings of a self-proclaimed “dating guru” named David DeAngelo.

Now, I’m not saying that I condone everything that David DeAngelo preaches, but he makes a claim that I find applies to this situation: attraction isn’t a choice.

David claims that attraction is the result of a subconscious process influenced by years and years of education and experiences. i.e. when I meet someone new, my subconscious instantly makes a decision as to whether I am: a) interested/attracted to this person, or b) closed/uninterested by this person, and the rational side of my brain has very little to say about it.

Think about this in the context of web browsing. When you are searching for information online, how long do you browse a website before moving on to the next one when you can’t find what you’re looking for? 5 seconds? Maybe 10?

What is your opinion of a website that contains errors and broken links? Would you recommend it? Would you revisit it later? You wouldn’t. Because you’ve already made up your mind: it stinks.

You have one chance at making a good impression. If your users do not love your product immediately, they will never return to it. Getting users to discover your product is half the battle, retaining them is the other half.

Mobile analytics company Flurry has published a post titled App Engagement: The Matrix Reloaded with statistics that demonstrate that average user retention of a mobile application after 90 days ranges from 10% to 55% at best. The conclusion of their study was:

Compared to Flurry’s 2009 analysis, 90-day retention rates have increased from 25% to 35% […] we attribute increased retention rates to increased quality in the market, driven by more competition.

The moral of the story is that building quality products is the key to success, more so than shipping fast.

To be an entrepreneur, you have to be eager to solve a problem. But you should also be patient and caring enough to make sure that you solution provides enough value for your users to be: 1) quickly satisfied in their quest to solve the problem they were facing, and 2) enchanted enough to keep using and to recommend your product.

The lean startup and quality assurance testing

Shipping fast doesn’t mean rushing to the finish line. It means breaking down your whole product into smaller chunks and shipping each of them individually. Each chunk should be solid, tested, and offer enough value that your users engage with it and recommend it. That is the true meaning of the lean startup philosophy.

Don’t skimp on either usability or quality assurance testing. Your startup will thank you.

If you are curious about crowdsourced testing and how it can help your startup deliver better web and software products, I invite you to continue your reading here: what is crowdsourced testing?