The importance of doing a beta phase in making a successful app

, ,

The process of creating an application does not only involve developing it. As discussed in a previous blog post, the marketing is a crucial aspect. There is another important aspect that we tend to avoid or not taking seriously enough: the testing phase, which I call the beta phase.

No matter which platform you build your app for, you need to test your app at some point. Yes, you worked really hard for many weeks and months on your app. Yes, you know your app by heart. Yes, it is bug free.

I don’t want to disappoint you, but there is always a little something that you didn’t think of, or a special usage scenario you didn’t pay attention to.

I speak from experience. I have submitted apps without doing a beta phase. I have also submitted an app where a crash could be easily reproduced. It is nothing to be proud of, but at least now, I do a beta phase.

True fact: developers are not good testers.

Before we released DualShot, we ran a beta phase.

So, how do you attract testers?

  • Contact your friends
  • Advertise on social networks that you are looking for testers. In my case with DualShot, I tweeted about it at two different times.

Believe me, you’ll get testers pretty easily. People are curious. Depending on how well known you are, you can keep track of the emails manually like I do or you can share an Excel sheet or use MailChimp like one of my friends does.

Unless you pay the testers, don’t expect to get feedback from everyone. This is normal, so don’t take it personally. Sometimes it is just not the tester’s type of app, or perhaps they are just too busy.

Tip: I use BugSense to keep track of crashes and I use Flurry for the analytic events. I highly recommend using these services (or any equivalent services) during the beta phase. As I previously mentioned, you might not get written feedback from all the testers, but at least you’ll receive traces of what they did with your app.

During the beta phase, the tester can provide feedback about the user experience (UX). As a concrete example, in DualShot, Vincent designed the following page:

clip_image001_thumb[1]

I really liked this page and I didn’t see a problem using it. When I submitted the beta to my testers, in the same evening, three users complained that the UX was really bad. It was clear to us that we needed to put more work into that view. We ended up with the following design:

image_thumb[2]

Unless you have the complete Windows Phone collection at home, you are most likely going to have only one or two devices to test with. When you have testers, you increase the chances that the app will be tested with many different Windows Phones. Don’t assume that all Windows Phone 8s from different manufactures behave the same. In DualShot, the image capture with some HTC 8XT (only this model) does not work. We didn’t have a tester with this rare device and Murphy’s Law struck again.

Not only will the testers find bugs/crashes and give you feedback, they will often give you two thumbs up. It will give you the extra energy to polish your app before certification.

Windows Phone

If you are developing a Windows Phone app, you are lucky because the Windows Phone team developed a feature in their portal where it is easy to run a beta phase. Essentially, you publish an app as if you were submitting it for certification. However, the app is kept private for the testers that you have specified and the certification passes automatically in the subsequent two hours.

In the Windows Phone developer portal, you need to select Beta as the Distribution channel, then you enter the list of tester email addresses (using a ‘;’ between addresses).

clip_image002_thumb[1]

After that you can fill out all the info and screenshots.

Tip: avoid flooding your testers with daily emails if you submit a new beta every day. If you submit an update, the tester will be notified via the Store Live Tile. Take advantage of this automatic notification to enter all the new features/bugs into the app description. The first time that you send the deep link of the beta app, you can tell your tester to check the app description when a new update is available.

Windows 8/8.1

Unfortunately, in the current Windows developer portal, it is not possible to easily distribute an app to testers. You need to create a package and send the package to your testers. The testers then need to manually install the app. Hopefully the Windows team will soon mirror the awesome work of the Windows Phone team.

Conclusion

I know the feeling when an app is finally complete and ready to be submitted for certification. It is so tempting to submit it right away in order to see it live in the store ready to be downloaded. Please resist this temptation and do a beta phase. It is better that your friends find the problems than strangers, as strangers will most likely give you a 1-star review if they find a bug or a crash.

Develop, test and release!

DualShot from the launch to 100 000 downloads in 20 days

, ,

My friend designer Vincent and I decided to share the story of our latest app DualShot. The goal is not to be show-off. We want to show diverse statistics that can help analyze the Windows Phone market and encourage other developers and designers to continue providing quality apps.

Before starting, DualShot has two restrictions:

  • It is only available to Windows Phone 8 devices.
  • It requires a back and front face camera. In other words, the most popular devices the Nokia Lumia 520 and 521 cannot download the app, because they don’t have a front face camera.

Statistics

The launch date was September 16th. The download count surpassed 100 000 on October 15th.

SNAGHTML207d90

Data from the Windows Phone dashboard

It’s not a big surprise that the peak arrived close to the launch date. DualShot received a lot of good press from many Windows Phone websites.

Here is the top 30 markets. We were quite surprised that India and Thailand arrived on the second and third spot despite the fact that DualShot was available in English and French only at launch. The two biggest continents with the most users are Europe and Asia.

SNAGHTML2060027

Complete list

Here is the percentage usage per device. There is no doubt that Nokia is dominating the Windows Phone market. The non-Nokia device with the biggest market share is HTC with 5.3%. Looking forward, I’ll be curious to see how the Lumia 1020/925 will evolve. Their percentages are low because they are new and not available worldwide yet.

image

Data from Flurry analytics service

Concerning the reviews, there is a clear relation between the number of reviews and the downloads. The American were the most critics about our app!

SNAGHTML21cf0b6

Data from App Spotlights

Pre-launch

Developing an app is one step, but advertise an app is another step. I’m not a firm believer that submitting an app without talking about it that the app will become popular.

The day before the app went live, we contacted a lot of editors and friends. A full day can be easily spent sending all the information.

For those who noticed, we created a little teaser on September 26th. It turned out that the one-day teaser worked well.

SNAGHTML22b8e79

SNAGHTML22cd967

Post-launch

At this point, this is where the team has less control. It is the turn to the public to decide about the success of the app. It is also on the hands of the editors if they want to talk or not about the app. This is the most exciting and stressful period where the team awaits the result.

In the case of DualShot, we were very happy with the first days following the launch. In reality, it surpassed our expectations.

We were proud to appear twice on Windows Phone Central:
http://www.wpcentral.com/dualshot-lands-windows-phone-store
http://www.wpcentral.com/dualshot-windows-phone-preview

We were delight to see a YouTube review from our friends at WinSource:

We had a review on a Microsoft Canadian website and many more sites.

Ranking

Having a lot of tractions at launch caused a positive domino effect. The more an app is downloaded, the more it appears on different categories. At some point DualShot was the top New+Rising app in USA.

SNAGHTMLca0cbf2

A week later, the download count probably caused a trigger on the algorithm of Microsoft to display the spotlights. With the help of my app App Spotlights, I was able to detect that DualShot was on the spotlight of many markets on October 4th.

SNAGHTMLc9a8fc1

Later on, we had one occasion with the big spot in the USA spotlight page. We can see the effect on the graphic at the top of the blog post.

SNAGHTMLc9ff46e

So far, we had a lot of exposures:

wp_ss_20131017_0002

Conclusion

While we didn’t make any money from releasing DualShot because we made it free, we had so much fun following the launch. It surely motivates us to create more Windows Phone apps. Now that the Windows Phone market is not small anymore, we see a lot of opportunities and we encourage developers and designers to jump in. You can also be surprised with the result.

Have you create a DualShot photo yet?  If not, go download and share your creation!

Download from Windows Phone Store

The importance of an about page

,

Your app is feature complete, here is what you should consider before sending it to the certification. Add a page that contains at least three sections:

  • About
  • What’s new
  • Credits

About

About 1

The first goal of an about section is providing a way to your users to connect with you. For DualShot, I provide diverse links. The important one is Support & Feedback. It opens the mail editor. I take the opportunity to enter information that might help like the current version of the app, language and the phone model.

Mail

Providing a way to contact you won’t prevent all bad ratings, but it will save you some if you respond accordingly.

In the list above, there is another interesting link called User Voice. User Voice is a website that provides a (free) service where user can enter anonymous feedback or bugs. Their website is very easy to setup and use. I have seen a lot of apps having a User Voice link. Even the Windows Phone developer team has a User Voice page.

What’s new

About 2

There are a lot of applications that provide updates, but often we are not aware of what’s new. Sometimes, they put the update description into the app’s description, but the second you click update you don’t have access to the description in one click, you need to find the app into the store. Starting with my app DualShot, I will provide a what’s new section for my future apps. Not only it tells the user what’s new, but it can help the user to discover features that are maybe not obvious at first sight.

Credits

About 3

Open your Windows Phone solution and check the references for each project. Do you see a lot of 3rd party libraries? The answer is most likely yes. Would you be able to release your app without using any free 3rd party libraries? I’m sure you can, but it would take a lot more time. Reinventing the wheel is probably not your motto neither. That’s been said, with my latest applications, I took the decision to thank the people bind the libraries. It doesn’t cost a dime to give a two thumbs up for the creators. For each library and translator, I put a link to their website or Twitter handle. After I released my app, I tweeted to each of time and 80% replied back.

You never know what you can expect: four gentlemen offered us to translate the app for free. It was my pleasure to put their names into the credits section.

Don’t worry, you won’t get hurt in the rating for putting a credits section. I have receive a lot of good and very bad reviews, but I never receive something related to: “Huh, your app suk and your credits too”.

Conclusion

Take the extra 2-3 hours to provide for your users to connect with you and thank the person who helps build your application directly or indirectly. If you provide updates, add a what’s new section.

MVP Client Development

Last week, I had the honor and privilege to be elected as a MVP Client Development.This award goes to people that have contributed to the community for the last twelve months in diverse ways. It is awarded every three months.

The Microsoft definition is:

The Microsoft Most Valuable Professional (MVP) Award is our way of saying thank you to exceptional, independent community leaders who share their passion, technical expertise, and real-world knowledge of Microsoft products with others. It is part of Microsoft’s commitment to supporting and enriching technical communities. Even before the rises of the Internet and social media, people have come together to willingly offer their ideas and best practices in technical communities.

My definition is shorter:

Love what you do, share what you know and make new friends.

I would like to thank all my developer friends and Microsoft employees who nominated me.

MVP