Tips for marketing specialists and product managers
In Ancient Greece, the Temple of Apollo at Delphi was considered a place of ultimate wisdom. Peasants and rulers alike used to come there seeking for answers and divine guidance. The very first piece of wisdom they’d encounter there was an aphorism inscribed in the forecourt of the Temple – “Know thyself”.
Such knowledge is exactly what we were aiming to achieve with our latest focus group survey for Roadmap Planner. Today, we are ready to share with you the experience we have gained and the conclusions we’ve come to in the process.
Laying the groundwork
The preparation stage took us about half a month. We had to figure out what goals we wanted to achieve with the survey, what were the potential issues, which tools would work the best, what materials we need, etc.
The main idea behind conducting a focus group survey was to figure out three things:
- First of all, we wanted to check our app’s usability. With new products, it is impossible to predict for sure which of all the awesome features you’ve implemented after numerous brainstorms are actually useful. It’s not uncommon for an app developer to find out that users only need half of its functions, at most.
- Next, we were looking to better know our clients and the market. Since our planning app can be used by different people for different needs, it’s important for us to figure out which categories of users actually benefit the most from it.
- Last but not the least, we were eager to receive the negative feedback. To learn what dissatisfies users, what features they think are missing, what can be done better or otherwise. Criticism is crucial for a healthy development. And there can be no better judge for your product than your customers!
Our team had had certain experience of working with focus groups. Due to this reason, we were aware of potential issues and made our precautions. For example, a likely problem we’d kept in mind is the way people usually fill surveys. If asked vague questions, they tend to give generalized answers, whereas we wanted specifics. That’s why we prepared sets of questions that covered the same areas from different angles, thus ensuring we’d get unequivocal results.
We’d also understand how important it is to use proper tools for such a global task. Predictably, the one we used to build a plan of the survey was Roadmap Planner. It proved extremely helpful for grasping the big picture of the project and planning specific sub-goals. We also utilized a number of third-party apps like Skype, Callnote, and SurveyPlanet. But we will go into this in depth in further articles.
TL;DR. We defined that with the survey we wanted to find out more about our customers and what they like and don’t like about Roadmap Planner. For this, we chose the appropriate tools and anticipated some of the potential complications.
Focus group, camera, action!
The focus group for our survey was created using various sources: email invitations, SMM, PR, push notifications, and CTA on the main page and our blog. This resulted in the initial group comprising over 400 respondents who agreed to participate.
“Important notice: we have provided the respondents with as much information as possible from the very start on the landing page. This included the terms, number and time of surveys, and any other details that could possibly cause people to quit the focus group if discovered later.”
We decided to organize our interaction with the focus group in three stages. On each stage, respondents had a choice of the way they would prefer to be interviewed: by phone or online. Unsurprisingly, most of them chose the latter option since people often don’t feel confident enough during a direct dialogue.
Nonetheless, we always suggested a phone call in the first place since this option was more beneficial for us. And then as an alternative we offered the online survey so that we wouldn’t lose potential participants if their would plans suddenly change. Still, some of the respondents actually agreed for the call (mostly the users of our other products).They were the ones who we got the most useful feedback from.
Stage 1
Over 400 participants were invited to pass the first survey to gain access to Roadmap Planner. It mostly consisted of closed and quantitative questions, and was designed to learn more about respondents and their expectations.
The feedback we’ve received was quite abundant and helped us draw an image of our Person. We have learned a lot about our potential users and their needs, some of the info even turned out to be quite useful despite not being directly related to planning. For example, we found out that people want more educational content about various topics, such as videos, guides, how-to’s, etc.
After respondents completed the survey, they were rewarded with a year-long subscription for Roadmap Planner. Then, they were asked to use the app as they normally would for 2 weeks until the next stage of the survey. We didn’t ask them to perform any specific actions, and assigned managers to support them and answer any questions.
Stage 2
Over 20% of initial respondents agreed to participate in the second stage, and a large part of them agreed to a phone call interview. This time, the survey was more broad and consisted mostly of open questions. At this point, we’d like to remind you about the need to record your calls, since some of them lasted for whopping 60 minutes!
Obviously, we asked what people used Roadmap Planner for and if they would recommend it to their friends. Also, we wanted to gain feedback on the app’s features: which it lacked and why, which were unnecessary, etc.
Less obvious questions we asked were about competitors, like “which other planning tools do you use?”. It gave us certain surprising results, as users would sometimes compare us with unexpected services that we’d have never seen as rivals. Finally, it’s highly important to leave some space for “other” comments. We received a lot of insight from this section, and people were often happy to share their less concrete thoughts with us.
Overall, the second stage interview was significantly longer than the first one. And yes, we know that it’s often considered a bad idea to take too much of the respondent’s time. But the results we’ve got and the information we’ve gathered turned out to be so fruitful that it was completely worth the risk.
Stage 3
After letting the focus group use our app for two more weeks, it was time for the third and the last survey. This time, we’ve managed to keep over 65% of respondents from the previous stage.
We continued to ask our questions broad and open, like which features users liked the best or what would they have added to the app. The question “how have you used Roadmap Planner on different platforms?” also provided valuable information. Additionally, we requested some meta-information, such as which means of communication would be convenient for future contacts with participants, and whether they were happy with their focus group experience in general.
We were aware that there’s probably much more they would like to share with us, thus we also included questions like “Is there anything else we should know about you?” and “Did we miss anything?” You never know what hidden pieces of wisdom your audience may have noticed. Finally, we asked respondents to rate our app on the Apple marketplace and were pleased to see the reviews were quite positive.
TL;DR. We gathered the focus group using various sources. The research was conducted in multiple phases, and each one had its own set of questions and served different purpose. Each stage corresponded with different levels of users’ immersion in the product. This way we could model their behavior and prepare next questions sets according to this. The results were useful and gave us plenty of insight on both our product and customers.
Net result
After the survey has come to an end, it is time to bear fruits. Overall, we consider that the global goals have been achieved. The feedback we received from our work with the focus group proved extremely beneficial both tactically and strategically.
In terms of tactics, we have already implemented some of the results to our processes. For example, we have prepared a number of ad banners for different target audiences based on the Person images. Additionally, we have drawn many insights for our forthcoming webinar.
Examples of Person image answers:
Q: What is your main purpose of using planning tools?
A: Other: share workflows and process journey with senior management who are not technical specialists
Q: Is there anything else you want us to know about you?
A: I am a religious volunteer and not involved in profit-making business. However, my field of volunteer involves making project/planning. So if I can have a tool to help me, this would be nice. Also, I like to try out new apps and hopefully can contribute to the development of the app even in the smallest way. Thank you.
Strategy-wise, we have built a better understanding of the direction our product should be moving and what we’re doing right. We have also collected wishes and suggestions from respondents to a backlog and will certainly utilize them in the future.
Example of suggestions sent to a backlog:
Q: How would you rate your user experience?
A: I have not spent enough time to answer this question. But, I would like to have my strategic plans in the local drive and have the option to upload it into the cloud.
Another thing we’ve learned indirectly from the experience is that you shouldn’t close the survey’s landing page after it has begun. Despite we had stopped advertising this event and updated the info on a landing page informing that the subscription period was over, we received a number of requests. Understandably, we decided to let them participate.
And even after the survey was concluded, we didn’t shut down its landing page, but rather edited it to inform that the event has ended. In the result, we have received a couple of applications afterwards, thus we already have a potential user base for future tests.
Other than that, there are two major lessons we have learned from this experience. First, less really is more. People seem to love simple products that solve their problems more than complicated and all-inclusive goliaths of an app.
Example of benefits of simplicity:
Q: What Roadmap Planner advantages have you found out?
A: Graphic representations of interrelated activities – mini projects. I can use it on iPad in meetings.
Second, surveys are an awesome tool for a company development. Sure, they can be cumbersome at times, with hundreds of individual interview results to check and analyze. But all in all, the benefits you get as the result are just too sweet to ignore. Besides, it’s just so cool to know your products actually help people!
Q: Is there anything else you want us to know about you?
A: I am an engineer, freelance project manager and consultant, owner of a foundation for emergency management and disaster recovery, father of 3 daughters and husband. So i really need planning !
Special Thanks: We would like to express our deepest gratitude to Andrei Karol, CTO of Cardiolyse, for his priceless assistance, experience, and advice.