Group 4: Research For Design Part 2 Week 2
Assignment
1. Identify stakeholders — individuals or groups — to work with.
2. Choose at least two different primary research methods to use.
3. Conduct your research.
4. Prepare a 5 to 10 minute presentation explaining the stakeholders you worked with, the research methods you chose, and the findings and initial conclusions from your research.
5. Update your process site with a post describing your role in this week’s assignment and reflecting on the process.
Week 2/3 Update
Group 4 has set out to better understand the conditions of New York’s rivers and beaches. You can find our week 1 update of part 2 here.
In week 1, we set our targets on specific stakeholders and chose our primary research methodologies.
Originally, we had targeted NY citizens, which we thought would be best explored through surveys. We also wanted to understand motivations on a deeper level which we researched through the technique of netnography. And finally, we conducted analogous inspiration for local business owners and residents to find inspiration in solutions from other similar spaces. We initially targeted inspiring groups that were already working on solutions in the space. We thought these people would be great resources given their drive to solve these problems and their deep understanding of the problems they faced. unfortunately, we were unable to schedule interviews during the holiday break.
Dividing The Work
Being the remote distributed team we are, we did our best to collaborate online via google sweet. We used google hangouts for working sessions and to divvy up the work in a fair way. We decided to follow three different research methodologies and each group member would be the primary person responsible for it. We would then collaborate as we progressed through the project.
My main responsibility was to create a survey for NY citizens. As a group, we decided to reduce our scope to just New York City, which helped us to focus on a few specific beaches and rivers. I was also responsible for conducting some of the Netnography research.
The Survey
To get started, I drafted a version of the survey in a Google doc and invited them to review its contents. In addition to the weekly lesson on primary research, I looked into other relevant surveys to use as a reference point for good practices. The most applicable version I found was the MTA user research study available on the MTA website.
There were several best practices I took from their example. I made sure to include a brief introduction that gave respondents a sense of what they were going to do, how long it would take, and what the purpose would be. I liked how they included an esimate for how long it would take. Also, throughout the survey, they made responses optional, and demographic questions included an option to not answer.
I shared the Google doc with the group and gathered their feedback, which helped me to order my questions effectively. Once we were in agreement on the questions, I created a google form to collect responses.
We chose Google Forms because it was simple, free, and easy to review and distribute. Google also has great interfaces to reviewing data after the fact, and a Google Sheet export option to explore the data further.
Advantages and Disadvantages
Surveys have several advantages, including:
- They are easy to distribute
- They can reach a broader audience quickly
- They are great for standardizing quantitative responses
- They offer flexibility for qualitative responses
We wanted to get a starting baseline for general sentiment of the NYC rivers and beaches. One problem we faced was reaching our target users. Since only one of the three group members was actually in New York, we decided we would instead target anyone who had visited New York City. That way, we could all participate in distributing the survey.
We also wanted clean data to refer to. By using multiple choice questions, we were able to standardize the responses we received in order to compare them easily.
We also offered space for qualitative responses with open ended questions. This was great as it allowed users to share exactly what they were feeling.
There were also several disadvantages to gathering data via a survey:
- Sampling Error
- bias
- Limited depth
- Annonymity
- Survey platform limitations
Sampling error became a problem. Despite asking our friends and posting on social media, we were only able to get 23 participants. While there was certainly concencus, it was hardly enough to reach any solution with great confience. the good thing was that we had a significant portion of residents and non residents to compare which was nice.
We also may have had some bias or influence on our users. Many referenced the lack of cleanliness as the biggest problem of the rivers and beaches. Either that question was so obvious that we needed to ask a deeper question or it was influenced by the format of our survey. In either case, if we were to make another version, we might assume the rivers are dirty and ask questions more specifically about the dirtiness to get more novel answers.
We also faced some limitations of the survey platform. We weren’t able to separate the survey into sections like we originally intended. I wanted users to submit their responses more frequently so we could put demographic information on the last page. That way, if a user wanted to skip that step, they could simply not continue and they would have already submitted the other responses. Instead, to reduce friction in the survey completion, we decided to omit any demographic questions and instead focus on whether or not our respondents were NYC residents or not. This anonymity could have lead to different answers than had we just asked for some identity information.
Survey Results
After we conducted the survey, we took a look at our results. One thing I saw immediately was that there was a relationship between the sizeof the river and its recognition on the survey. This was probably obvious, but it could tell us something about the quality of the data, or could have illuminated an outlier river.
I refered to this above, but we saw a lot of very similar responses in our open ended questions. I think this either means our users were influenced in some way by the way we wrote our questions or intro, or this was so obvious that we probably should have asked a more nuanced question. Doing this again, I would have written a better follow up. Given the original goal was just to establish sentiment, it does do its job.
We also saw a significant different in the way people view rivers vs. beaaches. If you look at the graph on the left for beaches, almost no one uses them, while nearly 83% use rivers in some capacity. That being said, a larger portion of users used beaches for recreational purposes than rivers.
One interesting thing to note was the difference in sentiment among residents and non residents. Collectively, residents were less satisfied with beaches and rivers than non residents. Both residents and non residents were more satisfied with the rivers in their current condition than the beaches.
Netnography
Justin took the lead on Netnography explorations, but I wanted to support that effort as well. He had originally done a deep dive on Twitter, and while I thought he captured a lot of great posts, I didn’t think the platform lended itself to a lot of discussion threads. I suggested we explore reddit and other longer forms of social media to see if we could view how longer discussions unfolded.
I originally looked at LinkedIn, which I thought would be a good place for lots of comments and discussion. While it was true that posts had high levels of engagement, they were often surface level comments that didn’t add much to a discussion. I believe, because the platform encourages a professional demeanor and because you have to use your own name, it probably limits users from being completely candid. Here’s an example when searching ‘Hudson River’.
As you can see, all the comments are positive, which is great, but there doesn’t seem to be any discussion or conversation happening.
Conversely, a discussion on the same topic using the platform Reddit yields a much different result. BurpFartBurp does not approve of the bridge ceremony.
I decided to pour through reddit and there were actually some surprising results! While there were certainly lots of comments bashing the condition of the rivers, there were many redditors defending the waterways.
In many cases, a snarky or negative comment is met with an explanation of how the responding user likes to swim in the river, or how much cleaner it has gotten over time. They also suggest experiencing the river at a location further north as a suggestion it might be cleaner there.
Lastly, and back to LinkedIn, I wanted to highlight something I found interesting. In this example someone pointed out a new art piece and seating at a pier along the Hudson river. A user follows up explaining how grateful they are that the OP shared this new space, and how they like to go and read at this peaceful location. It seemed to spark an area of opportunity — that perhaps public perception could be swayed if they were simply made aware of where to look!
You can find this and additional explanation in our team video below.
Thank you for reading!