Healthier Personalization with Surveys
#エンジニア一般
PinterestEngineering
>Leif Sigerson | Senior Data Scientist, Data Platforms Science,Stephanie Chen | Staff Quantitative Product Researcher, Product Research,Wendy Matheny | Senior Lead Public Policy Manager, Government and Corporate AffairsOne of the best parts about working at Pinterest is meeting someone new and telling them where you work. Often, they smile and immediately say, “I love Pinterest! I go there for…!” It’s a great feeling that never gets old.A major reason so many people feel good about their time on Pinterest is that it feels personal. Pinterest is personal — it’s about yourself, not your selfie.People come to Pinterest (who we call “Pinners”) with an idea — something like “funky blue shoes for my birthday party” — and our goal is to help them to find those funky blue shoes, and then get out on the dance floor — not to have them hang out all day on Pinterest.A search might lead them to velvet shoes with baby blue bows, or patent leather shoes with navy polka dots, but we follow their lead and show them results aligned with their current and past searches. Pinners follow their inspiration, know good when they see it, and keep coming back for more. And we know they’re finding what inspires them, because, over the past few years, we’ve seen Pinner engagement soar — with Gen Z leading the way.As decision makers around the globe debate the merits and faults of engagement-based ranking and depersonalized feeds, Pinterest is demonstrating there is a better and healthier way to personalize content. And while we can’t account for what every other platform is doing, we will stick to our company values to ‘Put Pinners First’ as Pinterest optimizes for quality time over time spent on our platform.The false choice between pure engagement ranking and chronological rankingIt’s well-known that ranking content purely by what is most engaging can lead to harmful outcomes — prioritizing “clickbaity” content that draws us in, even when we know it’s not great for us.This inverse relationship between content engagement and content quality appears to be a challenge across the social media industry (Cunningham et. al., 2024). And it has led some to call for a ban on any kind of personalization — taking away content, experiences, or advertisements tailored to an individual based on their preferences, behaviors and interactions — and replacing it with generic “reverse-chronological” feeds where people simply see the most recent content first.While reverse-chronological feeds are appealing in theory, as they sidestep an individual’s engagement with one type of content or another, this seemingly simple solution can be deeply problematic. That’s why independent experts strongly recommend against reverse chronological feeds. For example, Bengani, Stray & Thorburn, (2022) note that these types of feeds only make sense for some platforms, and would be “near useless” for many platforms. They conclude that reverse chronological feeds are “not a reasonable replacement for recommender systems.” Similarly, an expert report by the Knight Georgetown Institute noted that reverse chronological feeds “have drawbacks, and there are numerous other ways to design systems that provide users with valuable, high-quality experiences.”Cooper (2025), notes that these feeds “can shift the mix of recommended items in unexpected ways” and points to real-world evidence that reverse-chronological feeds can actually promote worse content in some cases.This makes intuitive sense — if users only see the most recent content, this would incentivize creators (or bots) to post a lot of spammy content, rather than focus on high-quality, tailored content that will genuinely inspire or resonate with their audience.In place of reverse-chronological feeds, these experts (along with others like Stray et. al., 2022) point to surveys as a way to give people what they truly value. Many platforms invite users to complete surveys while they are using the app. These surveys are an opportunity for users to provide direct, thoughtful feedback about their experience so that platforms can ensure they’re aligned with what their users want.An invitation to a real survey we recently showed to Pinners to make sure we were showing them relevant search resultsAt Pinterest, we agree that we don’t have to choose between pure engagement-based ranking and depersonalized ranking — there’s a better way. We have found that Pinner surveys can help us move toward “healthier personalization” that aligns with Pinterest’s commitment to building a more inspired internet.Pinterest’s approach — healthier personalization with surveysPinterest is the founding signatory of the Inspired Internet Pledge, which calls for companies to:Tune for wellbeing: understanding which actions and content correlate with wellbeing outcomes.Listen and act: listening to and learning from those who have experienced harm online and the experts who support these communities, to inform our businessCommit to openness: sharing best practices, key research findings, and creative solutions to make the internet a healthier place for everyone — especially young people.Healthier personalization with surveys is part of how we fulfill our commitments to the first two of the Pledge’s principles, and public shareouts like this (and our work on the Field Guide to Non-Engagement Signals) are part of how we fulfill our commitments to the third one.At Pinterest, we believe the best feed incorporates a balance of different approaches to content ranking. We aim to incorporate a) “explicit” engagement signals like saves, b) rigorous Community Guidelines that everyone has to follow, and c) survey-based personalization to give Pinners a feed of high-quality content that’s tuned to inspire them.From Pinner surveys, we know that inspiration is highly personal. Cherry Coded outfits (from Pinterest Predicts 2025) might inspire you, but Rococo Revival could be exactly what I was searching for without knowing what to call it. Each Pinner’s unique tastes and preferences help us refine the platform to meet them where they are, and this is why a “de-personalized” feed just won’t work for the 570 million creative, curious, inspiration-seeking Pinners who come to Pinterest every month.Pinner surveys are the “secret sauce” to healthier personalization because they give us direct feedback from Pinners that we can then use to create even more relevant, inspiring, nourishing experiences for all. For a person searching for inspiration on Pinterest, these surveys invite them to pause and reflect on their Pinner experience. Some examples of questions we’ve asked in surveys are “is this content useful?”, “does this content inspire you?”, and “does this content help you find something to buy?” (Pinner surveys are also central to our work to build an inspirational shopping experience). From Pinner responses, Pinterest learns what we’re getting right and wrong, and we can make adjustments to better tune for Pinner wellbeing with content that is relevant, useful and inspirational for all, which in turn helps Pinners find what they’re looking to be inspired by even fasterWe care deeply about running rigorous surveys. If we’re going to ask people to put their time into filling out a survey, we want to make sure it’s a good experience for them and that we’re going to use their responses effectively. That’s why we have an internal “guild” of highly trained survey experts (all with PhD’s in psychology) to design and run surveys. Every survey a Pinner sees has been approved by this guild.This process has paid off for us, as recent research by USC shows that Pinterest is unique in terms of our ability to provide Pinners with useful, important content while minimizing negative experiences. USC’s findings corroborate our own research that Pinterest leads the industry in terms of our impact on Pinner wellbeing. Additionally, surveys have been critical to some of our most inspirational, industry-leading work like body size inclusion and skin tone filters.Next, let’s take a look at a real survey we use to shape our product strategy.Deep Dive — the Home feed Relevance SurveyFirst impressions are everything and Home feed is the first place Pinners land when they open the Pinterest app. It’s an important place to show the most personally relevant inspiration. Pinterest has developed various models to understand each Pinner’s unique interests. While models might seem effective by driving lots of views or engagement from Pinners, the only way to know if personalized content is hitting the mark is by asking them.The Home feed Relevance Survey was developed to do just that. This survey is shown to a small sample of Pinners each day and asks them to take a moment to rate a few Pins that are queued to be shown on their home feed. In other words, they’re rating Pins that are specifically chosen for them based on their activity on Pinterest, not a random or generic set of Pins. Pinners first rate how personally relevant each Pin is. If they indicate a Pin is not relevant to them, they’re asked why. The follow-up question captures more specific issues like Pins not being in the Pinner’s taste or style, not matching their interests, or reflecting an old interest that is no longer relevant.We’ve used this survey to proactively evaluate our models before launching and to quickly make changes when we discover we’ve missed the mark after launches. For example, our data science team proactively tested a new Pinner interest model against the Home feed Survey’s negative responses. When the model suggested that a Pinner might be interested in a particular category like women’s fashion, the team checked that Pinner’s survey responses to make sure the Pinner didn’t rate any women’s fashion Pins as not matching their interests. The team only tested the model in experiments after confirming that it had a very low rate of suggesting irrelevant categories.Another example: In late 2020 we observed a deep drop in rated Pin relevance for the survey. This triggered an investigation which found a particular type of Pin had started to be distributed more widely in that same timespan. That type of Pin had lower relevance ratings and higher incidence of complaints about the content being in the wrong language. We refined the updates to our recommendation system and were able to restore our home feed Pins to the high level of relevance that we had been delivering before.ConclusionPinterest is different. We hear from Pinners every day about how much they appreciate coming to Pinterest to take a break from the messy online world, to seek inspiration, dream, shop, and build a life they love. And while we are certainly proud of the work we have done to carve out this positive corner of the internet, technology and life move quickly, and there is always more we can do.While global decision makers debate the merits and faults of engagement-based ranking and depersonalized feeds, Pinterest is proving there is a better and healthier way to personalize content. By using surveys to create personalized, inspirational experiences for users and embodying the principles of the Inspired Internet Pledge, we are proving that platforms can create a safer and healthier experience for everyone.Healthier Personalization with Surveys was originally published in Pinterest Engineering Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.