Launching Google Maps in South Korea

Problem statement: Imagine Google Maps wants to launch in South Korea and needs to decide between three local vendors providing map data. How will you as a Product Manager help your VP products make this decision?

To scope out the problem let’s start with a couple of clarifying questions.

Me: I assume we are trying to launch for the first time in South Korea, right? If yes, why now? The reason I ask this is because if we have tried it in the past, I can build on those learnings.

Interviewer: Ya this is the first time. It just fits our strategy right now to launch there.

Me: We have to make a choice between three vendors, were there only three or have we narrowed it down to three from a bigger set?

Interviewer: These are the top 3 market leaders who even have the kind of data we need for Google Maps. There are a few smaller vendors but they can’t seem to fulfill a few legal and scale requirements.

Me: Great, thanks for clarifying. Now that I have these answers in place, I’d like to start with what Google Maps does and then evaluate these vendors on a few parameters and finally come up with how we will validate the data sets. Sounds good?

Interviewer: Ya.

So Google Maps is a product that helps people explore places and find directions from point A to point B. Customers trust Google to provide all of the information accurately.

So here are a few parameters that I’d like to start with:

• 1. Cost: What is the cost associated with each of the vendors, how does pricing work ? Is it per API call, is it based on # of results returned etc? Even though cost is an important factor for Google, “accuracy” will trump over pricing.
• 2. Data: This feels like a blanket statement and therefore needs to be broken down further. Some important questions that need to be asked are:

• - How did the vendor get the data? What were the methods used, how was it sourced/collected? This gives us a lot of context and an insight into vendor practices and is an indicator of data quality.
• - Does the data cover rural and urban areas equally well?
• - Are we getting all the dimensions like images, lat and long, modes of transport etc that we need to fulfill all the functionalities of Google Maps?
• - If there are some dimensions missing, which ones are they? Can we get that elsewhere? For example: Store phone number is missing as a part of the business address, can we crowdsource it ?
• - How often is the data updated? If a place is closed, when does it get removed from the data set? How are we notified? This is to ensure we show the right information to our users every time.
• - Is the data in English or Korean or both? Is there any translation required ?
• Is there any IP or added advantage that we are getting from any of the vendors?

• 3. Integrations: What sort of integrations are required to use the data? Are there implementation challenges that we foresee? Again this is quite low on the deciding factor as Google houses some of the best engineers who solve tough problems day in and day out.

Now after considering all these factors we also need to validate the data/claims made by the vendors. For this, let’s look at the use-cases for Google Maps.

There are two broad use-cases:

• - Finding businesses on the map
• - Getting directions

and the two metrics that matter are:

• - How many people are able to find the right results and accurate information every time they search for a location/business.
• - How accurate is the route, the estimated travel time and location details of the business/place as calculated by Google Maps.

Therefore, precision and recall are two important factors to consider for data quality.

We can do an A/B test between the vendors where users will be split in 3 groups and each will use different vendor data to find places and directions. The users will need to input their feedback about getting the right search result and reaching the right destination in the stipulated time that will lead up to a final score. The dimensions of rural vs urban, different modes of transport will also have to be tested to see the accuracy of results in all use cases. This experiment will have to run till we reach statistically significant results.

We can also use a golden data set (data that we know is true) to validate each vendor data against it.

To summarize this, people trust Google for the information it provides and therefore that is the most important factor while making trade-offs and selecting a vendor. There are several factors outlined above that contribute towards the decision, some weighing heavier than the other.

‍