Skip to content

Clarify review weighting in Yelp average-rating questions #97

Description

@jackn11

Several Yelp questions ask for the “average rating of businesses,” but their golden answers pool all individual reviews. This weights businesses according to their number of reviews.

A natural alternative is:

  1. Calculate the mean review rating for each business.
  2. Average those business-level ratings, giving every business equal weight.

The two calculations differ in all three affected questions:

Query Pooled-review mean Equal-business mean Golden
Indianapolis businesses 3.547008547 3.497493225 3.547008547
Businesses in top-review state (PA) 3.699395770 3.648005824 3.699395770
WiFi businesses in PA 3.483985765 3.582654975 3.48

The third pooled value matches the golden after rounding.

Affected questions include:

What is the average rating of all businesses located in Indianapolis, Indiana?

Which U.S. state has the highest number of reviews, and what is the average rating of businesses in that state?

Which U.S. state has the highest number of businesses that offer WiFi, and what is the average rating for those businesses?

Could the intended weighting be clarified?

If pooling reviews is intended, wording such as “average rating across all reviews for those businesses” would make the expected calculation explicit. If businesses should be weighted equally, the golden answers should instead use the business-level means.

Without clarification, semantically reasonable solutions can produce different answers.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions