Content-based Restaurant Recommender System

from ast import literal_eval import nltk from nltk.corpus import stopwords from nltk.tokenize import word_tokenize from nltk.stem.wordnet import WordNetLemmatizer from sklearn.feature_extraction.text import TfidfVectorizer from sklearn.metrics.pairwise import linear_kernel import pandas as pd import sidetable # download nltk packages nltk.download('stopwords') nltk.download('punkt') nltk.download('averaged_perceptron_tagger') nltk.download('wordnet')

df = pd.read_csv('TA_restaurants_curated.csv') print('The dataset has %s rows and %s columns' % df.shape) display("Initially dataset", df)

The dataset has 125527 rows and 11 columns

Unnamed: 0int64

Nameobject

Martine of Martine's Table

De Silveren Spiegel

La Rive

Vinkeles

Librije's Zusje Amsterdam

Ciel Bleu Restaurant

Zaza's

Blue Pepper Restaurant And Candlelight Cruises

Teppanyaki Restaurant Sazanka

Rob Wigboldus Vishandel

# Drop unnecessary columns df.drop(columns=['Unnamed: 0','URL_TA','ID_TA', 'Ranking', 'Number of Reviews', 'Rating'], inplace=True) # Rename columns for better convention df.rename(columns={'Name': 'name', 'City': 'city', 'Cuisine Style': 'style', 'Price Range': 'price', 'Reviews': 'reviews' }, inplace=True) display('Dataset after attributes reduction and renaming', df.head())

nameobject

cityobject

Martine of Martine's Table

Amsterdam

De Silveren Spiegel

Amsterdam

La Rive

Amsterdam

Vinkeles

Amsterdam

Librije's Zusje Amsterdam

Amsterdam

display('Missing values', df.stb.missing())

missingint64

totalint64

price

47855

125527

style

31351

125527

reviews

9616

125527

name

125527

city

125527

# Replace reviews that are empty to pandas NA df['reviews'].replace("[[], []]", pd.NA, inplace=True) print("Before drop: %s" % df.shape[0]) df.dropna(inplace=True) print("After drop: %s" % df.shape[0])

Before drop: 125527
After drop: 71631

display('Dataset after data reduction', df)

nameobject

McDonald's0.1%

Pizza Express0.1%

65483 others99.8%

cityobject

Martine of Martine's Table

Amsterdam

De Silveren Spiegel

Amsterdam

La Rive

Amsterdam

Vinkeles

Amsterdam

Librije's Zusje Amsterdam

Amsterdam

Ciel Bleu Restaurant

Amsterdam

Zaza's

Amsterdam

Blue Pepper Restaurant And Candlelight Cruises

Amsterdam

Teppanyaki Restaurant Sazanka

Amsterdam

Rob Wigboldus Vishandel

Amsterdam

df['reviews'] = df['reviews'].str.lower() df['style'] = df['style'].str.lower() df['city'] = df['city'].str.lower() display('Dataset after converting to lower case', df.head())

nameobject

cityobject

Martine of Martine's Table

amsterdam

De Silveren Spiegel

amsterdam

La Rive

amsterdam

Vinkeles

amsterdam

Librije's Zusje Amsterdam

amsterdam

# Convert strings to python list df['style'] = df['style'].apply(lambda x: literal_eval(x)) # Join the list items together df['style'] = df['style'].apply(lambda x: ', '.join(x))

# For `reviews` column, we have some (5) dirty items that we need # to handle explicitly and then remove them from dataset. def convert_to_list(x): try: return literal_eval(x) except ValueError: return pd.NA # Convert strings to python list and dirty data to pandas NA df['reviews'] = df['reviews'].apply(convert_to_list) # Drop dirty items print("Before drop: %s" % df.shape[0]) df.dropna(inplace=True) print("After drop: %s" % df.shape[0]) # Join the list items together, notice that we only send first item # of list (reviews) to the function and ignore the second (dates) df['reviews'] = df['reviews'].apply(lambda x: ', '.join(x[0])) display('Dataset after we clean `reviews` and `style` columns', df.head())

Before drop: 71631
After drop: 71626

nameobject

cityobject

Martine of Martine's Table

amsterdam

De Silveren Spiegel

amsterdam

La Rive

amsterdam

Vinkeles

amsterdam

Librije's Zusje Amsterdam

amsterdam

df['price'].unique()

df['price'].replace(['$', '$$ - $$$', '$$$$'], ['cheap-eats', 'mid-range', 'fine-dining'], inplace=True)

nameobject

McDonald's0.1%

Pizza Express0.1%

65478 others99.8%

cityobject

Martine of Martine's Table

amsterdam

De Silveren Spiegel

amsterdam

La Rive

amsterdam

Vinkeles

amsterdam

Librije's Zusje Amsterdam

amsterdam

Ciel Bleu Restaurant

amsterdam

Zaza's

amsterdam

Blue Pepper Restaurant And Candlelight Cruises

amsterdam

Teppanyaki Restaurant Sazanka

amsterdam

Rob Wigboldus Vishandel

amsterdam

print('Number of cities: %s ' % df['city'].nunique()) cities_list = df['city'].unique() print(cities_list)

Number of cities: 31 
['amsterdam' 'athens' 'barcelona' 'berlin' 'bratislava' 'brussels'
 'budapest' 'copenhagen' 'dublin' 'edinburgh' 'geneva' 'hamburg'
 'helsinki' 'krakow' 'lisbon' 'ljubljana' 'london' 'luxembourg' 'lyon'
 'madrid' 'milan' 'munich' 'oporto' 'oslo' 'paris' 'prague' 'rome'
 'stockholm' 'vienna' 'warsaw' 'zurich']

# Load cities dataset as pandas data frame: word_cities_df = pd.read_csv('cities.csv') # Convert all attributes to lowercase: word_cities_df['country'] = word_cities_df['country'].str.lower() word_cities_df['city'] = word_cities_df['city'].str.lower() display("Sample of word cities dataset", word_cities_df.head())

cityobject

countryobject

amsterdam

netherlands

athens

greece

barcelona

spain

berlin

germany

bratislava

slovakia

# Merge for adding country column to data frame df = pd.merge(df, word_cities_df, on='city', how='inner') # Reindex data frame to move country next to city df = df.reindex(columns=['name', 'city', 'country', 'price', 'style', 'reviews']) display("Sample of our new data frame with country column", df.sample(5))

nameobject

cityobject

51658

Nodaiwa

paris

6228

Chicken Bar

barcelona

49610

KALUTITA - Restaurante Bar

oporto

13187

Chez le Grec

brussels

38153

Patin d'Or

luxembourg

print('Number of countries: %s ' % df['country'].nunique()) countries_list = df['country'].unique() print(countries_list)

Number of countries: 23 
['netherlands' 'greece' 'spain' 'germany' 'slovakia' 'brussels' 'hungary'
 'denmark' 'ireland' 'scotland' 'switzerland' 'finland' 'poland'
 'portugal' 'slovenia' 'england' 'luxembourg' 'france' 'italy' 'norway'
 'czech' 'sweden' 'austria']

lemmatizer = WordNetLemmatizer() stop_words = set(stopwords.words('english')) def process_sentences(text): temp_sent =[] # Tokenize words words = nltk.word_tokenize(text) # Lemmatize each of the words based on their position in the sentence tags = nltk.pos_tag(words) for i, word in enumerate(words): if tags[i][1] in ('VB', 'VBD', 'VBG', 'VBN', 'VBP', 'VBZ'): # only verbs lemmatized = lemmatizer.lemmatize(word, 'v') else: lemmatized = lemmatizer.lemmatize(word) # Remove stop words and non alphabet tokens if lemmatized not in stop_words and lemmatized.isalpha(): temp_sent.append(lemmatized) # Some other clean-up full_sentence = ' '.join(temp_sent) full_sentence = full_sentence.replace("n't", " not") full_sentence = full_sentence.replace("'m", " am") full_sentence = full_sentence.replace("'s", " is") full_sentence = full_sentence.replace("'re", " are") full_sentence = full_sentence.replace("'ll", " will") full_sentence = full_sentence.replace("'ve", " have") full_sentence = full_sentence.replace("'d", " would") return full_sentence

df['reviews_processed'] = df['reviews'].apply(process_sentences)

df['style_processed'] = df['style'].apply(process_sentences)

df[['reviews', 'reviews_processed', 'style', 'style_processed']].sample(5)

reviewsobject

reviews_processedobject

9322

great pub, great music!, food&fun

great pub great music food fun

56149

gluten free thai food, palatable food, disappointing service

gluten free thai food palatable food disappointing service

12879

a perfect place to relax and enjoy life, a bar restaurant that provides an atmosphe...

perfect place relax enjoy life bar restaurant provide atmosphe

60185

tasty pelmeni, best dumplings i've ever had!

tasty pelmeni best dumpling ever

43590

sushi on the run, sushi variety

sushi run sushi variety

df['bag_of_words'] = df['style_processed'] + ' ' + df['reviews_processed'] display('Sample of bag_of_words attribute', df[['style', 'reviews' ,'bag_of_words']].sample(5))

styleobject

reviewsobject

6629

turkish, middle eastern, mediterranean, vegetarian friendly, vegan options, halal

good and cheap, simple but sustaining

68807

italian, mediterranean, european

little italy in vienna, tooooooooooooo salty

37414

seafood

poor uncooked food, the best in town

27565

european, british, wine bar, gluten free options

excellent light late lunch, dinner

57508

french

crouded with no taste food, nice stop

price_map = { 'cheap-eats': ('cheap', 'inexpensive', 'low-price', 'low-cost', 'economical', 'economic', 'affordable'), 'mid-range': ('moderate', 'fair', 'mid-price', 'reasonable', 'average'), 'fine-dining': ('expensive', 'fancy', 'lavish') }

def recommend(description): # Convert user input to lowercase description = description.lower() data = df.copy() # Extract cities cities_input = [] for city in cities_list: if city in description: cities_input.append(city) description = description.replace(city, "") if cities_input: data = data[data['city'].isin(cities_input)] # Same thing for countries countries_input = [] for country in countries_list: if country in description: countries_input.append(country) description = description.replace(country, "") if countries_input: data = data[data['country'].isin(countries_input)] # Extract price class for key, value in price_map.items(): if any(v in description for v in value): data = data[data['price'] == key] break # Process user description text input description = process_sentences(description) description = description.strip() print('Processed user feedback:', description) # Init a TF-IDF vectorizer tfidfvec = TfidfVectorizer() # Fit data on processed reviews vec = tfidfvec.fit(data["bag_of_words"]) features = vec.transform(data["bag_of_words"]) # Transform user input data based on fitted model description_vector = vec.transform([description]) # Calculate cosine similarities between users processed input and reviews cos_sim = linear_kernel(description_vector, features) # Add similarities to data frame data['similarity'] = cos_sim[0] # Sort data frame by similarities data.sort_values(by='similarity', ascending=False, inplace=True) return data[['name', 'city', 'country', 'price', 'style', 'reviews', 'similarity']]

recommend('for vietnamese food')

Processed user feedback: vietnamese food

nameobject

McDonald's0.1%

Pizza Express0.1%

65478 others99.8%

cityobject

42281

Le Petit Hanoi

madrid

58498

Bo Bun

paris

58140

Nem 101

paris

56793

Restaurant Le Riz

paris

56931

Hanoi Vietnamien

paris

21649

Souply

hamburg

34251

Anjoy Kitchen

london

55423

Van-An

paris

58055

Piment d'Or

paris

55968

My Canh

paris

# Feedback with multiples cities recommend('good for party in Paris or London')

Processed user feedback: good party

nameobject

Pizza Express0.4%

Pret a Manger0.2%

18785 others99.4%

cityobject

london57.2%

paris42.8%

30676

Star & Garter - London

london

31709

The Joker of Penton Street

london

32932

Victorian Bath House

london

31829

Dirty Martini St Paul's

london

56692

ALIMENTARI

paris

36489

The Union Chiswick

london

57675

Le Pavillon des Princes

paris

31074

The Bootlegger

london

27568

Octoberfest Pub London

london

28687

The Chapel Bar

london

# feedback with country recommend('vegan in Finland')

Processed user feedback: vegan

nameobject

McDonald's0.4%

Sushibar + Wine0.4%

740 others99.2%

cityobject

helsinki100%

22325

Kippo

helsinki

22241

Soi Soi

helsinki

22069

Silvoplee

helsinki

22216

Ravintola Loung3

helsinki

22023

Farang

helsinki

22254

Korphai

helsinki

22231

Himalaya

helsinki

22203

Piece 'N' Love

helsinki

22028

Pompier Espa

helsinki

22347

Egg

helsinki

# with price class recommend('fancy for meetings')

Processed user feedback: fancy meeting

nameobject

Assunta Madre0.1%

Senses Restaurant0.1%

3867 others99.8%

cityobject

paris16.1%

london15.5%

29 others68.4%

56552

Pavillon Royal

paris

26580

The Greenhouse

london

30681

L'Etranger

london

30254

Amaranto Lounge at the Four Seasons

london

51446

Restaurant Le 39 V

paris

33178

Brunello Bar and Restaurant

london

62067

Ristorante Livello1

rome

38567

Restaurant Pierre Orsi

lyon

51137

Bon

paris

51547

Cristal Room Baccarat

paris

# with price class and location recommend('a reasonable breakfast in France')

Processed user feedback: reasonable breakfast

nameobject

Hippopotamus0.3%

Sushi Shop0.2%

7104 others99.5%

cityobject

paris88%

lyon12%

39327

Maison Pochat

lyon

55105

Le Musset

paris

58116

Le Petit Maillot

paris

38841

Soleil Levant

lyon

54049

Le Twickenham

paris

53724

Fujiyama paris etoile

paris

39529

Suelta

lyon

53186

Entre 2 Rives

paris

52410

PaperBoy

paris

58671

Le Daniel's

paris

recommend('romantic cafe for dating')

Processed user feedback: romantic cafe date

nameobject

McDonald's0.1%

Pizza Express0.1%

65478 others99.8%

cityobject

35235

Cafe La Diva

london

26943

Allora

london

59669

Soho+

prague

54345

Connetable Le

paris

15423

Korona restaurant

budapest

52526

Le Saint Placide

paris

36241

Seema Tandoori

london

60235

Barocco Veneziano Cafe

prague

8074

Ristorantes Sandro

barcelona

19842

China Palace

edinburgh

recommend('a bar with live music and good atmosphere')

Processed user feedback: bar live music good atmosphere

nameobject

McDonald's0.1%

Pizza Express0.1%

65478 others99.8%

cityobject

17870

Sin E Dublin

dublin

4686

Obama

barcelona

64844

Alvarado Street

rome

57170

Le Kaskad Cafe

paris

14414

Fat Mo's Restaurant & Music Pub

budapest

13346

Sazz'n Jazz

brussels

57166

Gandhi Mahal

paris

13410

Brasserie de l'Union

brussels

17618

O'donoghue's Bar

dublin

4846

Milano Cocktail Bar

barcelona

recommend('nice place for valentine date')

Processed user feedback: nice place valentine date

nameobject

McDonald's0.1%

Pizza Express0.1%

65478 others99.8%

cityobject

12007

Jedna basen - Dlhe Diely

bratislava

38723

Cuisine et dependances acte 2

lyon

35235

Cafe La Diva

london

36536

Talay Thai Restaurant

london

44917

Da Giannino L'angolo d'Abruzzo 2

milan

26508

Fredericks Restaurant

london

36241

Seema Tandoori

london

23593

IIIMPAR

lisbon

19842

China Palace

edinburgh

36780

Horse & Groom

london

recommend('good quality espresso in italy')

Processed user feedback: good quality espresso

nameobject

Alice Pizza0.1%

Old Wild West0.1%

7389 others99.8%

cityobject

rome55.1%

milan44.9%

64804

Pappagallo

rome

44909

caffe Rocca

milan

63763

No Name Cafe

rome

65924

Foglia d'Oro

rome

45555

Il Forno

milan

44973

Ristorante Da Stefano Il Marchigiano

milan

65902

La Pizza di Ma.Ma

rome

62584

Agora' Restaurant & Bar

rome

62819

D'Angelo - Gastronomia Caffe

rome

46825

Mitsui

milan