from google.colab import drive
drive.mount('/content/drive')
Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly
Enter your authorization code:
··········
Mounted at /content/drive
import sys
!apt-get update
!apt install chromium-chromedriver
!cp /usr/lib/chromium-browser/chromedriver /usr/bin
!pip install selenium
# set options to be headless, ..
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver import ActionChains
from bs4 import BeautifulSoup as bs
import re
import time
Get:1 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ InRelease [3,626 B]
Ign:2 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease
Ign:3 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 InRelease
Hit:4 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 Release
Hit:5 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 Release
Hit:6 http://archive.ubuntu.com/ubuntu bionic InRelease
Hit:7 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic InRelease
Get:8 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ Packages [92.5 kB]
Get:9 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:10 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:12 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic InRelease [15.4 kB]
Get:14 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:15 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main Sources [1,818 kB]
Get:16 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [930 kB]
Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [73.6 kB]
Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,226 kB]
Get:19 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [852 kB]
Get:20 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [59.3 kB]
Get:21 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [8,815 B]
Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,385 kB]
Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [20.1 kB]
Get:24 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic/main amd64 Packages [876 kB]
Fetched 7,614 kB in 4s (2,042 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
chromium-browser chromium-browser-l10n chromium-codecs-ffmpeg-extra
Suggested packages:
webaccounts-chromium-extension unity-chromium-extension adobe-flashplugin
The following NEW packages will be installed:
chromium-browser chromium-browser-l10n chromium-chromedriver
chromium-codecs-ffmpeg-extra
0 upgraded, 4 newly installed, 0 to remove and 52 not upgraded.
Need to get 77.3 MB of archives.
After this operation, 264 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 chromium-codecs-ffmpeg-extra amd64 81.0.4044.138-0ubuntu0.18.04.1 [1,095 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 chromium-browser amd64 81.0.4044.138-0ubuntu0.18.04.1 [68.9 MB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 chromium-browser-l10n all 81.0.4044.138-0ubuntu0.18.04.1 [3,231 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 chromium-chromedriver amd64 81.0.4044.138-0ubuntu0.18.04.1 [4,079 kB]
Fetched 77.3 MB in 5s (14.9 MB/s)
Selecting previously unselected package chromium-codecs-ffmpeg-extra.
(Reading database ... 144433 files and directories currently installed.)
Preparing to unpack .../chromium-codecs-ffmpeg-extra_81.0.4044.138-0ubuntu0.18.04.1_amd64.deb ...
Unpacking chromium-codecs-ffmpeg-extra (81.0.4044.138-0ubuntu0.18.04.1) ...
Selecting previously unselected package chromium-browser.
Preparing to unpack .../chromium-browser_81.0.4044.138-0ubuntu0.18.04.1_amd64.deb ...
Unpacking chromium-browser (81.0.4044.138-0ubuntu0.18.04.1) ...
Selecting previously unselected package chromium-browser-l10n.
Preparing to unpack .../chromium-browser-l10n_81.0.4044.138-0ubuntu0.18.04.1_all.deb ...
Unpacking chromium-browser-l10n (81.0.4044.138-0ubuntu0.18.04.1) ...
Selecting previously unselected package chromium-chromedriver.
Preparing to unpack .../chromium-chromedriver_81.0.4044.138-0ubuntu0.18.04.1_amd64.deb ...
Unpacking chromium-chromedriver (81.0.4044.138-0ubuntu0.18.04.1) ...
Setting up chromium-codecs-ffmpeg-extra (81.0.4044.138-0ubuntu0.18.04.1) ...
Setting up chromium-browser (81.0.4044.138-0ubuntu0.18.04.1) ...
update-alternatives: using /usr/bin/chromium-browser to provide /usr/bin/x-www-browser (x-www-browser) in auto mode
update-alternatives: using /usr/bin/chromium-browser to provide /usr/bin/gnome-www-browser (gnome-www-browser) in auto mode
Setting up chromium-chromedriver (81.0.4044.138-0ubuntu0.18.04.1) ...
Setting up chromium-browser-l10n (81.0.4044.138-0ubuntu0.18.04.1) ...
Processing triggers for hicolor-icon-theme (0.17-2) ...
Processing triggers for mime-support (3.60ubuntu1) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
cp: '/usr/lib/chromium-browser/chromedriver' and '/usr/bin/chromedriver' are the same file
Collecting selenium
Downloading https://files.pythonhosted.org/packages/80/d6/4294f0b4bce4de0abf13e17190289f9d0613b0a44e5dd6a7f5ca98459853/selenium-3.141.0-py2.py3-none-any.whl (904kB)
|████████████████████████████████| 911kB 4.7MB/s
Requirement already satisfied: urllib3 in /usr/local/lib/python3.6/dist-packages (from selenium) (1.24.3)
Installing collected packages: selenium
Successfully installed selenium-3.141.0
import pprint
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
top_url = 'https://play.google.com/store/apps/collection/topgrossing?clp=ChMKEQoLdG9wZ3Jvc3NpbmcQBxgD:S:ANO1ljJpBI4&gsr=ChUKEwoRCgt0b3Bncm9zc2luZxAHGAM%3D:S:ANO1ljJ_fZk&hl=ko'
wd = webdriver.Chrome('chromedriver',options=options)
wd.get(top_url)
wd.implicitly_wait(2)
for _ in range(2) :
wd.execute_script("window.scrollTo(0, document.body.scrollHeight);")
time.sleep(1)
html = bs(wd.page_source, 'html.parser')
# display (html)
wd.close()
apps = html.find_all('div', {'class' : 'b8cIId ReQCgd Q9MA7b'})
# apps
apps_list = []
for a in apps :
apps_list.append([a.text, a.find('a')['href'], ])
apps_list
# url
len(apps_list)
import pandas as pd
df_AppList = pd.DataFrame(apps_list)
df_AppList
'http://en.mobileindex.com/app/?code=com.kakao.talk&mk=12&c=kr'.index('=')
('http://en.mobileindex.com/app/?code=com.kakao.talk&mk=12&c=kr'[36:])[:-11]
wd = webdriver.Chrome('chromedriver', options=options)
url = 'http://en.mobileindex.com/app/rank'
wd.get(url)
wd.implicitly_wait(2)
apps_list = []
for idx, app in enumerate(wd.find_elements_by_class_name('appTitle')) :
if (idx+1) % 3 == 0 :
title = app.find_element_by_class_name('appname').text
link = (app.find_element_by_tag_name('a').get_attribute("href")[36:])[:-11]
apps_list.append([title, link])
display (apps_list)
wd.close()
len(apps_list)
apps_list[0][1]
url_prefix = 'https://play.google.com/store/apps/details?hl=ko&showAllReviews=true&id='
url_full = url_prefix + apps_list[0][1]
url_full
'별표 5개 만점에 4개를 받았습니다.'.index('4')
wd = webdriver.Chrome('chromedriver', options=options)
wd.get(url_full)
wd.implicitly_wait(2)
review_list = []
more_nums = 0
while True :
# more_nums += 1
wd.execute_script('window.scrollTo(0, document.body.scrollHeight);')
# wd.implicitly_wait(1)
time.sleep(1)
more_nums +=1
print (more_nums, end='\t')
if more_nums >30 : break
try :
# more_btn = wd.find_element_by_xpath('//*[@id="fcxH9b"]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div')
# if more_nums % 4 == 0 :
# more_btn = wd.find_element_by_class_name('M9Bg4d')
# ('#fcxH9b > div.WpDbMd > c-wiz > div > div.ZfcPIb > div > div.JNury.Ekdcne > div > div > div.W4P4ne > div:nth-child(2) > div.PFAhAf > div')
# more_btn = wd.find_element_by_class_name('U26fgb O0WRkf oG5Srb C0oVfc n9lfJ M9Bg4d')
more_btn = wd.find_element_by_xpath('/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div')
print (more_btn.text, end='\t')
if more_nums % 2 == 1 :
# more_btn = wd.find_element_by_xpath('//*[@id="fcxH9b"]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div/span/span')
# more_btn = wd.find_element_by_class_name('PFAhAf').find_element_by_class_name('RveJvd snByac')
# print (more_btn.text )
ActionChains(wd).click(more_btn).perform()
# more_btn.click
# print (more_btn.size)
print ('pass to..')
# break
except Exception:
print ('re..')
# continue
for idx, one in enumerate(wd.find_elements_by_class_name('zc7KVe')) :
# print (idx)
if idx % 2 ==0 :
# writer = one.find_element_by_class_name('X43Kjb').text
# print (writer)
try :
star = one.find_element_by_class_name('pf5lIe').find_element_by_tag_name('div').get_attribute('aria-label')[10]
# print (star)
review = one.find_element_by_class_name('UD7Dzf').text
# display (review)
review_list.append([int(idx/2), star, review])
except :
print (f"error __{int(idx/2)}")
review_list.append([int(idx/2), "error", "error",])
# display (review_list)
wd.close()
1 re.. 2 pass to.. 3 더보기 pass to.. 4 pass to.. 5 re.. 6 pass to.. 7 더보기 pass to.. 8 pass to.. 9 re.. 10 pass to.. 11 더보기 pass to.. 12 pass to.. 13 re.. 14 pass to.. 15 더보기 pass to.. 16 pass to.. 17 re.. 18 pass to.. 19 더보기 pass to.. 20 pass to.. 21 re.. 22 pass to.. 23 더보기 pass to.. 24 pass to.. 25 re.. 26 pass to.. 27 더보기 pass to.. 28 pass to.. 29 re.. 30 pass to.. 31
more_btn
wd.find_element_by_class_name('PFAhAf').find_element_by_class_name('CwaK9').text
len(wd.find_elements_by_class_name('zc7KVe'))
wd.find_elements_by_class_name('zc7KVe')[2].find_element_by_class_name('X43Kjb').text
review_list = []
for idx, one in enumerate(wd.find_elements_by_class_name('zc7KVe')) :
print (idx)
writer = one.find_element_by_class_name('X43Kjb').text
print (writer)
star = one.find_element_by_class_name('pf5lIe').find_element_by_tag_name('div').get_attribute('aria-label')
print (star)
# review = one.find_element_by_class_name('UD7Dzf').text
# display (review)
review_list.append([idx, star, review])
0
박주희
별표 5개 만점에 4개를 받았습니다.
1
박주희
별표 5개 만점에 4개를 받았습니다.
2
라푸랴 [lafurya]
별표 5개 만점에 4개를 받았습니다.
3
라푸랴 [lafurya]
별표 5개 만점에 4개를 받았습니다.
4
다키
별표 5개 만점에 4개를 받았습니다.
5
다키
별표 5개 만점에 4개를 받았습니다.
6
계정
별표 5개 만점에 1개를 받았습니다.
7
계정
별표 5개 만점에 1개를 받았습니다.
8
멍멍멍멍멍
별표 5개 만점에 3개를 받았습니다.
9
멍멍멍멍멍
별표 5개 만점에 3개를 받았습니다.
10
가온 GAON
별표 5개 만점에 4개를 받았습니다.
11
가온 GAON
별표 5개 만점에 4개를 받았습니다.
12
루민Lumin
별표 5개 만점에 2개를 받았습니다.
13
루민Lumin
별표 5개 만점에 2개를 받았습니다.
14
x
별표 5개 만점에 4개를 받았습니다.
15
x
별표 5개 만점에 4개를 받았습니다.
16
안인숙
별표 5개 만점에 1개를 받았습니다.
17
안인숙
별표 5개 만점에 1개를 받았습니다.
18
CHOI TORICONA
별표 5개 만점에 2개를 받았습니다.
19
CHOI TORICONA
별표 5개 만점에 2개를 받았습니다.
20
레이
별표 5개 만점에 3개를 받았습니다.
21
레이
별표 5개 만점에 3개를 받았습니다.
22
답글로 한 마디만 할게,야이 쉐끼야 내가
별표 5개 만점에 5개를 받았습니다.
23
답글로 한 마디만 할게,야이 쉐끼야 내가
별표 5개 만점에 5개를 받았습니다.
24
Zeus Red
별표 5개 만점에 1개를 받았습니다.
25
Zeus Red
별표 5개 만점에 1개를 받았습니다.
26
ᄂᄂ
별표 5개 만점에 2개를 받았습니다.
27
ᄂᄂ
별표 5개 만점에 2개를 받았습니다.
28
강병관
별표 5개 만점에 2개를 받았습니다.
29
강병관
별표 5개 만점에 2개를 받았습니다.
30
Ma木유안
별표 5개 만점에 5개를 받았습니다.
31
Ma木유안
별표 5개 만점에 5개를 받았습니다.
32
소미
별표 5개 만점에 1개를 받았습니다.
33
소미
별표 5개 만점에 1개를 받았습니다.
34
키리시마 코유키[霧島 粉雪]
별표 5개 만점에 5개를 받았습니다.
35
키리시마 코유키[霧島 粉雪]
별표 5개 만점에 5개를 받았습니다.
36
천열화
별표 5개 만점에 3개를 받았습니다.
37
천열화
별표 5개 만점에 3개를 받았습니다.
38
현
별표 5개 만점에 5개를 받았습니다.
39
현
별표 5개 만점에 5개를 받았습니다.
40
고경수
별표 5개 만점에 3개를 받았습니다.
41
고경수
별표 5개 만점에 3개를 받았습니다.
42
_고야미
별표 5개 만점에 1개를 받았습니다.
43
_고야미
별표 5개 만점에 1개를 받았습니다.
44
ᄀᄌᄒ
별표 5개 만점에 3개를 받았습니다.
45
ᄀᄌᄒ
별표 5개 만점에 3개를 받았습니다.
46
하리이뽀
별표 5개 만점에 5개를 받았습니다.
47
하리이뽀
별표 5개 만점에 5개를 받았습니다.
48
JIYOUNG LEE
별표 5개 만점에 3개를 받았습니다.
49
JIYOUNG LEE
별표 5개 만점에 3개를 받았습니다.
50
박민지
별표 5개 만점에 3개를 받았습니다.
51
박민지
별표 5개 만점에 3개를 받았습니다.
52
우승원
별표 5개 만점에 2개를 받았습니다.
53
우승원
별표 5개 만점에 2개를 받았습니다.
54
Googol
별표 5개 만점에 1개를 받았습니다.
55
Googol
별표 5개 만점에 1개를 받았습니다.
56
Ji Bong Jang
별표 5개 만점에 5개를 받았습니다.
57
Ji Bong Jang
별표 5개 만점에 5개를 받았습니다.
58
6mL리퍼 0
별표 5개 만점에 4개를 받았습니다.
59
6mL리퍼 0
별표 5개 만점에 4개를 받았습니다.
60
22 Raptor
별표 5개 만점에 2개를 받았습니다.
61
22 Raptor
별표 5개 만점에 2개를 받았습니다.
62
서정인
별표 5개 만점에 1개를 받았습니다.
63
서정인
별표 5개 만점에 1개를 받았습니다.
64
Dms 은방울꽃
별표 5개 만점에 4개를 받았습니다.
65
Dms 은방울꽃
별표 5개 만점에 4개를 받았습니다.
66
은아는치킨먹고싶오
별표 5개 만점에 3개를 받았습니다.
67
은아는치킨먹고싶오
별표 5개 만점에 3개를 받았습니다.
68
기경호
별표 5개 만점에 1개를 받았습니다.
69
기경호
별표 5개 만점에 1개를 받았습니다.
70
김재현
별표 5개 만점에 1개를 받았습니다.
71
김재현
별표 5개 만점에 1개를 받았습니다.
72
김경련
별표 5개 만점에 1개를 받았습니다.
73
김경련
별표 5개 만점에 1개를 받았습니다.
74
알리움
별표 5개 만점에 1개를 받았습니다.
75
알리움
별표 5개 만점에 1개를 받았습니다.
76
지옌링
별표 5개 만점에 3개를 받았습니다.
77
지옌링
별표 5개 만점에 3개를 받았습니다.
78
나만이
별표 5개 만점에 2개를 받았습니다.
79
나만이
별표 5개 만점에 2개를 받았습니다.
import pandas as pd
df_review_list = pd.DataFrame(review_list, columns=['idx', 'star', 'review'])
df_review_list.set_index('idx', inplace=True)
display(df_review_list)
df_review_list.to_csv('kakao_review.csv')
def get_top50() :
print ("get top 50 app list from mobileindex.com")
wd = webdriver.Chrome('chromedriver', options=options)
url = 'http://en.mobileindex.com/app/rank'
wd.get(url)
wd.implicitly_wait(2)
apps_list = []
for idx, app in enumerate(wd.find_elements_by_class_name('appTitle')) :
if (idx+1) % 3 == 0 :
title = app.find_element_by_class_name('appname').text
link = (app.find_element_by_tag_name('a').get_attribute("href")[36:])[:-11]
apps_list.append([title, link])
# display (apps_list)
wd.close()
return apps_list
def review_crawler (app_name, app_url) :
print ("app_name : ", app_name, end='\t')
print ("app_url : ", app_url)
with webdriver.Chrome('chromedriver', options=options) as wd :
full_url = 'https://play.google.com/store/apps/details?hl=ko&showAllReviews=true&id=' + app_url
wd.get(full_url)
wd.implicitly_wait(2)
review_list = []
print (f"let's looping!")
page_length_list = []
for _ in range(40) :
print (f"{_+1}", end=' ')
wd.execute_script('window.scrollTo(0, document.body.scrollHeight);')
time.sleep(0.5)
elements = wd.find_elements_by_class_name('zc7KVe')
page_length = len(elements)
page_length_list.append(page_length )
print (len(page_length_list), page_length)
if len(page_length_list) > 4 and page_length_list[-1] == page_length_list[-4] : break
try :
more_btn = wd.find_element_by_xpath('/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div')
if _ % 2 == 1 :
ActionChains(wd).click(more_btn).perform()
except Exception as e :
print(e)
continue
print ("\nget items... ")
print (f"writing ...", end=' ')
try :
for idx, one in enumerate(elements) :
if idx % 200 == 0 :
print (f"no. {idx//2:4d}", end=' ')
if idx % 2 == 0 :
star = one.find_element_by_class_name('pf5lIe').find_element_by_tag_name('div').get_attribute('aria-label')[10]
review = one.find_element_by_class_name('UD7Dzf').text
review_list.append([int(idx/2), star, review])
except Exception as e :
print (f"error __{int(idx/2)} {e }")
review_list.append([int(idx/2), "error", "error",])
print ('\nmake csv file ...')
pd.DataFrame(review_list, columns=['idx', 'star', 'review']).set_index('idx').to_csv(f"/content/drive/My Drive/Programming/1. Docker/VSCode_/Java_Spring/Python/MLDL/ToyProject/{app_name[:5]}.csv")
print (f"\n{app_name} reviews crawling finished!\n")
return None
# return [app_name, review_list]
# print(review_crawler(apps_list[30][0], apps_list[30][1]))
apps_list = [ ['V LIVE', 'com.naver.vapp'] ]
# apps_list = get_top50()
reviews_list = [['app_name', 'app_reviews']]
# for app in apps_list[17:] :
review_crawler(app[0], app[1])
app_name : 랭킹스타 app_url : com.etech.starranking
let's looping!
1 1 2
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
2 2 2
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
3 3 2
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
4 4 2
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
5 5 2
get items...
writing ... no. 0
make csv file ...
랭킹스타 reviews crawling finished!
app_name : Tinder 틴더 app_url : com.tinder
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
5 5 400
6 6 400
7 7 560
8 8 640
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
9 9 720
10 10 720
11 11 720
12 12 800
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
13 13 960
14 14 1040
15 15 1200
16 16 1360
17 17 1520
18 18 1600
19 19 1840
20 20 1920
21 21 2160
22 22 2320
23 23 2400
24 24 2640
25 25 2800
26 26 2960
27 27 3120
28 28 3280
29 29 3440
30 30 3600
31 31 3760
32 32 3920
33 33 4080
34 34 4240
35 35 4400
36 36 4560
37 37 4720
38 38 4880
39 39 5040
40 40 5200
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900 no. 1000 no. 1100 no. 1200 no. 1300 no. 1400 no. 1500 no. 1600 no. 1700 no. 1800 no. 1900 no. 2000 no. 2100 no. 2200 no. 2300 no. 2400 no. 2500
make csv file ...
Tinder 틴더 reviews crawling finished!
app_name : Kmate 케이메이트 - 한국을 사랑하는 외국인친구 사귀기, 미팅, 외국어 채팅, 언어교환 app_url : com.mate.korean
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
5 5 400
6 6 400
7 7 560
8 8 640
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
9 9 720
10 10 720
11 11 880
12 12 960
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
13 13 1040
14 14 1040
15 15 1200
16 16 1280
17 17 1520
18 18 1600
19 19 1840
20 20 2000
21 21 2160
22 22 2320
23 23 2480
24 24 2640
25 25 2800
26 26 2960
27 27 3120
28 28 3280
29 29 3440
30 30 3600
31 31 3760
32 32 3920
33 33 4080
34 34 4240
35 35 4400
36 36 4560
37 37 4720
38 38 4880
39 39 5040
40 40 5132
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900 no. 1000 no. 1100 no. 1200 no. 1300 no. 1400 no. 1500 no. 1600 no. 1700 no. 1800 no. 1900 no. 2000 no. 2100 no. 2200 no. 2300 no. 2400 no. 2500
make csv file ...
Kmate 케이메이트 - 한국을 사랑하는 외국인친구 사귀기, 미팅, 외국어 채팅, 언어교환 reviews crawling finished!
app_name : 리슨 Lysn app_url : com.everysing.lysn
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
5 5 400
6 6 400
7 7 560
8 8 640
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
9 9 720
10 10 720
11 11 880
12 12 1040
13 13 1120
14 14 1274
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
15 15 1274
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
16 16 1274
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
17 17 1274
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600
make csv file ...
리슨 Lysn reviews crawling finished!
app_name : 골드스푼 : 자격을 갖춘 남녀를 위한 안전한 소개팅 app_url : com.goldspoon
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
5 5 400
6 6 400
7 7 560
8 8 640
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
9 9 720
10 10 720
11 11 800
12 12 1040
13 13 1200
14 14 1280
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
15 15 1360
16 16 1360
17 17 1520
18 18 1680
19 19 1840
20 20 2000
21 21 2160
22 22 2320
23 23 2370
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
24 24 2370
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
25 25 2370
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
26 26 2370
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900 no. 1000 no. 1100
make csv file ...
골드스푼 : 자격을 갖춘 남녀를 위한 안전한 소개팅 reviews crawling finished!
app_name : 트롯픽 - TROTPICK app_url : com.donga.trotpick
let's looping!
1 1 80
2 2 126
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
3 3 126
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
4 4 126
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
5 5 126
get items...
writing ... no. 0
make csv file ...
트롯픽 - TROTPICK reviews crawling finished!
app_name : 아만다: 눈 높아도 괜찮은 프리미엄 소개팅 app_url : com.dorsia.amanda
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
5 5 400
6 6 400
7 7 560
8 8 640
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
9 9 720
10 10 720
11 11 800
12 12 960
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
13 13 1040
14 14 1040
15 15 1200
16 16 1360
17 17 1520
18 18 1680
19 19 1840
20 20 2000
21 21 2160
22 22 2320
23 23 2480
24 24 2640
25 25 2800
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
26 26 2960
27 27 3120
28 28 3280
29 29 3440
30 30 3600
31 31 3760
32 32 3920
33 33 4080
34 34 4240
35 35 4400
36 36 4560
37 37 4720
38 38 4880
39 39 5040
40 40 5200
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900 no. 1000 no. 1100 no. 1200 no. 1300 no. 1400 no. 1500 no. 1600 no. 1700 no. 1800 no. 1900 no. 2000 no. 2100 no. 2200 no. 2300 no. 2400 no. 2500
make csv file ...
아만다: 눈 높아도 괜찮은 프리미엄 소개팅 reviews crawling finished!
app_name : 오디오북의 시작 - 윌라 app_url : kr.co.influential.youngkangapp
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
5 5 400
6 6 400
7 7 560
8 8 640
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
9 9 720
10 10 720
11 11 800
12 12 960
13 13 1200
14 14 1280
15 15 1520
16 16 1680
17 17 1840
18 18 2000
19 19 2160
20 20 2320
21 21 2480
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
22 22 2640
23 23 2720
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
24 24 2868
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
25 25 2868
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
26 26 2868
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
27 27 2868
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900 no. 1000 no. 1100 no. 1200 no. 1300 no. 1400
make csv file ...
오디오북의 시작 - 윌라 reviews crawling finished!
app_name : 커넥팅 - 클린한 소셜 통화 어플 app_url : com.yangpark.connecting
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
5 5 400
6 6 400
7 7 560
8 8 720
9 9 800
10 10 960
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
11 11 1040
12 12 1040
13 13 1200
14 14 1280
15 15 1520
16 16 1680
17 17 1840
18 18 2000
19 19 2160
20 20 2320
21 21 2480
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
22 22 2640
23 23 2800
24 24 2960
25 25 3120
26 26 3280
27 27 3348
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
28 28 3348
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
29 29 3348
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
30 30 3348
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900 no. 1000 error __1044 Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
(Session info: headless chrome=81.0.4044.138)
make csv file ...
커넥팅 - 클린한 소셜 통화 어플 reviews crawling finished!
app_name : 너랑나랑 - 매일16명 소개팅 (소셜데이팅,연애) app_url : com.endressdreamky.loveting
let's looping!
1 1 80
2 2 160
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
3 3 160
4 4 240
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
5 5 240
6 6 320
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
7 7 320
8 8 400
9 9 480
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
10 10 560
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
11 11 640
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
12 12 720
13 13 800
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
14 14 960
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
15 15 1040
16 16 1040
17 17 1120
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
18 18 1280
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
19 19 1360
20 20 1360
21 21 1520
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
22 22 1600
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
23 23 1680
24 24 1680
25 25 1760
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
26 26 1920
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
27 27 2000
28 28 2000
29 29 2000
30 30 2000
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900
make csv file ...
너랑나랑 - 매일16명 소개팅 (소셜데이팅,연애) reviews crawling finished!
app_name : ZEPETO app_url : me.zepeto.main
let's looping!
1 1 80
2 2 160
3 3 320
4 4 400
5 5 560
6 6 640
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
7 7 720
8 8 720
9 9 880
10 10 1040
11 11 1120
12 12 1360
13 13 1520
14 14 1600
15 15 1840
16 16 1920
17 17 2160
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
18 18 2320
19 19 2480
20 20 2640
21 21 2800
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
22 22 2960
23 23 3120
24 24 3280
25 25 3360
26 26 3600
27 27 3760
28 28 3920
29 29 4080
30 30 4240
31 31 4400
32 32 4560
33 33 4720
34 34 4880
35 35 5040
36 36 5200
37 37 5360
38 38 5520
39 39 5680
40 40 5840
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900 error __999 Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
(Session info: headless chrome=81.0.4044.138)
make csv file ...
ZEPETO reviews crawling finished!
app_name : VivaVideo - 동영상 편집 어플 & 무비메이커 app_url : com.quvideo.xiaoying
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
5 5 400
6 6 400
7 7 560
8 8 640
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
9 9 720
10 10 720
11 11 800
12 12 960
13 13 1200
14 14 1280
15 15 1520
16 16 1680
17 17 1840
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
18 18 2000
19 19 2160
20 20 2320
21 21 2480
22 22 2640
23 23 2800
24 24 2960
25 25 3120
26 26 3280
27 27 3440
28 28 3600
29 29 3760
30 30 3920
31 31 4080
32 32 4240
33 33 4400
34 34 4560
35 35 4720
36 36 4880
37 37 5040
38 38 5200
39 39 5360
40 40 5520
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900 no. 1000 no. 1100 no. 1200 no. 1300 no. 1400 no. 1500 no. 1600 no. 1700 no. 1800 no. 1900 no. 2000 no. 2100 no. 2200 no. 2300 no. 2400 no. 2500 no. 2600 no. 2700
make csv file ...
VivaVideo - 동영상 편집 어플 & 무비메이커 reviews crawling finished!
app_name : 당연시 - 당신도 연애를 시작할 때, 미팅/소개팅/동네 친구 찾기 app_url : com.koikatsu.android.dokidoki2.kr
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
5 5 400
6 6 400
7 7 480
8 8 640
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
9 9 720
10 10 720
11 11 880
12 12 960
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
13 13 1040
14 14 1040
15 15 1200
16 16 1280
17 17 1440
18 18 1600
19 19 1840
20 20 2000
21 21 2160
22 22 2320
23 23 2480
24 24 2640
25 25 2800
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
26 26 2960
27 27 3120
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
28 28 3280
29 29 3440
30 30 3600
31 31 3760
32 32 3920
33 33 4080
34 34 4240
35 35 4400
36 36 4560
37 37 4720
38 38 4880
39 39 5040
40 40 5200
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900 no. 1000 no. 1100 no. 1200 no. 1300 no. 1400 no. 1500 no. 1600 no. 1700 no. 1800 no. 1900 no. 2000 no. 2100 no. 2200 no. 2300 no. 2400 no. 2500
make csv file ...
당연시 - 당신도 연애를 시작할 때, 미팅/소개팅/동네 친구 찾기 reviews crawling finished!
app_name : 원앙챗(안심결제) app_url : kr.chat.wangtalk
let's looping!
1 1 16
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
2 2 16
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
3 3 16
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
4 4 16
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
5 5 16
get items...
writing ... no. 0
make csv file ...
원앙챗(안심결제) reviews crawling finished!
app_name : 아르고 ARGO - 영상메신저, 영상채팅, 친구찾기 app_url : com.probits.argo
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
5 5 400
6 6 400
7 7 560
8 8 640
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
9 9 720
10 10 720
11 11 880
12 12 1040
13 13 1200
14 14 1280
15 15 1520
16 16 1600
17 17 1840
18 18 1870
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
19 19 1870
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
20 20 1870
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
21 21 1870
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900
make csv file ...
아르고 ARGO - 영상메신저, 영상채팅, 친구찾기 reviews crawling finished!
app_name : 증권플러스 for Kakao - 국민 증권앱 app_url : com.dunamu.stockplus
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
5 5 400
6 6 400
7 7 560
8 8 640
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
9 9 720
10 10 720
11 11 880
12 12 960
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
13 13 1040
14 14 1040
15 15 1200
16 16 1360
17 17 1520
18 18 1680
19 19 1840
20 20 2000
21 21 2160
22 22 2320
23 23 2480
24 24 2640
25 25 2800
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
26 26 2960
27 27 3120
28 28 3280
29 29 3440
30 30 3600
31 31 3760
32 32 3920
33 33 4080
34 34 4240
35 35 4400
36 36 4560
37 37 4720
38 38 4880
39 39 5040
40 40 5200
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900 error __967 Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
(Session info: headless chrome=81.0.4044.138)
make csv file ...
증권플러스 for Kakao - 국민 증권앱 reviews crawling finished!
app_name : 소모임 - No.1 취미모임 앱 app_url : com.friendscube.somoim
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
5 5 400
6 6 400
7 7 560
8 8 640
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
9 9 720
10 10 720
11 11 800
12 12 1040
13 13 1200
14 14 1280
15 15 1440
16 16 1680
17 17 1840
18 18 1920
19 19 2160
20 20 2320
21 21 2480
22 22 2640
23 23 2800
24 24 2960
25 25 3120
26 26 3280
27 27 3360
28 28 3600
29 29 3680
30 30 3920
31 31 4080
32 32 4240
33 33 4320
34 34 4560
35 35 4720
36 36 4880
37 37 5040
38 38 5200
39 39 5360
40 40 5520
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900 error __958 Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
(Session info: headless chrome=81.0.4044.138)
make csv file ...
소모임 - No.1 취미모임 앱 reviews crawling finished!
app_name : 아이쉐어링 위치추적 - 가족 자녀안심, 친구찾기, GPS 안심이 app_url : com.isharing.isharing
let's looping!
1 1 80
2 2 160
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
3 3 240
4 4 320
5 5 400
6 6 400
7 7 560
8 8 640
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
9 9 720
10 10 720
11 11 880
12 12 960
13 13 1200
14 14 1280
15 15 1520
16 16 1680
17 17 1840
18 18 2000
19 19 2160
20 20 2320
21 21 2480
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
22 22 2640
23 23 2800
24 24 2960
25 25 3120
26 26 3280
27 27 3440
28 28 3600
29 29 3760
30 30 3920
31 31 4000
32 32 4240
33 33 4400
34 34 4560
35 35 4720
36 36 4880
37 37 4960
38 38 5200
39 39 5360
40 40 5520
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 error __484 Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
(Session info: headless chrome=81.0.4044.138)
make csv file ...
아이쉐어링 위치추적 - 가족 자녀안심, 친구찾기, GPS 안심이 reviews crawling finished!
app_name : 포스텔러 - 애정운, 신년운세, 무료운세, 사주, 타로, 궁합, 토정비결, 별자리, 손금 app_url : com.un7qi3.forceteller
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
5 5 400
6 6 400
7 7 480
8 8 640
9 9 880
10 10 960
11 11 1200
12 12 1280
13 13 1520
14 14 1680
15 15 1840
16 16 2000
17 17 2080
18 18 2320
19 19 2480
20 20 2640
21 21 2800
22 22 2960
23 23 3120
24 24 3280
25 25 3440
26 26 3600
27 27 3760
28 28 3920
29 29 4080
30 30 4240
31 31 4400
32 32 4560
33 33 4720
34 34 4880
35 35 5040
36 36 5200
37 37 5360
38 38 5520
39 39 5680
40 40 5840
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 error __795 Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
(Session info: headless chrome=81.0.4044.138)
make csv file ...
포스텔러 - 애정운, 신년운세, 무료운세, 사주, 타로, 궁합, 토정비결, 별자리, 손금 reviews crawling finished!
app_name : 썸데이 - 이상형 만남 소개팅 (만남, 데이트, 소개팅 어플) app_url : com.abletree.someday
let's looping!
1 1 80
2 2 160
3 3 240
4 4 240
5 5 240
6 6 320
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
7 7 400
8 8 400
9 9 560
10 10 640
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
11 11 720
12 12 720
13 13 800
14 14 960
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
15 15 1040
16 16 1040
17 17 1120
18 18 1280
19 19 1520
20 20 1680
21 21 1840
22 22 1920
23 23 2160
24 24 2320
25 25 2480
26 26 2640
27 27 2800
28 28 2960
29 29 3120
30 30 3280
31 31 3440
32 32 3600
33 33 3760
34 34 3920
35 35 4080
36 36 4240
37 37 4400
38 38 4560
39 39 4720
40 40 4880
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900 error __925 Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
(Session info: headless chrome=81.0.4044.138)
make csv file ...
썸데이 - 이상형 만남 소개팅 (만남, 데이트, 소개팅 어플) reviews crawling finished!
app_name : 만남,돌싱,중년,채팅어플,아줌마중매,무료채팅,소개팅,재혼만남,만남어플 - 불팅 app_url : fire.ting.fireting.chat
let's looping!
1 1 80
2 2 160
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
3 3 160
4 4 240
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
5 5 240
6 6 320
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
7 7 320
8 8 400
9 9 480
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
10 10 480
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
11 11 640
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
12 12 720
13 13 800
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
14 14 880
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
15 15 960
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
16 16 1040
17 17 1200
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
18 18 1280
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
19 19 1360
20 20 1360
21 21 1520
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
22 22 1600
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
23 23 1680
24 24 1680
25 25 1840
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
26 26 1920
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
27 27 2000
28 28 2000
29 29 2080
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
30 30 2240
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
31 31 2320
32 32 2320
33 33 2480
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
34 34 2560
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
35 35 2640
36 36 2640
37 37 2800
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
38 38 2880
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
39 39 2960
40 40 2960
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900 no. 1000 no. 1100 no. 1200 no. 1300 no. 1400
make csv file ...
만남,돌싱,중년,채팅어플,아줌마중매,무료채팅,소개팅,재혼만남,만남어플 - 불팅 reviews crawling finished!
app_name : IZ*ONE Private Mail app_url : com.ca_smart.izonemail
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
5 5 400
6 6 400
7 7 480
8 8 560
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
9 9 720
10 10 720
11 11 880
12 12 1040
13 13 1070
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
14 14 1070
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
15 15 1070
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div"}
(Session info: headless chrome=81.0.4044.138)
16 16 1070
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500
make csv file ...
IZ*ONE Private Mail reviews crawling finished!
app_name : 키네마스터 - 동영상 편집, 동영상 자막, 필터, vlog 편집 앱 app_url : com.nexstreaming.app.kinemasterfree
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
5 5 400
6 6 400
7 7 480
8 8 560
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
9 9 720
10 10 720
11 11 800
12 12 1040
13 13 1120
14 14 1280
15 15 1520
16 16 1600
17 17 1760
18 18 1920
19 19 2160
20 20 2320
21 21 2480
22 22 2640
23 23 2800
24 24 2960
25 25 3120
26 26 3280
27 27 3440
28 28 3600
29 29 3760
30 30 3920
31 31 4080
32 32 4240
33 33 4400
34 34 4560
35 35 4720
36 36 4880
37 37 5040
38 38 5200
39 39 5360
40 40 5520
get items...
writing ... no. 0 no. 100 no. 200 no. 300
apps_list.index( ['여보야 - 결혼, 재혼을 위한 중매쟁이 앱', 'com.yeoboya.com.yeoboya'] )
import pandas as pd
df_reviews_list = pd.DataFrame(reviews_list, columns=['app_name', 'app_reviews'])
df_reviews_list.set_index('app_name', inplace=True)
display(df_reviews_list)
df_reviews_list.iloc[6]
reviews_list
import sys
!apt-get update
!apt install chromium-chromedriver
!cp /usr/lib/chromium-browser/chromedriver /usr/bin
!pip install selenium
# set options to be headless, ..
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver import ActionChains
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
# from bs4 import BeautifulSoup as bs
import re
import time
import pandas as pd
def review_crawler (app_name, app_url) :
print ("app_name : ", app_name, end='\t')
print ("app_url : ", app_url)
with webdriver.Chrome('chromedriver', options=options) as wd :
full_url = 'https://play.google.com/store/apps/details?hl=ko&showAllReviews=true&id=' + app_url
wd.get(full_url)
wd.implicitly_wait(2)
review_list = []
print (f"let's looping!")
page_length_list = []
for _ in range(40) :
print (f"{_+1}", end=' ')
wd.execute_script('window.scrollTo(0, document.body.scrollHeight);')
time.sleep(0.5)
elements = wd.find_elements_by_class_name('zc7KVe')
page_length = len(elements)
page_length_list.append(page_length )
print (len(page_length_list), page_length)
if len(page_length_list) > 4 and page_length_list[-1] == page_length_list[-4] : break
try :
more_btn = wd.find_element_by_xpath('/html/body/div[1]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div[2]/div')
if _ % 2 == 1 :
ActionChains(wd).click(more_btn).perform()
except Exception as e :
print(e)
continue
print ("\nget items... ")
print (f"writing ...", end=' ')
try :
for idx, one in enumerate(elements) :
if idx % 200 == 0 :
print (f"no. {idx//2:4d}", end=' ')
if idx % 2 == 0 :
star = one.find_element_by_class_name('pf5lIe').find_element_by_tag_name('div').get_attribute('aria-label')[10]
review = one.find_element_by_class_name('UD7Dzf').text
review_list.append([int(idx/2), star, review])
except Exception as e :
print (f"error __{int(idx/2)} {e }")
review_list.append([int(idx/2), "error", "error",])
print ('\nmake csv file ...')
pd.DataFrame(review_list, columns=['idx', 'star', 'review']).set_index('idx').to_csv(f"/content/drive/My Drive/Programming/1. Docker/VSCode_/Java_Spring/Python/MLDL/ToyProject/{app_name[:5]}.csv")
print (f"\n{app_name} reviews crawling finished!\n")
return None
# return [app_name, review_list]
# print(review_crawler(apps_list[30][0], apps_list[30][1]))
apps_list = [ ['V LIVE', 'com.naver.vapp'] ]
# apps_list = get_top50()
# reviews_list = [['app_name', 'app_reviews']]
for app in apps_list :
review_crawler(app[0], app[1])
Hit:1 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic InRelease
Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Hit:3 http://archive.ubuntu.com/ubuntu bionic InRelease
Hit:4 http://ppa.launchpad.net/marutter/c2d4u3.5/ubuntu bionic InRelease
Hit:5 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/ InRelease
Get:6 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Ign:8 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease
Ign:9 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 InRelease
Hit:10 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 Release
Hit:11 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 Release
Fetched 252 kB in 2s (159 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
chromium-chromedriver is already the newest version (81.0.4044.138-0ubuntu0.18.04.1).
0 upgraded, 0 newly installed, 0 to remove and 60 not upgraded.
cp: '/usr/lib/chromium-browser/chromedriver' and '/usr/bin/chromedriver' are the same file
Requirement already satisfied: selenium in /usr/local/lib/python3.6/dist-packages (3.141.0)
Requirement already satisfied: urllib3 in /usr/local/lib/python3.6/dist-packages (from selenium) (1.24.3)
app_name : V LIVE app_url : com.naver.vapp
let's looping!
1 1 80
2 2 160
3 3 240
4 4 320
5 5 400
6 6 400
7 7 560
8 8 720
9 9 880
10 10 1040
11 11 1200
12 12 1280
Message: stale element reference: element is not attached to the page document
(Session info: headless chrome=81.0.4044.138)
13 13 1360
14 14 1360
15 15 1520
16 16 1680
17 17 1840
18 18 2000
19 19 2160
20 20 2320
21 21 2480
22 22 2640
23 23 2800
24 24 2960
25 25 3120
26 26 3280
27 27 3440
28 28 3600
29 29 3760
30 30 3920
31 31 4080
32 32 4240
33 33 4400
34 34 4560
35 35 4720
36 36 4880
37 37 5040
38 38 5200
39 39 5360
40 40 5520
get items...
writing ... no. 0 no. 100 no. 200 no. 300 no. 400 no. 500 no. 600 no. 700 no. 800 no. 900 no. 1000 no. 1100 no. 1200 no. 1300 no. 1400 no. 1500 no. 1600 error __1627 Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
(Session info: headless chrome=81.0.4044.138)
make csv file ...
V LIVE reviews crawling finished!