Libraries Used

Pretrained models were used for sentiment prediction

!pip install transformers

Collecting transformers
  Downloading transformers-4.29.2-py3-none-any.whl (7.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.1/7.1 MB 52.9 MB/s eta 0:00:00
Requirement already satisfied: pyyaml>=5.1 in /root/venv/lib/python3.9/site-packages (from transformers) (6.0)
Requirement already satisfied: requests in /shared-libs/python3.9/py/lib/python3.9/site-packages (from transformers) (2.28.1)
Collecting huggingface-hub<1.0,>=0.14.1
  Downloading huggingface_hub-0.15.1-py3-none-any.whl (236 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 236.8/236.8 KB 29.0 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.17 in /shared-libs/python3.9/py/lib/python3.9/site-packages (from transformers) (1.23.4)
Requirement already satisfied: tqdm>=4.27 in /shared-libs/python3.9/py/lib/python3.9/site-packages (from transformers) (4.64.1)
Collecting tokenizers!=0.11.3,<0.14,>=0.11.1
  Downloading tokenizers-0.13.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 61.8 MB/s eta 0:00:00
Requirement already satisfied: regex!=2019.12.17 in /shared-libs/python3.9/py/lib/python3.9/site-packages (from transformers) (2022.9.13)
Requirement already satisfied: packaging>=20.0 in /shared-libs/python3.9/py-core/lib/python3.9/site-packages (from transformers) (21.3)
Requirement already satisfied: filelock in /shared-libs/python3.9/py/lib/python3.9/site-packages (from transformers) (3.8.0)
Collecting fsspec
  Downloading fsspec-2023.5.0-py3-none-any.whl (160 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 160.1/160.1 KB 9.8 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions>=3.7.4.3 in /shared-libs/python3.9/py/lib/python3.9/site-packages (from huggingface-hub<1.0,>=0.14.1->transformers) (4.4.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /shared-libs/python3.9/py-core/lib/python3.9/site-packages (from packaging>=20.0->transformers) (3.0.9)
Requirement already satisfied: charset-normalizer<3,>=2 in /shared-libs/python3.9/py-core/lib/python3.9/site-packages (from requests->transformers) (2.1.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /shared-libs/python3.9/py/lib/python3.9/site-packages (from requests->transformers) (1.26.12)
Requirement already satisfied: certifi>=2017.4.17 in /shared-libs/python3.9/py/lib/python3.9/site-packages (from requests->transformers) (2022.9.24)
Requirement already satisfied: idna<4,>=2.5 in /shared-libs/python3.9/py-core/lib/python3.9/site-packages (from requests->transformers) (3.4)
Installing collected packages: tokenizers, fsspec, huggingface-hub, transformers
Successfully installed fsspec-2023.5.0 huggingface-hub-0.15.1 tokenizers-0.13.3 transformers-4.29.2
WARNING: You are using pip version 22.0.4; however, version 23.1.2 is available.
You should consider upgrading via the '/root/venv/bin/python -m pip install --upgrade pip' command.

from transformers import AutoModelForSequenceClassification from transformers import TFAutoModelForSequenceClassification from transformers import AutoTokenizer import matplotlib.pyplot as plt import pandas as pd import numpy as np from scipy.special import softmax import nltk from nltk.corpus import stopwords from nltk import FreqDist nltk.download('stopwords')

huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
[nltk_data] Downloading package stopwords to /root/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!

Preparing Tokenizer and Model

tokenizer = AutoTokenizer.from_pretrained("cardiffnlp/twitter-roberta-base-sentiment") model = AutoModelForSequenceClassification.from_pretrained("cardiffnlp/twitter-roberta-base-sentiment") model.save_pretrained("cardiffnlp/twitter-roberta-base-sentiment")

Downloading (…)lve/main/config.json: 100%|██████████| 747/747 [00:00<00:00, 23.0kB/s]
Downloading (…)olve/main/vocab.json: 100%|██████████| 899k/899k [00:00<00:00, 15.3MB/s]
Downloading (…)olve/main/merges.txt: 100%|██████████| 456k/456k [00:00<00:00, 94.4MB/s]
Downloading (…)cial_tokens_map.json: 100%|██████████| 150/150 [00:00<00:00, 23.7kB/s]
Downloading pytorch_model.bin: 100%|██████████| 499M/499M [00:02<00:00, 188MB/s]

airasia_tweets = pd.read_csv("cleaned_airasia_tweets_1.csv")

airasia_tweets

Unnamed: 0int64

0 - 1420

Textobject

nak share total expense i day trip to singapore last saturday mind you trip ni sangat last minute dan sangat tight budget lolbus kl sentral to klia rmflight kulsin airasia rm no meal no seat selection no luggagebus sinkul aeroline rm seat meal0.1%

happy flight0.1%

1199 others99.8%

190

199

312

335

happy flight

559

607

i dont know abt others but for cheap flight id still choose myairline over airasiaso far takde masalah fly dgn anak orang wout daddy dorang pun crew friendly flight on time murahanyways thats a lot of luggage to handcarry dgn anak lagi ku tengok pun stress

1094

1302

update g delegate accompanied by india tourism minister reached srinagar airport from delhi by chartered flight of at am received by joint secretary g bhavna saxena jks adg security divisional commissioner

871

981

the thing is it their choice to pick airasia ke dhl ke apake it the service that were given that matter dont blame them for choosing airasia it their decision mustahil kot if they didnt pay benda dah bayar so tanggungjawap company tu untuk jaga barang orang

639

697

life can be unpredictable but thats what make it exciting again and enjoy gb data for all site gb data for your choice of apps valid for day youll even get adventure voucher of your choice from either grab airasia klook lazada or pickaroo

414

440

airport wise international traffic april is maintaining it th position and of market share expecting more traffic soon

405

431

the best convocation mai with abang rider airasia

1162

1381

dreamy we hope to fly here again soon and bring the beauty of to the world

128

134

flight schedule for

Sentiment Analysis

Tokenize sentence and predicts sentiment

labels = [] for i,v in enumerate(airasia_tweets['Text']): encoded_input = tokenizer(v, return_tensors='pt') output = model(**encoded_input) scores = output[0][0].detach().numpy() scores = softmax(scores) ranking = np.argsort(scores) ranking = ranking[::-1] if ranking[0] == 0: labels.append("Negative") elif ranking[0] == 1: labels.append("Neutral") elif ranking[0] == 2: labels.append("Positive")

labels

Assign the sentiment labels to the sentence

airasia_tweets['Sentiment'] = labels

airasia_tweets.head()

Unnamed: 0int64

Textobject

saf produced in india help power airasia flight

hello refund tic yg beli dkt ur website from jogja to jakarta still not respondingthen simply said case closed flight hell not available for month

my overseas flight in cancelled by airasia due to covid already year now and still no cash refund despite numerous email to aax n mavcom with passenger forced to accept only useless travel voucher without any choice or option pay our full cash refund now

hi i need to cancel a ticket and get a refund but i cant do it on you app or web page either do you have a real person a a customer service instead of a chatbot please an email address would also help my flight will depart from hong kongthank you

okay lang ang air asia for me chsr singit actually naka try ako ng airasia just this month bc my cebpac flight wa cancelled for no apparent reason tapos i have to fly to mnl on that day dapatmaganda rin siguro experience ko kasi first flight ako hehe

print(airasia_tweets.columns)

Index(['Unnamed: 0', 'Text', 'Retweet-Count', 'Sentiment'], dtype='object')

Exporting Tweets with Sentiment

airasia_tweets.to_csv("airasia_tweets_sentiment_1.csv")

Visualizing Tweets Sentiment

Count plot by sentiment

filtered_negative_sentiments = airasia_tweets[airasia_tweets['Sentiment'] == "Negative"].head()

Top 5 negative tweets

filtered_negative_sentiments

Unnamed: 0int64

Textobject

hello refund tic yg beli dkt ur website from jogja to jakarta still not respondingthen simply said case closed flight hell not available for month

chatting with bo is useless because they are automated robot

hi i sent a dm regarding online checkin but havent received a reply