Merging all Articles
I scheduled this notebook for 3 times a week. Then merged every dataframe into one big dataset called 'combined.csv'. Before merging datasets, each data is cleaned in their individual notebooks. All dates were converted into datetime. Missing descriptions were replaced with 'No Description', nans in the category columns were filled. After merging, the dataframe is also cleaned to drop duplicates, drop excess columns that were created when merged.
16869
248.0
CABINET DECIDES TO BAN RAW COAL CONSUMPTION IN 2019
16870
249.0
TRQ: “OYU TOLGOI” COPPER PRODUCTION AND GOLD PRODUCTION WILL INCREASE BY 340 PERCENT AND 150 PERCENT RESPECTIVELY IN 2018-2025
16871
250.0
FOREIGN MINISTER HIGHLIGHTS ECONOMIC RECOVERY AT BCM MONTHLY MEETING
16872
251.0
JEROME POWELL: THE FEDERAL RESERVE RAISING U.S. INTEREST RATES FOUR TIMES THIS YEAR
19185
251.0
JEROME POWELL: THE FEDERAL RESERVE RAISING U.S. INTEREST RATES FOUR TIMES THIS YEAR
15587
251.0
JEROME POWELL: THE FEDERAL RESERVE RAISING U.S. INTEREST RATES FOUR TIMES THIS YEAR
19184
250.0
FOREIGN MINISTER HIGHLIGHTS ECONOMIC RECOVERY AT BCM MONTHLY MEETING
15586
250.0
FOREIGN MINISTER HIGHLIGHTS ECONOMIC RECOVERY AT BCM MONTHLY MEETING
19183
249.0
TRQ: “OYU TOLGOI” COPPER PRODUCTION AND GOLD PRODUCTION WILL INCREASE BY 340 PERCENT AND 150 PERCENT RESPECTIVELY IN 2018-2025
19182
248.0
CABINET DECIDES TO BAN RAW COAL CONSUMPTION IN 2019
297
nan
iDAX Exchange receives Virtual Asset Service Provider license.
298
nan
Michidmaa Budsuren named as CEO of MEC partners (ArdShop.mn)
299
nan
Odbayar O. named as Chairman of Ard Insurance JSC
300
nan
Ganzorig Ch. appointed as Chairman of Ard Credit NBFI JSC
301
nan
The 3rd Alumni Economic Forum Opens in Arxan in Inner Mongolia - PR Newswire
302
nan
The 3rd Alumni Economic Forum Opens in Arxan in Inner Mongolia - PR Newswire APAC
303
nan
Western Inner Mongolia Series | Episode 9: Demoiselle hermit - CGTN
304
nan
The £2.2 billion offer from Rio Tinto for the copper site in Mongolia was rejected.
305
nan
Rio 'disappointed' in Turquoise Hill decision - Mining Weekly
306
nan
Turquoise Hill shares fall after Quebec miner rejects Rio Tinto's $2.7-billion ... - Yahoo Finance
/tmp/ipykernel_73/2806525063.py:1: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
combined['Category'][102:137] = combined['Category'][102:137].fillna('International')
1960
P.Orkhon triumphs at Asian Wrestling Championships
2018-03-29 00:00:00
1961
Lessons from UBIFF-2022 should stick around
2022-10-30 00:00:00
1962
Z.Tumenjargal and B.Bayartsengel collaborate on new exhibition
2022-10-26 00:00:00
1963
‘B-Atelier-209’ art show unveiled
2022-10-14 00:00:00
1964
‘Art has become something only for a certain group of people’
2022-10-07 00:00:00
1965
Students document social issues in photos
2022-09-17 00:00:00
1966
Viewing rural life from children’s perspective
2022-06-03 00:00:00
1967
‘Fantastic Creatures’ art show on display
2022-05-20 00:00:00
1968
Traverse nature at ‘Anir’ art show
2022-05-07 00:00:00
1969
Postmodern art show highlights social issues
2022-04-29 00:00:00
Find Old/New Rows
When I though I had a good amount of articles to train (804 articles) I manually inserted 1's an 0's into a target column of that dataset which shows which articles I would want to send out to people and which ones I wouldn't and named that dataset 'combined_csv'.
The dataset 'combined_csv' will become my old dataset, and then the newly merged dataset would just be called 'combined'. Then I loop through the files and get the recent file which would contain old and new articles. Then I would get the length of the old dataset and slice the newly combined dataset to get a dataframe called 'new_articles'. Then I save the new_article to a csv, concat the dataframe to the old dataset for when the next new articles merge it would become the old dataset.
0
THE HU PLANS TO HOST A VIRTUAL CONCERT IN THE METAVERSE IN PARTNERSHIP WITH ARD
2022-07-01
1
INVESTOR NATION – MONGOLIA’S FIRST OPEN-END MUTUAL FUND WAS ESTABLISHED AND IS SET TO MAKE AN INITIAL PUBLIC OFFERING
2022-05-16
2
BULGAN UNDRAGA JSC CHANGES NAME TO CRYPTO NATION JSC
2022-03-18
3
ARD FINANCIAL GROUP COMMITS TO WORKING WITH SMEs
2022-02-24
4
ARD FINANCIAL GROUP TO ALLOCATE DIVIDENDS TO SHAREHOLDERS
2022-02-16
5
Ard Financial Group highlighted as Technology Investor Entrepreneur of the Year
2021-12-29
6
MONGOLIA ADOPTS LAW ON VIRTUAL ASSET SERVICE PROVIDERS
2021-12-16
7
ARD STARTS PARTNERSHIP WITH ZEUS ESPORTS TEAM
2021-07-08
8
MONGOLIA’S FIRST PUBLIC MEDIA GROUP
2021-07-02
9
Trade between Mongolia and China reached USD 10 billion in 2021
2022-06-22
combined.csv
0
iDAX Exchange receives Virtual Asset Service Provider license.
2022-09-23
1
Michidmaa Budsuren named as CEO of MEC partners (ArdShop.mn)
2022-08-17
2
Odbayar O. named as Chairman of Ard Insurance JSC
2022-07-22
3
Ganzorig Ch. appointed as Chairman of Ard Credit NBFI JSC
2022-06-20
4
The 3rd Alumni Economic Forum Opens in Arxan in Inner Mongolia - PR Newswire
2022-08-16
5
The 3rd Alumni Economic Forum Opens in Arxan in Inner Mongolia - PR Newswire APAC
2022-08-16
6
Western Inner Mongolia Series | Episode 9: Demoiselle hermit - CGTN
2022-08-16
7
The £2.2 billion offer from Rio Tinto for the copper site in Mongolia was rejected.
2022-08-16
8
Rio 'disappointed' in Turquoise Hill decision - Mining Weekly
2022-08-16
9
Turquoise Hill shares fall after Quebec miner rejects Rio Tinto's $2.7-billion ... - Yahoo Finance
2022-08-16
0
THE HU PLANS TO HOST A VIRTUAL CONCERT IN THE METAVERSE IN PARTNERSHIP WITH ARD
2022-07-01
1
INVESTOR NATION – MONGOLIA’S FIRST OPEN-END MUTUAL FUND WAS ESTABLISHED AND IS SET TO MAKE AN INITIAL PUBLIC OFFERING
2022-05-16
2
BULGAN UNDRAGA JSC CHANGES NAME TO CRYPTO NATION JSC
2022-03-18
3
ARD FINANCIAL GROUP COMMITS TO WORKING WITH SMEs
2022-02-24
4
ARD FINANCIAL GROUP TO ALLOCATE DIVIDENDS TO SHAREHOLDERS
2022-02-16
5
Ard Financial Group highlighted as Technology Investor Entrepreneur of the Year
2021-12-29
6
MONGOLIA ADOPTS LAW ON VIRTUAL ASSET SERVICE PROVIDERS
2021-12-16
7
ARD STARTS PARTNERSHIP WITH ZEUS ESPORTS TEAM
2021-07-08
8
MONGOLIA’S FIRST PUBLIC MEDIA GROUP
2021-07-02
9
Trade between Mongolia and China reached USD 10 billion in 2021
2022-06-22
0
Prime Minister of Mongolia Oyun-Erdene Wants to Get the Country Out of Pandemic Slowdown
2022-04-29
1
Backlash over Development Bank of Mongolia Exposes Social and Political Tensions
2022-04-27
2
What You Need To Know About Mongolia and What's On the Horizon in H2 2022
2022-05-25
3
Mongolia Starts Austerity Measures
2022-05-09
4
Mongolia Cuts Red Tape By Breaking Up GASI
2022-10-30
5
Cracks Become More Visible Within the Ruling Party
2022-10-24
6
Government Offers Tax Breaks For Tech Startups & Financial Sector
2022-10-21
7
Germany Gets Interested In Mongolia’s Critical Minerals
2022-10-17
8
A Road Connecting China and Europe Probed For Corruption
2022-10-17
9
Chinese Steelmakers Turn Away From Coking Coal Risking Mongolia’s Key Earning
2022-10-03