Lab 4
#environment
#!pip install yfinance --upgrade --no-cache-dir
#!pip install PyPortfolioOpt==1.2.1
Collecting yfinance
Downloading yfinance-0.1.63.tar.gz (26 kB)
Requirement already satisfied: pandas>=0.24 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from yfinance) (1.2.5)
Requirement already satisfied: numpy>=1.15 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from yfinance) (1.19.5)
Requirement already satisfied: requests>=2.20 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from yfinance) (2.26.0)
Collecting multitasking>=0.0.7
Downloading multitasking-0.0.9.tar.gz (8.1 kB)
Requirement already satisfied: lxml>=4.5.1 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from yfinance) (4.6.3)
Requirement already satisfied: pytz>=2017.3 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from pandas>=0.24->yfinance) (2021.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from pandas>=0.24->yfinance) (2.8.2)
Requirement already satisfied: six>=1.5 in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from python-dateutil>=2.7.3->pandas>=0.24->yfinance) (1.16.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from requests>=2.20->yfinance) (1.26.6)
Requirement already satisfied: idna<4,>=2.5 in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from requests>=2.20->yfinance) (3.2)
Requirement already satisfied: charset-normalizer~=2.0.0 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from requests>=2.20->yfinance) (2.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from requests>=2.20->yfinance) (2021.5.30)
Building wheels for collected packages: yfinance, multitasking
Building wheel for yfinance (setup.py) ... done
Created wheel for yfinance: filename=yfinance-0.1.63-py2.py3-none-any.whl size=23918 sha256=3a3f79718a1f873fd629011f69f42db9512a31fe5885e936b95a958dbed1ad2d
Stored in directory: /tmp/pip-ephem-wheel-cache-kww3a6ly/wheels/fe/87/8b/7ec24486e001d3926537f5f7801f57a74d181be25b11157983
Building wheel for multitasking (setup.py) ... done
Created wheel for multitasking: filename=multitasking-0.0.9-py3-none-any.whl size=8375 sha256=cadc46f7a41c6c587ea56f43f4e41da2bc4d0b5b3091cd0925a8082df2456a5d
Stored in directory: /tmp/pip-ephem-wheel-cache-kww3a6ly/wheels/ae/25/47/4d68431a7ec1b6c4b5233365934b74c1d4e665bf5f968d363a
Successfully built yfinance multitasking
Installing collected packages: multitasking, yfinance
Successfully installed multitasking-0.0.9 yfinance-0.1.63
WARNING: You are using pip version 21.2.3; however, version 21.2.4 is available.
You should consider upgrading via the '/root/venv/bin/python -m pip install --upgrade pip' command.
Collecting PyPortfolioOpt==1.2.1
Downloading PyPortfolioOpt-1.2.1-py3-none-any.whl (49 kB)
|████████████████████████████████| 49 kB 7.6 MB/s
Collecting pandas<0.26.0,>=0.25.3
Downloading pandas-0.25.3-cp37-cp37m-manylinux1_x86_64.whl (10.4 MB)
|████████████████████████████████| 10.4 MB 33.8 MB/s
Requirement already satisfied: numpy in /shared-libs/python3.7/py/lib/python3.7/site-packages (from PyPortfolioOpt==1.2.1) (1.19.5)
Requirement already satisfied: scipy in /shared-libs/python3.7/py/lib/python3.7/site-packages (from PyPortfolioOpt==1.2.1) (1.7.1)
Collecting cvxpy<2.0.0,>=1.0.28
Downloading cvxpy-1.1.15-cp37-cp37m-manylinux_2_24_x86_64.whl (2.7 MB)
|████████████████████████████████| 2.7 MB 41.1 MB/s
Collecting ecos>=2
Downloading ecos-2.0.7.post1-cp37-cp37m-manylinux1_x86_64.whl (147 kB)
|████████████████████████████████| 147 kB 49.9 MB/s
Collecting osqp>=0.4.1
Downloading osqp-0.6.2.post0-cp37-cp37m-manylinux2014_x86_64.whl (212 kB)
|████████████████████████████████| 212 kB 49.3 MB/s
Collecting scs>=1.1.6
Downloading scs-2.1.4.tar.gz (6.6 MB)
|████████████████████████████████| 6.6 MB 19.3 MB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting qdldl
Downloading qdldl-0.1.5.post0-cp37-cp37m-manylinux2014_x86_64.whl (941 kB)
|████████████████████████████████| 941 kB 45.7 MB/s
Requirement already satisfied: python-dateutil>=2.6.1 in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from pandas<0.26.0,>=0.25.3->PyPortfolioOpt==1.2.1) (2.8.2)
Requirement already satisfied: pytz>=2017.2 in /shared-libs/python3.7/py/lib/python3.7/site-packages (from pandas<0.26.0,>=0.25.3->PyPortfolioOpt==1.2.1) (2021.1)
Requirement already satisfied: six>=1.5 in /shared-libs/python3.7/py-core/lib/python3.7/site-packages (from python-dateutil>=2.6.1->pandas<0.26.0,>=0.25.3->PyPortfolioOpt==1.2.1) (1.16.0)
Building wheels for collected packages: scs
Building wheel for scs (PEP 517) ... done
Created wheel for scs: filename=scs-2.1.4-cp37-cp37m-linux_x86_64.whl size=113462 sha256=f922e8264ac80673e96c2352518dc4a0b2509a5f01f591d7eb356c4d0ab55180
Stored in directory: /root/.cache/pip/wheels/d6/58/0b/a55ff6d4e2da4c9776e363a778ea13e4068037a7be82eb85e0
Successfully built scs
Installing collected packages: qdldl, scs, osqp, ecos, pandas, cvxpy, PyPortfolioOpt
Attempting uninstall: pandas
Found existing installation: pandas 1.2.5
Not uninstalling pandas at /shared-libs/python3.7/py/lib/python3.7/site-packages, outside environment /root/venv
Can't uninstall 'pandas'. No files were found to uninstall.
Successfully installed PyPortfolioOpt-1.2.1 cvxpy-1.1.15 ecos-2.0.7.post1 osqp-0.6.2.post0 pandas-0.25.3 qdldl-0.1.5.post0 scs-2.1.4
WARNING: You are using pip version 21.2.3; however, version 21.2.4 is available.
You should consider upgrading via the '/root/venv/bin/python -m pip install --upgrade pip' command.
# Import modules
import requests
import io
import pandas as pd
import requests
import json
from datetime import datetime
import numpy as np
import yfinance as yf
import seaborn as sns
import plotly.express as px
import plotly.graph_objs as go
import matplotlib.pyplot as plt
import scipy.stats as st
from pypfopt.efficient_frontier import EfficientFrontier
from pypfopt import risk_models
from pypfopt import expected_returns
from pypfopt import cla
from pypfopt.plotting import Plotting
from pypfopt.discrete_allocation import DiscreteAllocation, get_latest_prices
from matplotlib.ticker import FuncFormatter
## JHU Vaccination Rates (Taken From: https://github.com/owid/covid-19-data/tree/master/public/data)
url = 'https://raw.githubusercontent.com/owid/covid-19-data/master/public/data/owid-covid-data.csv'
download = requests.get(url).content
covid = pd.read_csv(io.StringIO(download.decode('utf-8')), parse_dates=['date'])
covid.tail()
sns.scatterplot('date', 'new_cases', data=covid, hue='continent')
/shared-libs/python3.7/py/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
FutureWarning
us = covid[covid['location']=='United States']
us_covid = us.set_index('date')
us_covid.info()
us_covid.head()
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 604 entries, 2020-01-22 to 2021-09-16
Data columns (total 61 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 iso_code 604 non-null object
1 continent 604 non-null object
2 location 604 non-null object
3 total_cases 604 non-null float64
4 new_cases 603 non-null float64
5 new_cases_smoothed 598 non-null float64
6 total_deaths 566 non-null float64
7 new_deaths 566 non-null float64
8 new_deaths_smoothed 598 non-null float64
9 total_cases_per_million 604 non-null float64
10 new_cases_per_million 603 non-null float64
11 new_cases_smoothed_per_million 598 non-null float64
12 total_deaths_per_million 566 non-null float64
13 new_deaths_per_million 566 non-null float64
14 new_deaths_smoothed_per_million 598 non-null float64
15 reproduction_rate 559 non-null float64
16 icu_patients 428 non-null float64
17 icu_patients_per_million 428 non-null float64
18 hosp_patients 428 non-null float64
19 hosp_patients_per_million 428 non-null float64
20 weekly_icu_admissions 0 non-null float64
21 weekly_icu_admissions_per_million 0 non-null float64
22 weekly_hosp_admissions 60 non-null float64
23 weekly_hosp_admissions_per_million 60 non-null float64
24 new_tests 559 non-null float64
25 total_tests 559 non-null float64
26 total_tests_per_thousand 559 non-null float64
27 new_tests_per_thousand 559 non-null float64
28 new_tests_smoothed 552 non-null float64
29 new_tests_smoothed_per_thousand 552 non-null float64
30 positive_rate 552 non-null float64
31 tests_per_case 552 non-null float64
32 tests_units 559 non-null object
33 total_vaccinations 252 non-null float64
34 people_vaccinated 251 non-null float64
35 people_fully_vaccinated 237 non-null float64
36 total_boosters 0 non-null float64
37 new_vaccinations 239 non-null float64
38 new_vaccinations_smoothed 269 non-null float64
39 total_vaccinations_per_hundred 252 non-null float64
40 people_vaccinated_per_hundred 251 non-null float64
41 people_fully_vaccinated_per_hundred 237 non-null float64
42 total_boosters_per_hundred 0 non-null float64
43 new_vaccinations_smoothed_per_million 269 non-null float64
44 stringency_index 601 non-null float64
45 population 604 non-null float64
46 population_density 604 non-null float64
47 median_age 604 non-null float64
48 aged_65_older 604 non-null float64
49 aged_70_older 604 non-null float64
50 gdp_per_capita 604 non-null float64
51 extreme_poverty 604 non-null float64
52 cardiovasc_death_rate 604 non-null float64
53 diabetes_prevalence 604 non-null float64
54 female_smokers 604 non-null float64
55 male_smokers 604 non-null float64
56 handwashing_facilities 0 non-null float64
57 hospital_beds_per_thousand 604 non-null float64
58 life_expectancy 604 non-null float64
59 human_development_index 604 non-null float64
60 excess_mortality 78 non-null float64
dtypes: float64(57), object(4)
memory usage: 292.6+ KB
df1 = pd.DataFrame()
df1['new cases'] = us_covid['new_cases_smoothed']
df1['new deaths'] = us_covid['new_deaths_smoothed']
df1.dropna()
df1.plot(secondary_y = ["new deaths"], title = 'New Cases and Deaths')
df2 = pd.DataFrame()
df2['positive rate'] = us_covid['positive_rate']
df2['fatality rate'] = us_covid['total_deaths']/us_covid['total_cases']
df2.dropna()
df2.plot(secondary_y = ["fatality rate"], title = 'Positive Rate and Fatality Rate')
df3 = pd.DataFrame()
df3['icu rate'] = us_covid['icu_patients']/us_covid['total_cases']
df3['hosp rate'] = us_covid['hosp_patients']/us_covid['total_cases']
df3['fatality rate'] = df2['fatality rate']
df3.dropna()
df3.plot(secondary_y = ["fatality rate"], title = 'Hospital Rate and Fatality Rate')
df4 = pd.DataFrame()
df4['vaccination rate'] = us_covid['people_fully_vaccinated']/us_covid['population']
df4['fatality rate'] = df2['fatality rate']
df4.dropna()
df4.plot(secondary_y = ["fatality rate"], title = 'Vaccination Rate and Fatality Rate')
Lab 5
assets_list = ['VV','VB', 'GOVT', 'LQD','HYG','USRT','GLD', 'BTC-USD']
start = datetime(2020,1,23)
end = datetime(2021,9,15)
assets_df = yf.download(assets_list, start=start, end=end)
assets = assets_df['Adj Close']
assets.ffill()
benchmark_df = yf.download(['SPY'], start=start, end=end)
benchmark = benchmark_df['Adj Close']
benchmark.ffill()
assets.tail()
[*********************100%***********************] 8 of 8 completed
[*********************100%***********************] 1 of 1 completed
# Check NaN values in the data
nullin_df = pd.DataFrame(assets,columns=assets_list)
print(nullin_df.isnull().sum())
# Calculate portfolio mean return
mu = expected_returns.mean_historical_return(assets)
print(mu)
# Calculate portfolio return variance
sigma = risk_models.sample_cov(assets)
print(sigma)
VV 187
VB 187
GOVT 187
LQD 187
HYG 187
USRT 187
GLD 187
BTC-USD 4
dtype: int64
BTC-USD 0.954596
GLD 0.068603
GOVT 0.023115
HYG 0.038999
LQD 0.043952
USRT 0.100019
VB 0.160255
VV 0.166999
dtype: float64
BTC-USD GLD GOVT HYG LQD USRT VB \
BTC-USD 0.426596 0.016641 -0.000846 0.015506 0.013818 0.046099 0.056703
GLD 0.016641 0.021769 0.001525 0.003135 0.004998 0.005528 0.006021
GOVT -0.000846 0.001525 0.002207 -0.001595 0.002040 -0.005130 -0.004945
HYG 0.015506 0.003135 -0.001595 0.013086 0.006868 0.027635 0.025171
LQD 0.013818 0.004998 0.002040 0.006868 0.013146 0.012050 0.010565
USRT 0.046099 0.005528 -0.005130 0.027635 0.012050 0.088634 0.071004
VB 0.056703 0.006021 -0.004945 0.025171 0.010565 0.071004 0.074049
VV 0.047805 0.006200 -0.004676 0.021801 0.008579 0.058407 0.058066
VV
BTC-USD 0.047805
GLD 0.006200
GOVT -0.004676
HYG 0.021801
LQD 0.008579
USRT 0.058407
VB 0.058066
VV 0.053954
Maximum Sharp Ratio Portfolio
# weight bounds in negative allows shorting of stocks
ef = EfficientFrontier(mu, sigma, weight_bounds=(-1,1))
# optional constraints possible, read pypfopt documentation.
sharpe_portfolio=ef.max_sharpe(risk_free_rate=0.005)
sharpe_portfolio_wt=ef.clean_weights()
print(sharpe_portfolio_wt)
Plotting.plot_weights(sharpe_portfolio_wt)
{'BTC-USD': 0.09318, 'GLD': -0.0131, 'GOVT': 0.96884, 'HYG': -0.11099, 'LQD': -0.16488, 'USRT': -0.08103, 'VB': -0.03686, 'VV': 0.34483}
/root/venv/lib/python3.7/site-packages/cvxpy/expressions/expression.py:556: UserWarning:
This use of ``*`` has resulted in matrix multiplication.
Using ``*`` for matrix multiplication has been deprecated since CVXPY 1.1.
Use ``*`` for matrix-scalar and vector-scalar multiplication.
Use ``@`` for matrix-matrix and matrix-vector multiplication.
Use ``multiply`` for elementwise multiplication.
This code path has been hit 1 times so far.
warnings.warn(msg, UserWarning)
latest_prices = get_latest_prices(assets)
# Allocate Portfolio Value in $ as required to show number of shares/stocks to buy,
# also bounds for shorting will affect allocation
# Maximum Sharpe Portfolio Allocation $1000000
da = DiscreteAllocation(sharpe_portfolio_wt, latest_prices, total_portfolio_value=1000000)
allocation, leftover = da.greedy_portfolio()
print(allocation)
print("Leftover Fund value for the maximum Sharpe portfolio is ${:.2f}".format(leftover))
{'GOVT': 25600, 'VV': 1178, 'BTC-USD': 1, 'LQD': -894, 'HYG': -929, 'USRT': -978, 'VB': -123, 'GLD': -57}
Leftover Fund value for the maximum Sharpe portfolio is $18235.91
max_sharpe_cla = cla.CLA(mu, sigma)
max_sharpe_cla.max_sharpe()
Plotting.plot_efficient_frontier(max_sharpe_cla, show_assets="True")
sharpe_portfolio_wt_list = list(sharpe_portfolio_wt.values())
ret_data1 = assets.pct_change()[1:]
weighted_returns1 = (sharpe_portfolio_wt_list * ret_data1)
portfolio_ret1 = pd.DataFrame(weighted_returns1.sum(axis=1))
ret_data1 = ret_data1.merge(portfolio_ret1, on="Date", how="left")
ret_data1 = ret_data1.rename(columns={0: "portfolio_ret"})
ret_data1.head()
ret_benchmark = benchmark.pct_change()[1:]
ret_data1['cumulative_portfolio_ret'] = (ret_data1['portfolio_ret'] + 1).cumprod()
ret_benchmark['cumulative_spy_ret'] = (ret_benchmark + 1).cumprod()
ret_data1.tail()
sns.scatterplot('Date', 'cumulative_portfolio_ret', data=ret_data1)
sns.scatterplot('Date', 'cumulative_spy_ret', data=ret_benchmark)
/shared-libs/python3.7/py/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
FutureWarning
/shared-libs/python3.7/py/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
FutureWarning
plt.hist(ret_data1['portfolio_ret'], bins=10)
st.norm.interval(alpha=0.95, loc=np.mean(ret_data1['portfolio_ret']), scale=st.sem(ret_data1['portfolio_ret']))
Minimum Volatility Portfolio
ef = EfficientFrontier(mu, sigma, weight_bounds=(-1,1))
min_vol_portfolio=ef.min_volatility()
min_vol_portfolio_wt=ef.clean_weights()
print(min_vol_portfolio_wt)
Plotting.plot_weights(min_vol_portfolio_wt)
{'BTC-USD': -0.00103, 'GLD': -0.01606, 'GOVT': 0.92769, 'HYG': 0.27317, 'LQD': -0.21225, 'USRT': -0.01612, 'VB': -0.01887, 'VV': 0.06347}
da = DiscreteAllocation(min_vol_portfolio_wt, latest_prices, total_portfolio_value=1000000)
allocation, leftover = da.greedy_portfolio()
print(allocation)
print("Leftover Fund value for the maximum Sharpe portfolio is ${:.2f}".format(leftover))
{'GOVT': 27276, 'HYG': 2455, 'VV': 241, 'LQD': -1771, 'VB': -97, 'USRT': -299, 'GLD': -108}
Leftover Fund value for the maximum Sharpe portfolio is $1228.04
min_vol_cla = cla.CLA(mu, sigma)
min_vol_cla.min_volatility()
Plotting.plot_efficient_frontier(min_vol_cla, show_assets="True")
min_vol_portfolio_wt_list = list(min_vol_portfolio_wt.values())
ret_data2 = assets.pct_change()[1:]
weighted_returns2 = (min_vol_portfolio_wt_list * ret_data2)
portfolio_ret2 = pd.DataFrame(weighted_returns2.sum(axis=1))
ret_data2 = ret_data2.merge(portfolio_ret2, on="Date", how="left")
ret_data2 = ret_data2.rename(columns={0: "portfolio_ret"})
ret_data2.head()
ret_data2['cumulative_portfolio_ret'] = (ret_data2['portfolio_ret'] + 1).cumprod()
ret_data2.tail()
sns.scatterplot('Date', 'cumulative_portfolio_ret', data=ret_data2)
sns.scatterplot('Date', 'cumulative_spy_ret', data=ret_benchmark)
/shared-libs/python3.7/py/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
FutureWarning
/shared-libs/python3.7/py/lib/python3.7/site-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
FutureWarning
plt.hist(ret_data2['portfolio_ret'], bins=10)
st.norm.interval(alpha=0.95, loc=np.mean(ret_data2['portfolio_ret']), scale=st.sem(ret_data2['portfolio_ret']))