Data Science Posts and Resources

Articles on Data Science

Applying Logistic Regression

A case study on applying logistic regression to stock market

Laxmi K Soni

38-Minute Read

1:Loading Stock data

1.1:Importing libraries and data

import investpy
from datetime import datetime
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix
from sklearn import metrics
from scipy import stats 
import numpy as np
import pandas as pd
import seaborn as sn
import matplotlib.pyplot as plt
import talib
import quandl
from sklearn.preprocessing import StandardScaler 
from sklearn.metrics import accuracy_score
from sklearn.preprocessing import MinMaxScaler 
import math
from sklearn.metrics import mean_squared_error
import investpy

1.2:Fetching the data

To fetch the stock data we use investpy library. This library fetchs data from investing for example:

1.2.1: Determining position based on volume and close

## C:/Users/slaxm/AppData/Local/r-miniconda/envs/r-reticulate/python.exe:1: SettingWithCopyWarning: 
## A value is trying to be set on a copy of a slice from a DataFrame.
## Try using .loc[row_indexer,col_indexer] = value instead
## 
## See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
##           Date     Open     High      Low    Close  Volume    Pos    weekday
## 544 2021-11-17  66597.0  66960.0  66400.0  66769.0  109119  Short  Wednesday
## 545 2021-11-18  66699.0  66800.0  65720.0  66141.0  152507  Short   Thursday
## 546 2021-11-19  66059.0  66360.0  65472.0  65714.0   95718  Short     Friday
## 547 2021-11-22  65900.0  66147.0  64587.0  64745.0  152068  Short     Monday
## 548 2021-11-23  64828.0  64948.0  62328.0  62575.0  134153  Short    Tuesday

1.2.2: Long positions gains

##          Date     Open     High      Low    Close  Volume    Pos    gain
## 52 2021-08-23  62047.0  63300.0  62047.0  63139.0  131692   Long  1092.0
## 53 2021-09-22  60799.0  61600.0  60799.0  61409.0  129281  Short   610.0
## 54 2021-09-27  60245.0  61340.0  60245.0  60894.0  129203  Short   649.0
## 55 2021-10-19  63500.0  65346.0  63500.0  64608.0  240332   Long  1108.0
## 56 2021-11-04  62880.0  64400.0  62880.0  64292.0   44317  Short  1412.0

1.2.3: Short positions gains

##          Date     Open     High      Low    Close  Volume    Pos    gain
## 43 2021-08-09  65270.0  65270.0  62651.0  62949.0  199170  Short  2321.0
## 44 2021-08-26  63250.0  63250.0  62485.0  62686.0    1439  Short   564.0
## 45 2021-09-23  61310.0  61310.0  60600.0  61024.0  136186  Short   286.0
## 46 2021-09-28  60950.0  60950.0  59783.0  60705.0  188625  Short   245.0
## 47 2021-10-26  66411.0  66411.0  64811.0  65173.0  178748  Short  1238.0
print(short['weekday'].value_counts())
print(lng['weekday'].value_counts())

1.2.2: Determining Average monthly closing prices

##                Open     High      Low    Close  Volume
## Date                                                  
## 2019-10-09  45997.0  46442.0  45845.0  46042.0   66843
## 2019-10-10  46200.0  46201.0  45111.0  45431.0   70005
## 2019-10-11  45350.0  45842.0  44907.0  45176.0   82039
## 2019-10-14  45290.0  45859.0  45230.0  45823.0   54049
## 2019-10-15  45725.0  46020.0  45185.0  45232.0   67928
## ...             ...      ...      ...      ...     ...
## 2021-11-17  66597.0  66960.0  66400.0  66769.0  109119
## 2021-11-18  66699.0  66800.0  65720.0  66141.0  152507
## 2021-11-19  66059.0  66360.0  65472.0  65714.0   95718
## 2021-11-22  65900.0  66147.0  64587.0  64745.0  152068
## 2021-11-23  64828.0  64948.0  62328.0  62575.0  134153
## 
## [549 rows x 5 columns]
##               Open    High     Low   Close  Volume Currency  VolChange  \
## Date                                                                     
## 2019-10-09  45.997  46.442  45.845  46.042   66843      INR  12.760066   
## 2019-10-10  46.200  46.201  45.111  45.431   70005      INR   0.047305   
## 2019-10-11  45.350  45.842  44.907  45.176   82039      INR   0.171902   
## 2019-10-14  45.290  45.859  45.230  45.823   54049      INR  -0.341179   
## 2019-10-15  45.725  46.020  45.185  45.232   67928      INR   0.256786   
## ...            ...     ...     ...     ...     ...      ...        ...   
## 2021-11-17  66.597  66.960  66.400  66.769  109119      INR  -0.380938   
## 2021-11-18  66.699  66.800  65.720  66.141  152507      INR   0.397621   
## 2021-11-19  66.059  66.360  65.472  65.714   95718      INR  -0.372370   
## 2021-11-22  65.900  66.147  64.587  64.745  152068      INR   0.588708   
## 2021-11-23  64.828  64.948  62.328  62.575  134153      INR  -0.117809   
## 
##             CloseChange  
## Date                     
## 2019-10-09     0.000765  
## 2019-10-10    -0.013270  
## 2019-10-11    -0.005613  
## 2019-10-14     0.014322  
## 2019-10-15    -0.012897  
## ...                 ...  
## 2021-11-17     0.005557  
## 2021-11-18    -0.009406  
## 2021-11-19    -0.006456  
## 2021-11-22    -0.014746  
## 2021-11-23    -0.033516  
## 
## [549 rows x 8 columns]
## Date
## 2019-10-31    45.765500
## 2019-11-30    44.712571
## 2019-12-31    44.983905
## 2020-01-31    46.715435
## 2020-02-29    46.769810
## 2020-03-31    42.008682
## 2020-04-30    43.175278
## 2020-05-31    46.064714
## 2020-06-30    48.667409
## 2020-07-31    55.923957
## 2020-08-31    68.698238
## 2020-09-30    65.459455
## 2020-10-31    61.675571
## 2020-11-30    62.081364
## 2020-12-31    65.897045
## 2021-01-31    67.184300
## 2021-02-28    69.205200
## 2021-03-31    66.495957
## 2021-04-30    67.755905
## 2021-05-31    71.557810
## 2021-06-30    70.014500
## 2021-07-31    68.521455
## 2021-08-31    63.957364
## 2021-09-30    62.362182
## 2021-10-31    63.489333
## 2021-11-30    65.243353
## Freq: M, Name: Close, dtype: float64
## meanprice is  58.71615118397085

1.2.3: Technical moving averages

##   period  sma_value sma_signal   ema_value ema_signal
## 0      5   62663.40       sell  62715.6620       sell
## 1     10   63021.50       sell  63050.2669       sell
## 2     20   63820.35       sell  63678.5642       sell
## 3     50   65133.28       sell  64768.3175       sell
## 4    100   65954.77       sell  65335.7978       sell

1.2.4: Determining the z-scores

## Date
## 2021-11-17    0.781437
## 2021-11-18    0.720497
## 2021-11-19    0.679061
## 2021-11-22    0.585031
## 2021-11-23    0.374457
## Name: zscore, dtype: float64
## Date
## 2021-11-17   -0.437323
## 2021-11-18    0.036801
## 2021-11-19   -0.583762
## 2021-11-22    0.032003
## 2021-11-23   -0.163763
## Name: zscorevolume, dtype: float64

1.2.5: Determining the daily support and resistance levels

##         name       s3       s2       s1  pivot_points       r1       r2  \
## 0    Classic  62613.0  63600.0  64173.0       65160.0  65733.0  66720.0   
## 1  Fibonacci  63600.0  64196.0  64564.0       65160.0  65756.0  66124.0   
## 2  Camarilla  64316.0  64459.0  64602.0       65160.0  64888.0  65031.0   
## 3   Woodie's  62405.0  63496.0  63965.0       65056.0  65525.0  66616.0   
## 4   DeMark's      NaN      NaN  63886.0       65016.0  65446.0      NaN   
## 
##         r3  
## 0  67293.0  
## 1  66720.0  
## 2  65174.0  
## 3  67085.0  
## 4      NaN
##   technical_indicator    value      signal
## 0             RSI(14)   38.047        sell
## 1          STOCH(9,6)   96.292  overbought
## 2        STOCHRSI(14)    0.000    oversold
## 3         MACD(12,26)  500.224         buy
## 4             ADX(14)   24.366        sell
## name            DeMark's
## s3                   NaN
## s2                   NaN
## s1                 63886
## pivot_points       65016
## r1                 65446
## r2                   NaN
## r3                   NaN
## Name: 4, dtype: object

1.2.6: Fibonnacci retracement levels

## Retracement levels for rising price
##                               0
## 0             {'min': [62.328]}
## 1  {'level5(61.8)': [63.32884]}
## 2      {'level4(50)': [63.638]}
## 3  {'level3(38.2)': [63.94716]}
## 4  {'level2(23.6)': [64.32968]}
## 5            {'zero': [64.948]}
## Retracement levels for falling price
##                               0
## 0            {'zero': [64.948]}
## 1  {'level2(23.6)': [62.94632]}
## 2  {'level3(38.2)': [63.32884]}
## 3      {'level4(50)': [63.638]}
## 4  {'level5(61.8)': [63.94716]}
## 5             {'min': [62.328]}

As we can see, we now have a data frame with all the entries from start date to end date. We have multiple columns here and not only the closing stock price of the respective day. Let’s take a quick look at the individual columns and their meaning.

Open: That’s the share price the stock had when the markets opened that day.

Close: That’s the share price the stock had when the markets closed that day.

High: That’s the highest share price that the stock had that day.

Low: That’s the lowest share price that the stock had that day.

Volume: Amount of shares that changed hands that day.

1.3:Reading individual values

Since our data is stored in a Pandas data frame, we can use the indexing we already know, to get individual values. For example, we could only print the closing values using print (df[ 'Close' ])

Also, we can go ahead and print the closing value of a specific date that we are interested in. This is possible because the date is our index column.

print (df[ 'Close' ][ '2020-07-14' ])

But we could also use simple indexing to access certain positions.

print (df[ 'Close' ][ 5 ])
## 45.247

Here we printed the closing price of the fifth entry.

2:Graphical Visualization

Even though tables are nice and useful, we want to visualize our financial data, in order to get a better overview. We want to look at the development of the share price.

Actually plotting our share price curve with Pandas and Matplotlib is very simple. Since Pandas builds on top of Matplotlib, we can just select the column we are interested in and apply the plot method. The results are amazing. Since the date is the index of our data frame, Matplotlib uses it for the x-axis. The y-values are then our adjusted close values.

2.1:CandleStick Charts

The best way to visualize stock data is to use so-called candlestick charts . This type of chart gives us information about four different values at the same time, namely the high, the low, the open and the close value. In order to plot candlestick charts, we will need to import a function of the MPL-Finance library.

import mplfinance as fplt

We are importing the candlestick_ohlc function. Notice that there also exists a candlestick_ochl function that takes in the data in a different order. Also, for our candlestick chart, we will need a different date format provided by Matplotlib. Therefore, we need to import the respective module as well. We give it the alias mdates .

import matplotlib.dates as mdates

2.2: Preparing the data for CandleStick charts

Now in order to plot our stock data, we need to select the four columns in the right order.

df1 = df[[ 'Open' , 'High' , 'Low' , 'Close' ]]

Now, we have our columns in the right order but there is still a problem. Our date doesn’t have the right format and since it is the index, we cannot manipulate it. Therefore, we need to reset the index and then convert our datetime to a number.

df1.reset_index( inplace = True )
df1[ 'Date' ] = df1[ 'Date' ].map(mdates.date2num)
## C:/Users/slaxm/AppData/Local/r-miniconda/envs/r-reticulate/python.exe:1: SettingWithCopyWarning: 
## A value is trying to be set on a copy of a slice from a DataFrame.
## Try using .loc[row_indexer,col_indexer] = value instead
## 
## See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

For this, we use the reset_index function so that we can manipulate our Date column. Notice that we are using the inplace parameter to replace the data frame by the new one. After that, we map the date2num function of the matplotlib.dates module on all of our values. That converts our dates into numbers that we can work with.

2.3:Plotting the data

Now we can start plotting our graph. For this, we just define a subplot (because we need to pass one to our function) and call our candlestick_ohlc function.

One candlestick gives us the information about all four values of one specific day. The highest point of the stick is the high and the lowest point is the low of that day. The colored area is the difference between the open and the close price. If the stick is green, the close value is at the top and the open value at the bottom, since the close must be higher than the open. If it is red, it is the other way around.

2.4:Analysis and Statistics

Now let’s get a little bit deeper into the numbers here and away from the visual. From our data we can derive some statistical values that will help us to analyze it.

PERCENTAGE CHANGE

One value that we can calculate is the percentage change of that day. This means by how many percent the share price increased or decreased that day.

The calculation is quite simple. We create a new column with the name PCT_Change and the values are just the difference of the closing and opening values divided by the opening values. Since the open value is the beginning value of that day, we take it as a basis. We could also multiply the result by 100 to get the actual percentage.

##        PCT_Change
## count  549.000000
## mean     0.000369
## std      0.019417
## min     -0.107200
## 25%     -0.006647
## 50%      0.001196
## 75%      0.008981
## max      0.062274
##              Close
## Date              
## 2021-11-23  62.575

*** HIGH LOW PERCENTAGE ***

Another interesting statistic is the high low percentage. Here we just calculate the difference between the highest and the lowest value and divide it by the closing value.

By doing that we can get a feeling of how volatile the stock is.

##               Open    High     Low   Close  Volume Currency  VolChange  \
## Date                                                                     
## 2021-11-17  66.597  66.960  66.400  66.769  109119      INR  -0.380938   
## 2021-11-18  66.699  66.800  65.720  66.141  152507      INR   0.397621   
## 2021-11-19  66.059  66.360  65.472  65.714   95718      INR  -0.372370   
## 2021-11-22  65.900  66.147  64.587  64.745  152068      INR   0.588708   
## 2021-11-23  64.828  64.948  62.328  62.575  134153      INR  -0.117809   
## 
##             CloseChange  usdinr  usdsilver  Open-Close  Open-Open    zscore  \
## Date                                                                          
## 2021-11-17     0.005557  74.238     25.167       0.197     -0.393  0.781437   
## 2021-11-18    -0.009406  74.160     24.900      -0.070      0.102  0.720497   
## 2021-11-19    -0.006456  74.333     24.781      -0.082     -0.640  0.679061   
## 2021-11-22    -0.014746  74.428     24.297       0.186     -0.159  0.585031   
## 2021-11-23    -0.033516  74.447     23.670       0.083     -1.072  0.374457   
## 
##             zscorevolume  PCT_Change    HL_PCT  
## Date                                            
## 2021-11-17     -0.437323    0.002583  0.008387  
## 2021-11-18      0.036801   -0.008366  0.016329  
## 2021-11-19     -0.583762   -0.005223  0.013513  
## 2021-11-22      0.032003   -0.017527  0.024095  
## 2021-11-23     -0.163763   -0.034754  0.041870

These statistical values can be used with many others to get a lot of valuable information about specific stocks. This improves the decision making

*** MOVING AVERAGE ***

we are going to derive the different moving averages . It is the arithmetic mean of all the values of the past n days. Of course this is not the only key statistic that we can derive, but it is the one we are going to use now. We can play around with other functions as well.

What we are going to do with this value is to include it into our data frame and to compare it with the share price of that day.

For this, we will first need to create a new column. Pandas does this automatically when we assign values to a column name. This means that we don’t have to explicitly define that we are creating a new column.

##              Close  5d_ma  20d_ma  50d_ma  100d_ma  200d_ma  5d_ema  20d_ema  \
## Date                                                                           
## 2021-11-15  66.715  66.36   65.24   63.27    64.72    66.85   66.39    65.04   
## 2021-11-16  66.400  66.69   65.33   63.30    64.71    66.84   66.39    65.17   
## 2021-11-17  66.769  66.84   65.38   63.35    64.70    66.82   66.52    65.32   
## 2021-11-18  66.141  66.65   65.43   63.38    64.67    66.81   66.39    65.40   
## 2021-11-19  65.714  66.35   65.42   63.42    64.62    66.79   66.17    65.43   
## 2021-11-22  64.745  65.95   65.35   63.45    64.57    66.77   65.69    65.36   
## 2021-11-23  62.575  65.19   65.22   63.42    64.50    66.74   64.65    65.10   
## 
##             50d_ema  100d_ema  200d_ema  
## Date                                     
## 2021-11-15    64.29     64.81     65.03  
## 2021-11-16    64.37     64.84     65.04  
## 2021-11-17    64.47     64.88     65.06  
## 2021-11-18    64.53     64.91     65.07  
## 2021-11-19    64.58     64.92     65.07  
## 2021-11-22    64.59     64.92     65.07  
## 2021-11-23    64.51     64.87     65.05

Here we define a three new columns with the name 20d_ma, 50d_ma, 100d_ma,200d_ma . We now fill this column with the mean values of every n entries. The rolling function stacks a specific amount of entries, in order to make a statistical calculation possible. The window parameter is the one which defines how many entries we are going to stack. But there is also the min_periods parameter. This one defines how many entries we need to have as a minimum in order to perform the calculation. This is relevant because the first entries of our data frame won’t have a n entries previous to them. By setting this value to zero we start the calculations already with the first number, even if there is not a single previous value. This has the effect that the first value will be just the first number, the second one will be the mean of the first two numbers and so on, until we get to a b values.

By using the mean function, we are obviously calculating the arithmetic mean. However, we can use a bunch of other functions like max, min or median if we like to.

*** Standard Deviation ***

The variability of the closing stock prices determinies how vo widely prices are dispersed from the average price. If the prices are trading in narrow trading range the standard deviation will return a low value that indicates low volatility. If the prices are trading in wide trading range the standard deviation will return high value that indicates high volatility.

## Date
## 2021-11-15    1.144474
## 2021-11-16    0.966832
## 2021-11-17    0.836325
## 2021-11-18    0.449441
## 2021-11-19    0.531065
## 2021-11-22    0.823838
## 2021-11-23    1.496344
## Name: Std_dev, dtype: float64

*** Relative Strength Index ***

The relative strength index is a indicator of mementum used in technical analysis that measures the magnitude of current price changes to know overbought or oversold conditions in the price of a stock or other asset. If RSI is above 70 then it is overbought. If RSI is below 30 then it is oversold condition.

##                   RSI
## Date                 
## 2021-11-17  63.444713
## 2021-11-18  55.928622
## 2021-11-19  51.281560
## 2021-11-22  42.307138
## 2021-11-23  29.361737

*** Average True range ***

##                     ATR     20dayEMA    ATRdiff
## Date                                           
## 2021-11-17  1139.113873  1178.266050 -39.152178
## 2021-11-18  1134.891453  1174.135136 -39.243683
## 2021-11-19  1117.256349  1168.718109 -51.461760
## 2021-11-22  1148.880896  1166.828851 -17.947955
## 2021-11-23  1253.960832  1175.127135  78.833697

*** Wiliams %R ***

Williams %R, or just %R, is a technical analysis oscillator showing the current closing price in relation to the high and low of the past N days.The oscillator is on a negative scale, from −100 (lowest) up to 0 (highest). A value of −100 means the close today was the lowest low of the past N days, and 0 means today’s close was the highest high of the past N days.

##             Williams %R
## Date                   
## 2021-11-15   -19.225936
## 2021-11-16   -35.655995
## 2021-11-17   -24.074074
## 2021-11-18   -45.963756
## 2021-11-19   -88.275194
## 2021-11-22   -94.642252
## 2021-11-23   -95.257296

Readings below -80 represent oversold territory and readings above -20 represent overbought.

*** ADX ***

ADX is used to quantify trend strength. ADX calculations are based on a moving average of price range expansion over a given period of time. The average directional index (ADX) is used to determine when the price is trending strongly.

0-25: Absent or Weak Trend

25-50: Strong Trend

50-75: Very Strong Trend

75-100: Extremely Strong Trend

##                   ADX
## Date                 
## 2021-11-15  32.025064
## 2021-11-16  34.385418
## 2021-11-17  36.408579
## 2021-11-18  33.278604
## 2021-11-19  29.142274
## 2021-11-22  28.386003
## 2021-11-23  32.586853
##                    CCI
## Date                  
## 2021-11-15  127.841868
## 2021-11-16  101.518505
## 2021-11-17   81.639650
## 2021-11-18   46.355650
## 2021-11-19   21.763050
## 2021-11-22  -24.587954
## 2021-11-23 -147.378309
##                  ROC
## Date                
## 2021-11-15  2.707987
## 2021-11-16  4.589988
## 2021-11-17  6.445493
## 2021-11-18  2.875941
## 2021-11-19  1.885330
## 2021-11-22 -0.429072
## 2021-11-23 -3.366535

*** MACD ***

Moving Average Convergence Divergence (MACD) is a trend-following momentum indicator that shows the relationship between two moving averages of a security’s price. The MACD is calculated by subtracting the 26-period Exponential Moving Average (EMA) from the 12-period EMA.

##             MACD_IND
## Date                
## 2021-11-17  0.140514
## 2021-11-18  0.076529
## 2021-11-19 -0.001808
## 2021-11-22 -0.119619
## 2021-11-23 -0.332368
## Date
## 2019-10-09            NaN
## 2019-10-10            NaN
## 2019-10-11            NaN
## 2019-10-14            NaN
## 2019-10-15            NaN
##                  ...     
## 2021-11-17     196.105947
## 2021-11-18     282.436527
## 2021-11-19    1274.761270
## 2021-11-22    -111.075635
## 2021-11-23       1.463797
## Name: STC, Length: 549, dtype: float64

*** Bollinger Bands ***

Bollinger Bands are a type of statistical chart characterizing the prices and volatility over time of a financial instrument or commodity.

## 67810.0 65220.0 62620.0

In case we choose another value than zero for our min_periods parameter, we will end up with a couple of NaN-Values . These are not a number values and they are useless. Therefore, we would want to delete the entries that have such values.

We do this by using the dropna function. If we would have had any entries with NaN values in any column, they would now have been deleted

3: Predicting the movement of stock

To predict the movement of the stock we use 5 lag returns as the dependent variables. The first leg is return yesterday, leg2 is return day before yesterday and so on. The dependent variable is whether the prices went up or down on that day. Other variables include the technical indicators which along with 5 lag returns are used to predict the movement of stock using logistic regression.

3.1: Creating lag returns

3.2: Creating returns dataframe

3.2: create the lagged percentage returns columns

##                Today      Lag1      Lag2      Lag3      Lag4      Lag5  \
## Date                                                                     
## 2021-11-10  1.996757 -0.413693  0.815529  0.320413  2.496572 -1.197114   
## 2021-11-11  1.506480  1.996757 -0.413693  0.815529  0.320413  2.496572   
## 2021-11-12  0.308757  1.506480  1.996757 -0.413693  0.815529  0.320413   
## 2021-11-15 -0.795539  0.308757  1.506480  1.996757 -0.413693  0.815529   
## 2021-11-16 -0.472158 -0.795539  0.308757  1.506480  1.996757 -0.413693   
## 2021-11-17  0.555723 -0.472158 -0.795539  0.308757  1.506480  1.996757   
## 2021-11-18 -0.940556  0.555723 -0.472158 -0.795539  0.308757  1.506480   
## 2021-11-19 -0.645590 -0.940556  0.555723 -0.472158 -0.795539  0.308757   
## 2021-11-22 -1.474572 -0.645590 -0.940556  0.555723 -0.472158 -0.795539   
## 2021-11-23 -3.351610 -1.474572 -0.645590 -0.940556  0.555723 -0.472158   
## 
##                 Lag6      Lag7      Lag8      Lag9     Lag10     Lag11  \
## Date                                                                     
## 2021-11-10 -2.263070  0.356895 -0.586726 -0.352021  0.251638 -1.633084   
## 2021-11-11 -1.197114 -2.263070  0.356895 -0.586726 -0.352021  0.251638   
## 2021-11-12  2.496572 -1.197114 -2.263070  0.356895 -0.586726 -0.352021   
## 2021-11-15  0.320413  2.496572 -1.197114 -2.263070  0.356895 -0.586726   
## 2021-11-16  0.815529  0.320413  2.496572 -1.197114 -2.263070  0.356895   
## 2021-11-17 -0.413693  0.815529  0.320413  2.496572 -1.197114 -2.263070   
## 2021-11-18  1.996757 -0.413693  0.815529  0.320413  2.496572 -1.197114   
## 2021-11-19  1.506480  1.996757 -0.413693  0.815529  0.320413  2.496572   
## 2021-11-22  0.308757  1.506480  1.996757 -0.413693  0.815529  0.320413   
## 2021-11-23 -0.795539  0.308757  1.506480  1.996757 -0.413693  0.815529   
## 
##                Lag12     Lag13     Lag14     Lag15  
## Date                                                
## 2021-11-10  0.677719  0.910833 -0.726116  1.677811  
## 2021-11-11 -1.633084  0.677719  0.910833 -0.726116  
## 2021-11-12  0.251638 -1.633084  0.677719  0.910833  
## 2021-11-15 -0.352021  0.251638 -1.633084  0.677719  
## 2021-11-16 -0.586726 -0.352021  0.251638 -1.633084  
## 2021-11-17  0.356895 -0.586726 -0.352021  0.251638  
## 2021-11-18 -2.263070  0.356895 -0.586726 -0.352021  
## 2021-11-19 -1.197114 -2.263070  0.356895 -0.586726  
## 2021-11-22  2.496572 -1.197114 -2.263070  0.356895  
## 2021-11-23  0.320413  2.496572 -1.197114 -2.263070

3.3: “Direction” column (+1 or -1) indicating an up/down day

3.4: Create the dependent and independent variables

## C:/Users/slaxm/AppData/Local/r-miniconda/envs/r-reticulate/python.exe:1: SettingWithCopyWarning: 
## A value is trying to be set on a copy of a slice from a DataFrame.
## Try using .loc[row_indexer,col_indexer] = value instead
## 
## See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
##                 Lag1      Lag2      Lag3      Lag4      Lag5      Lag6  \
## Date                                                                     
## 2021-11-17 -0.472158 -0.795539  0.308757  1.506480  1.996757 -0.413693   
## 2021-11-18  0.555723 -0.472158 -0.795539  0.308757  1.506480  1.996757   
## 2021-11-19 -0.940556  0.555723 -0.472158 -0.795539  0.308757  1.506480   
## 2021-11-22 -0.645590 -0.940556  0.555723 -0.472158 -0.795539  0.308757   
## 2021-11-23 -1.474572 -0.645590 -0.940556  0.555723 -0.472158 -0.795539   
## 
##                 Lag7      Lag8      Lag9     Lag10  usdinr  usdsilver  20d_ma  \
## Date                                                                            
## 2021-11-17  0.815529  0.320413  2.496572 -1.197114  74.238     25.167   65.38   
## 2021-11-18 -0.413693  0.815529  0.320413  2.496572  74.160     24.900   65.43   
## 2021-11-19  1.996757 -0.413693  0.815529  0.320413  74.333     24.781   65.42   
## 2021-11-22  1.506480  1.996757 -0.413693  0.815529  74.428     24.297   65.35   
## 2021-11-23  0.308757  1.506480  1.996757 -0.413693  74.447     23.670   65.22   
## 
##             20d_ema  50d_ma  50d_ema  100d_ma  100d_ema  200d_ma  200d_ema  \
## Date                                                                         
## 2021-11-17    65.32   63.35    64.47    64.70     64.88    66.82     65.06   
## 2021-11-18    65.40   63.38    64.53    64.67     64.91    66.81     65.07   
## 2021-11-19    65.43   63.42    64.58    64.62     64.92    66.79     65.07   
## 2021-11-22    65.36   63.45    64.59    64.57     64.92    66.77     65.07   
## 2021-11-23    65.10   63.42    64.51    64.50     64.87    66.74     65.05   
## 
##             250d_ma  250d_ema   Std_dev        RSI  Williams %R  MACD_IND  \
## Date                                                                        
## 2021-11-17    66.81     66.81  0.836325  63.444713   -24.074074  0.140514   
## 2021-11-18    66.83     66.83  0.449441  55.928622   -45.963756  0.076529   
## 2021-11-19    66.85     66.85  0.531065  51.281560   -88.275194 -0.001808   
## 2021-11-22    66.85     66.85  0.823838  42.307138   -94.642252 -0.119619   
## 2021-11-23    66.85     66.85  1.496344  29.361737   -95.257296 -0.332368   
## 
##             VolChange  CloseChange    zscore         CCI       ROC         DX  \
## Date                                                                            
## 2021-11-17  -0.380938     0.005557  0.781437   81.639650  6.445493  30.697864   
## 2021-11-18   0.397621    -0.009406  0.720497   46.355650  2.875941  17.084232   
## 2021-11-19  -0.372370    -0.006456  0.679061   21.763050  1.885330  12.482535   
## 2021-11-22   0.588708    -0.014746  0.585031  -24.587954 -0.429072   2.277591   
## 2021-11-23  -0.117809    -0.033516  0.374457 -147.378309 -3.366535  28.182975   
## 
##                   SAR  Open-Close  Open-Open  
## Date                                          
## 2021-11-17  63.501552       0.197     -0.393  
## 2021-11-18  63.824308      -0.070      0.102  
## 2021-11-19  64.121243      -0.082     -0.640  
## 2021-11-22  64.394424       0.186     -0.159  
## 2021-11-23  67.536000       0.083     -1.072

3.5: Create training and test sets

##                 Lag1      Lag2      Lag3      Lag4      Lag5      Lag6  \
## Date                                                                     
## 2021-11-17 -0.472158 -0.795539  0.308757  1.506480  1.996757 -0.413693   
## 2021-11-18  0.555723 -0.472158 -0.795539  0.308757  1.506480  1.996757   
## 2021-11-19 -0.940556  0.555723 -0.472158 -0.795539  0.308757  1.506480   
## 2021-11-22 -0.645590 -0.940556  0.555723 -0.472158 -0.795539  0.308757   
## 2021-11-23 -1.474572 -0.645590 -0.940556  0.555723 -0.472158 -0.795539   
## 
##                 Lag7      Lag8      Lag9     Lag10  usdinr  usdsilver  20d_ma  \
## Date                                                                            
## 2021-11-17  0.815529  0.320413  2.496572 -1.197114  74.238     25.167   65.38   
## 2021-11-18 -0.413693  0.815529  0.320413  2.496572  74.160     24.900   65.43   
## 2021-11-19  1.996757 -0.413693  0.815529  0.320413  74.333     24.781   65.42   
## 2021-11-22  1.506480  1.996757 -0.413693  0.815529  74.428     24.297   65.35   
## 2021-11-23  0.308757  1.506480  1.996757 -0.413693  74.447     23.670   65.22   
## 
##             20d_ema  50d_ma  50d_ema  100d_ma  100d_ema  200d_ma  200d_ema  \
## Date                                                                         
## 2021-11-17    65.32   63.35    64.47    64.70     64.88    66.82     65.06   
## 2021-11-18    65.40   63.38    64.53    64.67     64.91    66.81     65.07   
## 2021-11-19    65.43   63.42    64.58    64.62     64.92    66.79     65.07   
## 2021-11-22    65.36   63.45    64.59    64.57     64.92    66.77     65.07   
## 2021-11-23    65.10   63.42    64.51    64.50     64.87    66.74     65.05   
## 
##             250d_ma  250d_ema   Std_dev        RSI  Williams %R  MACD_IND  \
## Date                                                                        
## 2021-11-17    66.81     66.81  0.836325  63.444713   -24.074074  0.140514   
## 2021-11-18    66.83     66.83  0.449441  55.928622   -45.963756  0.076529   
## 2021-11-19    66.85     66.85  0.531065  51.281560   -88.275194 -0.001808   
## 2021-11-22    66.85     66.85  0.823838  42.307138   -94.642252 -0.119619   
## 2021-11-23    66.85     66.85  1.496344  29.361737   -95.257296 -0.332368   
## 
##             VolChange  CloseChange    zscore         CCI       ROC         DX  \
## Date                                                                            
## 2021-11-17  -0.380938     0.005557  0.781437   81.639650  6.445493  30.697864   
## 2021-11-18   0.397621    -0.009406  0.720497   46.355650  2.875941  17.084232   
## 2021-11-19  -0.372370    -0.006456  0.679061   21.763050  1.885330  12.482535   
## 2021-11-22   0.588708    -0.014746  0.585031  -24.587954 -0.429072   2.277591   
## 2021-11-23  -0.117809    -0.033516  0.374457 -147.378309 -3.366535  28.182975   
## 
##                   SAR  Open-Close  Open-Open  
## Date                                          
## 2021-11-17  63.501552       0.197     -0.393  
## 2021-11-18  63.824308      -0.070      0.102  
## 2021-11-19  64.121243      -0.082     -0.640  
## 2021-11-22  64.394424       0.186     -0.159  
## 2021-11-23  67.536000       0.083     -1.072

3.6: Create model

3.7: train the model on the training set

## LogisticRegression(max_iter=1000000)

3.8: make an array of predictions on the test set

3.9: output the hit-rate and the confusion matrix for the model

## 
## Train Accuracy: 94.33%
## Test Accuracy: 85.28%
## [[ 84   3]
##  [ 31 113]]

3.10: Predict movement of stock for tomorrow.

##             y_test  y_pred
## Date                      
## 2021-11-17       1       1
## 2021-11-18      -1      -1
## 2021-11-19      -1      -1
## 2021-11-22      -1      -1
## 2021-11-23      -1      -1
## [-1]
## last Close:  62575
## Hourly Technical Indicators:
## name            Fibonacci
## s3                  62342
## s2                  62429
## s1                  62483
## pivot_points        62570
## r1                  62657
## r2                  62711
## r3                  62798
## Name: 1, dtype: object
##   technical_indicator    value      signal
## 0             RSI(14)   20.847    oversold
## 1          STOCH(9,6)   96.980  overbought
## 2        STOCHRSI(14)   92.506  overbought
## 3         MACD(12,26) -777.427        sell
## 4             ADX(14)   78.343    oversold
##   period  sma_value sma_signal   ema_value ema_signal
## 0      5   62663.40       sell  62715.6620       sell
## 1     10   63021.50       sell  63050.2669       sell
## 2     20   63820.35       sell  63678.5642       sell
## 3     50   65133.28       sell  64768.3175       sell
## 4    100   65954.77       sell  65335.7978       sell
## Daily Technical Indicators:
## name            Fibonacci
## s3                  63600
## s2                  64196
## s1                  64564
## pivot_points        65160
## r1                  65756
## r2                  66124
## r3                  66720
## Name: 1, dtype: object
##   technical_indicator    value      signal
## 0             RSI(14)   38.047        sell
## 1          STOCH(9,6)   96.292  overbought
## 2        STOCHRSI(14)    0.000    oversold
## 3         MACD(12,26)  500.224         buy
## 4             ADX(14)   24.366        sell
##   period  sma_value sma_signal   ema_value ema_signal
## 0      5   65202.80       sell  64739.3522       sell
## 1     10   65947.00       sell  65168.4025       sell
## 2     20   65218.80       sell  65003.4585       sell
## 3     50   63422.02       sell  64538.6863       sell
## 4    100   64497.02       sell  64876.3212       sell
## Lag1            -1.474572
## Lag2            -0.645590
## Lag3            -0.940556
## Lag4             0.555723
## Lag5            -0.472158
## Lag6            -0.795539
## Lag7             0.308757
## Lag8             1.506480
## Lag9             1.996757
## Lag10           -0.413693
## usdinr          74.447000
## usdsilver       23.670000
## 20d_ma          65.220000
## 20d_ema         65.100000
## 50d_ma          63.420000
## 50d_ema         64.510000
## 100d_ma         64.500000
## 100d_ema        64.870000
## 200d_ma         66.740000
## 200d_ema        65.050000
## 250d_ma         66.850000
## 250d_ema        66.850000
## Std_dev          1.496344
## RSI             29.361737
## Williams %R    -95.257296
## MACD_IND        -0.332368
## VolChange       -0.117809
## CloseChange     -0.033516
## zscore           0.374457
## CCI           -147.378309
## ROC             -3.366535
## DX              28.182975
## SAR             67.536000
## Open-Close       0.083000
## Open-Open       -1.072000
## Name: 2021-11-23 00:00:00, dtype: float64

pd.set_option('display.max_columns', None)

dfusd2 = df[['Open','High','Low','Close','Volume']]
diffdf = dfusd2.diff()
dfusd2['Pos'] = np.where((dfusd2['Volume'] > dfusd2['Volume'].shift(1)) & ((dfusd2['Close'] > dfusd2['Close'].shift(1))),"Long","Short")



short = dfusd2[dfusd2['Open'] == dfusd2['High']]
lng = dfusd2[dfusd2['Open'] == dfusd2['Low']]


short['gain'] = short['High'] - short['Close']
## C:/Users/slaxm/AppData/Local/r-miniconda/envs/r-reticulate/python.exe:1: SettingWithCopyWarning: 
## A value is trying to be set on a copy of a slice from a DataFrame.
## Try using .loc[row_indexer,col_indexer] = value instead
## 
## See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
lng['gain'] = lng['Close'] - lng['Low']

dfusd2 = dfusd2.reset_index()

dfusd2['weekday'] = dfusd2['Date'].dt.day_name()

print(dfusd2.tail(5))
##            Date    Open    High     Low   Close  Volume    Pos    weekday
## 2143 2021-11-19  24.850  25.040  24.610  24.781   67983  Short     Friday
## 2144 2021-11-21  24.655  24.688  24.555  24.650       0  Short     Sunday
## 2145 2021-11-22  24.670  24.955  24.140  24.297       0  Short     Monday
## 2146 2021-11-23  24.260  24.337  23.277  23.670       0  Short    Tuesday
## 2147 2021-11-24  23.672  23.688  23.640  23.652       0  Short  Wednesday
short = short.reset_index()
lng = lng.reset_index()

short['weekday'] = short['Date'].dt.day_name()
lng['weekday'] = lng['Date'].dt.day_name()

1.2.2: Determining Average monthly closing prices

##               Open    High     Low   Close  Volume
## Date                                              
## 2015-01-01  15.695  15.903  15.695  15.813       0
## 2015-01-02  15.790  15.815  15.535  15.734      13
## 2015-01-04  15.782  15.790  15.633  15.740       0
## 2015-01-05  16.179  16.179  16.179  16.179   47134
## 2015-01-06  16.603  16.603  16.603  16.603       2
## ...            ...     ...     ...     ...     ...
## 2021-11-19  24.850  25.040  24.610  24.781   67983
## 2021-11-21  24.655  24.688  24.555  24.650       0
## 2021-11-22  24.670  24.955  24.140  24.297       0
## 2021-11-23  24.260  24.337  23.277  23.670       0
## 2021-11-24  23.672  23.688  23.640  23.652       0
## 
## [2148 rows x 5 columns]
##               Open    High     Low   Close  Volume Currency   VolChange  \
## Date                                                                      
## 2015-01-01  15.695  15.903  15.695  15.813       0      USD  354.438439   
## 2015-01-02  15.790  15.815  15.535  15.734      13      USD  354.438439   
## 2015-01-04  15.782  15.790  15.633  15.740       0      USD   -1.000000   
## 2015-01-05  16.179  16.179  16.179  16.179   47134      USD  354.438439   
## 2015-01-06  16.603  16.603  16.603  16.603       2      USD   -0.999958   
## ...            ...     ...     ...     ...     ...      ...         ...   
## 2021-11-19  24.850  25.040  24.610  24.781   67983      USD    0.068579   
## 2021-11-21  24.655  24.688  24.555  24.650       0      USD   -1.000000   
## 2021-11-22  24.670  24.955  24.140  24.297       0      USD  354.438439   
## 2021-11-23  24.260  24.337  23.277  23.670       0      USD  354.438439   
## 2021-11-24  23.672  23.688  23.640  23.652       0      USD  354.438439   
## 
##             CloseChange  
## Date                     
## 2015-01-01     0.000907  
## 2015-01-02    -0.004996  
## 2015-01-04     0.000381  
## 2015-01-05     0.027891  
## 2015-01-06     0.026207  
## ...                 ...  
## 2021-11-19    -0.004779  
## 2021-11-21    -0.005286  
## 2021-11-22    -0.014320  
## 2021-11-23    -0.025806  
## 2021-11-24    -0.000760  
## 
## [2148 rows x 8 columns]
## Date
## 2015-01-31    17.157615
## 2015-02-28    16.767750
## 2015-03-31    16.223519
## 2015-04-30    16.347077
## 2015-05-31    16.808962
##                 ...    
## 2021-07-31    25.753680
## 2021-08-31    23.982963
## 2021-09-30    23.296385
## 2021-10-31    23.422654
## 2021-11-30    24.474571
## Freq: M, Name: Close, Length: 83, dtype: float64
## meanprice is  18.861714152700188

1.2.3: Technical moving averages

##   period  sma_value sma_signal   ema_value ema_signal
## 0      5   65202.80       sell  64739.3522       sell
## 1     10   65947.00       sell  65168.4025       sell
## 2     20   65218.80       sell  65003.4585       sell
## 3     50   63422.02       sell  64538.6863       sell
## 4    100   64497.02       sell  64876.3212       sell

1.2.4: Determining the z-scores

## Date
## 2021-11-19    1.623277
## 2021-11-21    1.587352
## 2021-11-22    1.490547
## 2021-11-23    1.318602
## 2021-11-24    1.313665
## Name: zscore, dtype: float64
## Date
## 2021-11-19    4.429318
## 2021-11-21   -0.287525
## 2021-11-22   -0.287525
## 2021-11-23   -0.287525
## 2021-11-24   -0.287525
## Name: zscorevolume, dtype: float64

1.2.5: Determining the daily support and resistance levels

##                    PP         R1         S1         R2         S2         R3  \
## Date                                                                           
## 2021-11-19  24.810333  25.010667  24.580667  25.240333  24.380333  25.440667   
## 2021-11-21  24.631000  24.707000  24.574000  24.764000  24.498000  24.840000   
## 2021-11-22  24.464000  24.788000  23.973000  25.279000  23.649000  25.603000   
## 2021-11-23  23.761333  24.245667  23.185667  24.821333  22.701333  25.305667   
## 2021-11-24  23.660000  23.680000  23.632000  23.708000  23.612000  23.728000   
## 
##                    S3  
## Date                   
## 2021-11-19  24.150667  
## 2021-11-21  24.441000  
## 2021-11-22  23.158000  
## 2021-11-23  22.125667  
## 2021-11-24  23.584000

1.2.6: Fibonnacci retracement levels

## Retracement levels for rising price
##                                0
## 0               {'min': [23.64]}
## 1  {'level5(61.8)': [23.658336]}
## 2       {'level4(50)': [23.664]}
## 3  {'level3(38.2)': [23.669664]}
## 4  {'level2(23.6)': [23.676672]}
## 5             {'zero': [23.688]}
## Retracement levels for falling price
##                                0
## 0             {'zero': [23.688]}
## 1  {'level2(23.6)': [23.651328]}
## 2  {'level3(38.2)': [23.658336]}
## 3       {'level4(50)': [23.664]}
## 4  {'level5(61.8)': [23.669664]}
## 5               {'min': [23.64]}

As we can see, we now have a data frame with all the entries from start date to end date. We have multiple columns here and not only the closing stock price of the respective day. Let’s take a quick look at the individual columns and their meaning.

Open: That’s the share price the stock had when the markets opened that day.

Close: That’s the share price the stock had when the markets closed that day.

High: That’s the highest share price that the stock had that day.

Low: That’s the lowest share price that the stock had that day.

Volume: Amount of shares that changed hands that day.

1.3:Reading individual values

Since our data is stored in a Pandas data frame, we can use the indexing we already know, to get individual values. For example, we could only print the closing values using print (df[ 'Close' ])

Also, we can go ahead and print the closing value of a specific date that we are interested in. This is possible because the date is our index column.

print (df[ 'Close' ][ '2020-07-14' ])
## 20.114

But we could also use simple indexing to access certain positions.

print (df[ 'Close' ][ 5 ])
## 16.51

Here we printed the closing price of the fifth entry.

2:Graphical Visualization

Even though tables are nice and useful, we want to visualize our financial data, in order to get a better overview. We want to look at the development of the share price.

Actually plotting our share price curve with Pandas and Matplotlib is very simple. Since Pandas builds on top of Matplotlib, we can just select the column we are interested in and apply the plot method. The results are amazing. Since the date is the index of our data frame, Matplotlib uses it for the x-axis. The y-values are then our adjusted close values.

2.1:CandleStick Charts

The best way to visualize stock data is to use so-called candlestick charts . This type of chart gives us information about four different values at the same time, namely the high, the low, the open and the close value. In order to plot candlestick charts, we will need to import a function of the MPL-Finance library.

import mplfinance as fplt

We are importing the candlestick_ohlc function. Notice that there also exists a candlestick_ochl function that takes in the data in a different order. Also, for our candlestick chart, we will need a different date format provided by Matplotlib. Therefore, we need to import the respective module as well. We give it the alias mdates .

import matplotlib.dates as mdates

2.2: Preparing the data for CandleStick charts

Now in order to plot our stock data, we need to select the four columns in the right order.

df1 = df[[ 'Open' , 'High' , 'Low' , 'Close' ]]

Now, we have our columns in the right order but there is still a problem. Our date doesn’t have the right format and since it is the index, we cannot manipulate it. Therefore, we need to reset the index and then convert our datetime to a number.

df1.reset_index( inplace = True )
df1[ 'Date' ] = df1[ 'Date' ].map(mdates.date2num)
## C:/Users/slaxm/AppData/Local/r-miniconda/envs/r-reticulate/python.exe:1: SettingWithCopyWarning: 
## A value is trying to be set on a copy of a slice from a DataFrame.
## Try using .loc[row_indexer,col_indexer] = value instead
## 
## See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

For this, we use the reset_index function so that we can manipulate our Date column. Notice that we are using the inplace parameter to replace the data frame by the new one. After that, we map the date2num function of the matplotlib.dates module on all of our values. That converts our dates into numbers that we can work with.

2.3:Plotting the data

Now we can start plotting our graph. For this, we just define a subplot (because we need to pass one to our function) and call our candlestick_ohlc function.

One candlestick gives us the information about all four values of one specific day. The highest point of the stick is the high and the lowest point is the low of that day. The colored area is the difference between the open and the close price. If the stick is green, the close value is at the top and the open value at the bottom, since the close must be higher than the open. If it is red, it is the other way around.

2.4:Analysis and Statistics

Now let’s get a little bit deeper into the numbers here and away from the visual. From our data we can derive some statistical values that will help us to analyze it.

PERCENTAGE CHANGE

One value that we can calculate is the percentage change of that day. This means by how many percent the share price increased or decreased that day.

The calculation is quite simple. We create a new column with the name PCT_Change and the values are just the difference of the closing and opening values divided by the opening values. Since the open value is the beginning value of that day, we take it as a basis. We could also multiply the result by 100 to get the actual percentage.

##         PCT_Change
## count  2148.000000
## mean      0.033093
## std       0.039290
## min      -0.075191
## 25%       0.000351
## 50%       0.012114
## 75%       0.070744
## max       0.136919
##              Close
## Date              
## 2021-11-24  23.652

*** HIGH LOW PERCENTAGE ***

Another interesting statistic is the high low percentage. Here we just calculate the difference between the highest and the lowest value and divide it by the closing value.

By doing that we can get a feeling of how volatile the stock is.

##               Open    High     Low   Close  Volume  ... Open-Open    zscore  \
## Date                                                ...                       
## 2021-11-19  24.850  25.040  24.610  24.781   67983  ...    -0.300  1.623277   
## 2021-11-21  24.655  24.688  24.555  24.650       0  ...    -0.195  1.587352   
## 2021-11-22  24.670  24.955  24.140  24.297       0  ...     0.015  1.490547   
## 2021-11-23  24.260  24.337  23.277  23.670       0  ...    -0.410  1.318602   
## 2021-11-24  23.672  23.688  23.640  23.652       0  ...    -0.588  1.313665   
## 
##             zscorevolume  PCT_Change    HL_PCT  
## Date                                            
## 2021-11-19      4.429318   -0.002777  0.017352  
## 2021-11-21     -0.287525   -0.000203  0.005396  
## 2021-11-22     -0.287525   -0.015120  0.033543  
## 2021-11-23     -0.287525   -0.024320  0.044782  
## 2021-11-24     -0.287525   -0.000845  0.002029  
## 
## [5 rows x 15 columns]

These statistical values can be used with many others to get a lot of valuable information about specific stocks. This improves the decision making

*** MOVING AVERAGE ***

we are going to derive the different moving averages . It is the arithmetic mean of all the values of the past n days. Of course this is not the only key statistic that we can derive, but it is the one we are going to use now. We can play around with other functions as well.

What we are going to do with this value is to include it into our data frame and to compare it with the share price of that day.

For this, we will first need to create a new column. Pandas does this automatically when we assign values to a column name. This means that we don’t have to explicitly define that we are creating a new column.

##              Close  5d_ma  20d_ma  50d_ma  100d_ma  ...  5d_ema  20d_ema  \
## Date                                                ...                    
## 2021-11-17  25.167  25.20   24.41   23.57    23.82  ...   25.08    24.54   
## 2021-11-18  24.900  25.11   24.45   23.62    23.81  ...   25.02    24.58   
## 2021-11-19  24.781  24.98   24.48   23.66    23.81  ...   24.94    24.60   
## 2021-11-21  24.650  24.89   24.51   23.70    23.81  ...   24.84    24.60   
## 2021-11-22  24.297  24.76   24.52   23.73    23.80  ...   24.66    24.57   
## 2021-11-23  23.670  24.46   24.52   23.76    23.78  ...   24.33    24.49   
## 2021-11-24  23.652  24.21   24.49   23.78    23.76  ...   24.11    24.41   
## 
##             50d_ema  100d_ema  200d_ema  
## Date                                     
## 2021-11-17    24.07     24.21     24.61  
## 2021-11-18    24.10     24.22     24.62  
## 2021-11-19    24.13     24.23     24.62  
## 2021-11-21    24.15     24.24     24.62  
## 2021-11-22    24.16     24.24     24.62  
## 2021-11-23    24.14     24.23     24.61  
## 2021-11-24    24.12     24.22     24.60  
## 
## [7 rows x 11 columns]

Here we define a three new columns with the name 20d_ma, 50d_ma, 100d_ma,200d_ma . We now fill this column with the mean values of every n entries. The rolling function stacks a specific amount of entries, in order to make a statistical calculation possible. The window parameter is the one which defines how many entries we are going to stack. But there is also the min_periods parameter. This one defines how many entries we need to have as a minimum in order to perform the calculation. This is relevant because the first entries of our data frame won’t have a n entries previous to them. By setting this value to zero we start the calculations already with the first number, even if there is not a single previous value. This has the effect that the first value will be just the first number, the second one will be the mean of the first two numbers and so on, until we get to a b values.

By using the mean function, we are obviously calculating the arithmetic mean. However, we can use a bunch of other functions like max, min or median if we like to.

*** Standard Deviation ***

The variability of the closing stock prices determinies how vo widely prices are dispersed from the average price. If the prices are trading in narrow trading range the standard deviation will return a low value that indicates low volatility. If the prices are trading in wide trading range the standard deviation will return high value that indicates high volatility.

## Date
## 2021-11-17    0.232938
## 2021-11-18    0.200996
## 2021-11-19    0.237321
## 2021-11-21    0.259725
## 2021-11-22    0.296033
## 2021-11-23    0.502586
## 2021-11-24    0.596463
## Name: Std_dev, dtype: float64

*** Relative Strength Index ***

The relative strength index is a indicator of mementum used in technical analysis that measures the magnitude of current price changes to know overbought or oversold conditions in the price of a stock or other asset. If RSI is above 70 then it is overbought. If RSI is below 30 then it is oversold condition.

##                   RSI
## Date                 
## 2021-11-19  52.672052
## 2021-11-21  49.313888
## 2021-11-22  41.326476
## 2021-11-23  31.221488
## 2021-11-24  30.976863

*** Average True range ***

##                    ATR    20dayEMA    ATRdiff
## Date                                         
## 2021-11-19  518.449148  525.188977  -6.739829
## 2021-11-21  497.559923  522.557639 -24.997716
## 2021-11-22  520.234214  522.336360  -2.102146
## 2021-11-23  558.788913  525.808032  32.980881
## 2021-11-24  522.303991  525.474314  -3.170323

*** Wiliams %R ***

Williams %R, or just %R, is a technical analysis oscillator showing the current closing price in relation to the high and low of the past N days.The oscillator is on a negative scale, from −100 (lowest) up to 0 (highest). A value of −100 means the close today was the lowest low of the past N days, and 0 means today’s close was the highest high of the past N days.

##             Williams %R
## Date                   
## 2021-11-17   -22.275862
## 2021-11-18   -69.822485
## 2021-11-19   -80.568182
## 2021-11-21   -89.839572
## 2021-11-22   -88.370370
## 2021-11-23   -82.241301
## 2021-11-24   -81.324701

Readings below -80 represent oversold territory and readings above -20 represent overbought.

*** ADX ***

ADX is used to quantify trend strength. ADX calculations are based on a moving average of price range expansion over a given period of time. The average directional index (ADX) is used to determine when the price is trending strongly.

0-25: Absent or Weak Trend

25-50: Strong Trend

50-75: Very Strong Trend

75-100: Extremely Strong Trend

##                   ADX
## Date                 
## 2021-11-17  25.413472
## 2021-11-18  22.622768
## 2021-11-19  21.603363
## 2021-11-21  21.354521
## 2021-11-22  24.724864
## 2021-11-23  31.008422
## 2021-11-24  36.394329
##                    CCI
## Date                  
## 2021-11-17   64.715801
## 2021-11-18   40.095278
## 2021-11-19   10.374389
## 2021-11-21  -31.875044
## 2021-11-22  -75.887411
## 2021-11-23 -192.510491
## 2021-11-24 -171.581145
##                  ROC
## Date                
## 2021-11-17  4.180983
## 2021-11-18  2.828825
## 2021-11-19  0.973841
## 2021-11-21  1.365244
## 2021-11-22 -1.917487
## 2021-11-23 -6.446386
## 2021-11-24 -6.683500

*** MACD ***

Moving Average Convergence Divergence (MACD) is a trend-following momentum indicator that shows the relationship between two moving averages of a security’s price. The MACD is calculated by subtracting the 26-period Exponential Moving Average (EMA) from the 12-period EMA.

##             MACD_IND
## Date                
## 2021-11-19 -0.005007
## 2021-11-21 -0.036860
## 2021-11-22 -0.080771
## 2021-11-23 -0.147239
## 2021-11-24 -0.185162

*** Bollinger Bands ***

Bollinger Bands are a type of statistical chart characterizing the prices and volatility over time of a financial instrument or commodity.

## 25.77 24.49 23.22

In case we choose another value than zero for our min_periods parameter, we will end up with a couple of NaN-Values . These are not a number values and they are useless. Therefore, we would want to delete the entries that have such values.

We do this by using the dropna function. If we would have had any entries with NaN values in any column, they would now have been deleted

3: Predicting the movement of stock

To predict the movement of the stock we use 5 lag returns as the dependent variables. The first leg is return yesterday, leg2 is return day before yesterday and so on. The dependent variable is whether the prices went up or down on that day. Other variables include the technical indicators which along with 5 lag returns are used to predict the movement of stock using logistic regression.

3.1: Creating lag returns

3.2: Creating returns dataframe

3.2: create the lagged percentage returns columns

##                Today      Lag1      Lag2      Lag3      Lag4  ...     Lag11  \
## Date                                                          ...             
## 2021-11-04  2.927123 -1.174118 -1.845589 -0.708955 -0.293498  ... -0.545214   
## 2021-11-05  1.028815  2.927123 -1.174118 -1.845589 -0.708955  ...  2.287036   
## 2021-11-09  0.666473  1.028815  2.927123 -1.174118 -1.845589  ...  2.353138   
## 2021-11-10  1.866930  0.666473  1.028815  2.927123 -1.174118  ... -1.124974   
## 2021-11-11  2.135476  1.866930  0.666473  1.028815  2.927123  ...  1.154324   
## 2021-11-12  0.177859  2.135476  1.866930  0.666473  1.028815  ... -1.476543   
## 2021-11-16 -1.586049  0.177859  2.135476  1.866930  0.666473  ...  0.427599   
## 2021-11-17  0.894003 -1.586049  0.177859  2.135476  1.866930  ... -0.293498   
## 2021-11-18 -1.060913  0.894003 -1.586049  0.177859  2.135476  ... -0.708955   
## 2021-11-19 -0.477912 -1.060913  0.894003 -1.586049  0.177859  ... -1.845589   
## 
##                Lag12     Lag13     Lag14     Lag15  
## Date                                                
## 2021-11-04  1.324989  2.913743 -0.841224  0.207432  
## 2021-11-05 -0.545214  1.324989  2.913743 -0.841224  
## 2021-11-09  2.287036 -0.545214  1.324989  2.913743  
## 2021-11-10  2.353138  2.287036 -0.545214  1.324989  
## 2021-11-11 -1.124974  2.353138  2.287036 -0.545214  
## 2021-11-12  1.154324 -1.124974  2.353138  2.287036  
## 2021-11-16 -1.476543  1.154324 -1.124974  2.353138  
## 2021-11-17  0.427599 -1.476543  1.154324 -1.124974  
## 2021-11-18 -0.293498  0.427599 -1.476543  1.154324  
## 2021-11-19 -0.708955 -0.293498  0.427599 -1.476543  
## 
## [10 rows x 16 columns]

3.3: “Direction” column (+1 or -1) indicating an up/down day

3.4: Create the dependent and independent variables

## C:/Users/slaxm/AppData/Local/r-miniconda/envs/r-reticulate/python.exe:1: SettingWithCopyWarning: 
## A value is trying to be set on a copy of a slice from a DataFrame.
## Try using .loc[row_indexer,col_indexer] = value instead
## 
## See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

3.5: Create training and test sets

##                 Lag1      Lag2      Lag3      Lag4      Lag5  ...         CCI  \
## Date                                                          ...               
## 2021-11-12  2.135476  1.866930  0.666473  1.028815  2.927123  ...  174.855331   
## 2021-11-16  0.177859  2.135476  1.866930  0.666473  1.028815  ...   73.733276   
## 2021-11-17 -1.586049  0.177859  2.135476  1.866930  0.666473  ...   64.715801   
## 2021-11-18  0.894003 -1.586049  0.177859  2.135476  1.866930  ...   40.095278   
## 2021-11-19 -1.060913  0.894003 -1.586049  0.177859  2.135476  ...   10.374389   
## 
##                  ROC         DX  Open-Close  Open-Open  
## Date                                                    
## 2021-11-12  5.288082  37.445274       0.049      0.640  
## 2021-11-16  4.320187  13.717347       0.030     -0.290  
## 2021-11-17  4.180983  13.717347      -0.084     -0.275  
## 2021-11-18  2.828825   8.829975      -0.017      0.290  
## 2021-11-19  0.973841   4.015686      -0.050     -0.300  
## 
## [5 rows x 34 columns]

3.6: Create model

3.7: train the model on the training set

## LogisticRegression(max_iter=10000)

3.8: make an array of predictions on the test set

3.9: output the hit-rate and the confusion matrix for the model

## 
## Train Accuracy: 83.06%
## Test Accuracy: 73.89%
## [[65 24]
##  [23 68]]

3.10: Predict movement of stock for tomorrow.

##             y_test  y_pred
## Date                      
## 2021-11-12       1       1
## 2021-11-16      -1      -1
## 2021-11-17       1       1
## 2021-11-18      -1      -1
## 2021-11-19      -1      -1
## [-1]

Say Something

Comments

Nothing yet.

Recent Posts

Categories

About

about