Data Science Posts and Resources

Articles on Data Science

Applying Logistic Regression

A case study on applying logistic regression to stock market

Laxmi K Soni

38-Minute Read

1:Loading Stock data

1.1:Importing libraries and data

import investpy
from datetime import datetime
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix
from sklearn import metrics
from scipy import stats 
import numpy as np
import pandas as pd
import seaborn as sn
import matplotlib.pyplot as plt
import talib
import quandl
from sklearn.preprocessing import StandardScaler 
from sklearn.metrics import accuracy_score
from sklearn.preprocessing import MinMaxScaler 
import math
from sklearn.metrics import mean_squared_error
import investpy

1.2:Fetching the data

To fetch the stock data we use investpy library. This library fetchs data from investing for example:

1.2.1: Determining position based on volume and close

## C:/Users/slaxm/AppData/Local/r-miniconda/envs/r-reticulate/python.exe:1: SettingWithCopyWarning: 
## A value is trying to be set on a copy of a slice from a DataFrame.
## Try using .loc[row_indexer,col_indexer] = value instead
## 
## See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
##           Date     Open     High      Low    Close  Volume    Pos    weekday
## 670 2022-05-16  59728.0  61325.0  59652.0  61247.0  167794  Short     Monday
## 671 2022-05-17  61324.0  61850.0  61122.0  61483.0  133010  Short    Tuesday
## 672 2022-05-18  61295.0  61630.0  60828.0  61123.0  136573  Short  Wednesday
## 673 2022-05-19  61012.0  62022.0  60525.0  61846.0  176482   Long   Thursday
## 674 2022-05-20  61686.0  62285.0  61495.0  61728.0  130140  Short     Friday

1.2.2: Long positions gains

##          Date     Open     High      Low    Close  Volume   Pos    gain
## 67 2022-03-07  69400.0  71320.0  69400.0  70069.0  268267  Long   669.0
## 68 2022-03-17  67577.0  69149.0  67577.0  68625.0  164704  Long  1048.0
## 69 2022-04-11  67055.0  68538.0  67055.0  67490.0  159334  Long   435.0
## 70 2022-04-12  67595.0  69040.0  67595.0  68865.0  161335  Long  1270.0
## 71 2022-04-18  69161.0  70730.0  69161.0  70053.0  116845  Long   892.0

1.2.3: Short positions gains

##          Date     Open     High      Low    Close  Volume    Pos    gain
## 56 2022-03-29  68230.0  68230.0  65600.0  67127.0  234277  Short  1103.0
## 57 2022-04-25  66700.0  66700.0  64905.0  65279.0   21752  Short  1421.0
## 58 2022-05-02  64504.0  64504.0  62390.0  63296.0  191083  Short  1208.0
## 59 2022-05-04  63396.0  63396.0  62310.0  62487.0  121462  Short   909.0
## 60 2022-05-12  61050.0  61050.0  59009.0  59125.0  206071  Short  1925.0
print(short['weekday'].value_counts())
print(lng['weekday'].value_counts())

1.2.2: Determining Average monthly closing prices

##                Open     High      Low    Close  Volume
## Date                                                  
## 2019-10-09  45997.0  46442.0  45845.0  46042.0   66843
## 2019-10-10  46200.0  46201.0  45111.0  45431.0   70005
## 2019-10-11  45350.0  45842.0  44907.0  45176.0   82039
## 2019-10-14  45290.0  45859.0  45230.0  45823.0   54049
## 2019-10-15  45725.0  46020.0  45185.0  45232.0   67928
## ...             ...      ...      ...      ...     ...
## 2022-05-16  59728.0  61325.0  59652.0  61247.0  167794
## 2022-05-17  61324.0  61850.0  61122.0  61483.0  133010
## 2022-05-18  61295.0  61630.0  60828.0  61123.0  136573
## 2022-05-19  61012.0  62022.0  60525.0  61846.0  176482
## 2022-05-20  61686.0  62285.0  61495.0  61728.0  130140
## 
## [675 rows x 5 columns]
##               Open    High     Low   Close  Volume Currency  VolChange  \
## Date                                                                     
## 2019-10-09  45.997  46.442  45.845  46.042   66843      INR  14.568225   
## 2019-10-10  46.200  46.201  45.111  45.431   70005      INR   0.047305   
## 2019-10-11  45.350  45.842  44.907  45.176   82039      INR   0.171902   
## 2019-10-14  45.290  45.859  45.230  45.823   54049      INR  -0.341179   
## 2019-10-15  45.725  46.020  45.185  45.232   67928      INR   0.256786   
## ...            ...     ...     ...     ...     ...      ...        ...   
## 2022-05-16  59.728  61.325  59.652  61.247  167794      INR  -0.079284   
## 2022-05-17  61.324  61.850  61.122  61.483  133010      INR  -0.207302   
## 2022-05-18  61.295  61.630  60.828  61.123  136573      INR   0.026787   
## 2022-05-19  61.012  62.022  60.525  61.846  176482      INR   0.292217   
## 2022-05-20  61.686  62.285  61.495  61.728  130140      INR  -0.262588   
## 
##             CloseChange  
## Date                     
## 2019-10-09     0.000617  
## 2019-10-10    -0.013270  
## 2019-10-11    -0.005613  
## 2019-10-14     0.014322  
## 2019-10-15    -0.012897  
## ...                 ...  
## 2022-05-16     0.026446  
## 2022-05-17     0.003853  
## 2022-05-18    -0.005855  
## 2022-05-19     0.011829  
## 2022-05-20    -0.001908  
## 
## [675 rows x 8 columns]
## Date
## 2019-10-31    45.765500
## 2019-11-30    44.712571
## 2019-12-31    44.983905
## 2020-01-31    46.715435
## 2020-02-29    46.769810
## 2020-03-31    42.008682
## 2020-04-30    43.175278
## 2020-05-31    46.064714
## 2020-06-30    48.667409
## 2020-07-31    55.923957
## 2020-08-31    68.698238
## 2020-09-30    65.459455
## 2020-10-31    61.675571
## 2020-11-30    62.012955
## 2020-12-31    65.897045
## 2021-01-31    67.186000
## 2021-02-28    69.205200
## 2021-03-31    66.495957
## 2021-04-30    67.700095
## 2021-05-31    71.556381
## 2021-06-30    70.013318
## 2021-07-31    68.518000
## 2021-08-31    63.957364
## 2021-09-30    62.362182
## 2021-10-31    63.489333
## 2021-11-30    64.592091
## 2021-12-31    61.979783
## 2022-01-31    62.549050
## 2022-02-28    63.473950
## 2022-03-31    68.742739
## 2022-04-30    67.061150
## 2022-05-31    61.663533
## Freq: M, Name: Close, dtype: float64
## meanprice is  59.75931111111112

1.2.3: Technical moving averages

##   period  sma_value sma_signal   ema_value ema_signal
## 0      5   61708.20        buy  61775.2371       sell
## 1     10   61898.40       sell  61820.9143       sell
## 2     20   61917.70       sell  61776.7570       sell
## 3     50   61483.46        buy  61429.7448        buy
## 4    100   60807.36        buy  61393.7202        buy

1.2.4: Determining the z-scores

## Date
## 2022-05-16    0.154454
## 2022-05-17    0.178956
## 2022-05-18    0.141580
## 2022-05-19    0.216644
## 2022-05-20    0.204393
## Name: zscore, dtype: float64
## Date
## 2022-05-16    0.257471
## 2022-05-17   -0.142751
## 2022-05-18   -0.101756
## 2022-05-19    0.357435
## 2022-05-20   -0.175773
## Name: zscorevolume, dtype: float64

1.2.5: Determining the daily support and resistance levels

##         name       s3       s2       s1  pivot_points       r1       r2  \
## 0    Classic  59468.0  60000.0  60944.0       61476.0  62420.0  62952.0   
## 1  Fibonacci  60000.0  60564.0  60912.0       61476.0  62040.0  62388.0   
## 2  Camarilla  61481.0  61616.0  61752.0       61476.0  62022.0  62158.0   
## 3   Woodie's  59672.0  60102.0  61148.0       61578.0  62624.0  63054.0   
## 4   DeMark's      NaN      NaN  61210.0       61609.0  62686.0      NaN   
## 
##         r3  
## 0  63896.0  
## 1  62952.0  
## 2  62293.0  
## 3  64100.0  
## 4      NaN
##   technical_indicator     value      signal
## 0             RSI(14)    41.339        sell
## 1          STOCH(9,6)    95.308  overbought
## 2        STOCHRSI(14)    97.166  overbought
## 3         MACD(12,26) -1430.173        sell
## 4             ADX(14)    39.844         buy
## name            DeMark's
## s3                   NaN
## s2                   NaN
## s1                 61210
## pivot_points       61609
## r1                 62686
## r2                   NaN
## r3                   NaN
## Name: 4, dtype: object

1.2.6: Fibonnacci retracement levels

## Retracement levels for rising price
##                                         0
## 0                       {'min': [61.495]}
## 1            {'level5(61.8)': [61.79678]}
## 2                 {'level4(50)': [61.89]}
## 3  {'level3(38.2)': [61.983219999999996]}
## 4            {'level2(23.6)': [62.09856]}
## 5                      {'zero': [62.285]}
## Retracement levels for falling price
##                                         0
## 0                      {'zero': [62.285]}
## 1  {'level2(23.6)': [61.681439999999995]}
## 2            {'level3(38.2)': [61.79678]}
## 3                 {'level4(50)': [61.89]}
## 4  {'level5(61.8)': [61.983219999999996]}
## 5                       {'min': [61.495]}

As we can see, we now have a data frame with all the entries from start date to end date. We have multiple columns here and not only the closing stock price of the respective day. Let’s take a quick look at the individual columns and their meaning.

Open: That’s the share price the stock had when the markets opened that day.

Close: That’s the share price the stock had when the markets closed that day.

High: That’s the highest share price that the stock had that day.

Low: That’s the lowest share price that the stock had that day.

Volume: Amount of shares that changed hands that day.

1.3:Reading individual values

Since our data is stored in a Pandas data frame, we can use the indexing we already know, to get individual values. For example, we could only print the closing values using print (df[ 'Close' ])

Also, we can go ahead and print the closing value of a specific date that we are interested in. This is possible because the date is our index column.

print (df[ 'Close' ][ '2020-07-14' ])

But we could also use simple indexing to access certain positions.

print (df[ 'Close' ][ 5 ])
## 45.247

Here we printed the closing price of the fifth entry.

2:Graphical Visualization

Even though tables are nice and useful, we want to visualize our financial data, in order to get a better overview. We want to look at the development of the share price.

Actually plotting our share price curve with Pandas and Matplotlib is very simple. Since Pandas builds on top of Matplotlib, we can just select the column we are interested in and apply the plot method. The results are amazing. Since the date is the index of our data frame, Matplotlib uses it for the x-axis. The y-values are then our adjusted close values.

2.1:CandleStick Charts

The best way to visualize stock data is to use so-called candlestick charts . This type of chart gives us information about four different values at the same time, namely the high, the low, the open and the close value. In order to plot candlestick charts, we will need to import a function of the MPL-Finance library.

import mplfinance as fplt

We are importing the candlestick_ohlc function. Notice that there also exists a candlestick_ochl function that takes in the data in a different order. Also, for our candlestick chart, we will need a different date format provided by Matplotlib. Therefore, we need to import the respective module as well. We give it the alias mdates .

import matplotlib.dates as mdates

2.2: Preparing the data for CandleStick charts

Now in order to plot our stock data, we need to select the four columns in the right order.

df1 = df[[ 'Open' , 'High' , 'Low' , 'Close' ]]

Now, we have our columns in the right order but there is still a problem. Our date doesn’t have the right format and since it is the index, we cannot manipulate it. Therefore, we need to reset the index and then convert our datetime to a number.

df1.reset_index( inplace = True )
df1[ 'Date' ] = df1[ 'Date' ].map(mdates.date2num)
## C:/Users/slaxm/AppData/Local/r-miniconda/envs/r-reticulate/python.exe:1: SettingWithCopyWarning: 
## A value is trying to be set on a copy of a slice from a DataFrame.
## Try using .loc[row_indexer,col_indexer] = value instead
## 
## See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

For this, we use the reset_index function so that we can manipulate our Date column. Notice that we are using the inplace parameter to replace the data frame by the new one. After that, we map the date2num function of the matplotlib.dates module on all of our values. That converts our dates into numbers that we can work with.

2.3:Plotting the data

Now we can start plotting our graph. For this, we just define a subplot (because we need to pass one to our function) and call our candlestick_ohlc function.

One candlestick gives us the information about all four values of one specific day. The highest point of the stick is the high and the lowest point is the low of that day. The colored area is the difference between the open and the close price. If the stick is green, the close value is at the top and the open value at the bottom, since the close must be higher than the open. If it is red, it is the other way around.

***2.4:Analysis and Statistics ***

Now let’s get a little bit deeper into the numbers here and away from the visual. From our data we can derive some statistical values that will help us to analyze it.

PERCENTAGE CHANGE

One value that we can calculate is the percentage change of that day. This means by how many percent the share price increased or decreased that day.

The calculation is quite simple. We create a new column with the name PCT_Change and the values are just the difference of the closing and opening values divided by the opening values. Since the open value is the beginning value of that day, we take it as a basis. We could also multiply the result by 100 to get the actual percentage.

##        PCT_Change
## count  675.000000
## mean     0.000166
## std      0.018259
## min     -0.107200
## 25%     -0.006964
## 50%      0.001150
## 75%      0.008420
## max      0.062274
##              Close
## Date              
## 2022-05-20  61.728

*** HIGH LOW PERCENTAGE ***

Another interesting statistic is the high low percentage. Here we just calculate the difference between the highest and the lowest value and divide it by the closing value.

By doing that we can get a feeling of how volatile the stock is.

##               Open    High     Low   Close  Volume Currency  VolChange  \
## Date                                                                     
## 2022-05-16  59.728  61.325  59.652  61.247  167794      INR  -0.079284   
## 2022-05-17  61.324  61.850  61.122  61.483  133010      INR  -0.207302   
## 2022-05-18  61.295  61.630  60.828  61.123  136573      INR   0.026787   
## 2022-05-19  61.012  62.022  60.525  61.846  176482      INR   0.292217   
## 2022-05-20  61.686  62.285  61.495  61.728  130140      INR  -0.262588   
## 
##             CloseChange  usdinr  usdsilver  Open-Close  Open-Open    zscore  \
## Date                                                                          
## 2022-05-16     0.026446  77.811     21.551       0.059      0.328  0.154454   
## 2022-05-17     0.003853  77.454     21.750       0.077      1.596  0.178956   
## 2022-05-18    -0.005855  77.800     21.544      -0.188     -0.029  0.141580   
## 2022-05-19     0.011829  77.400     21.908      -0.111     -0.283  0.216644   
## 2022-05-20    -0.001908  77.850     21.674      -0.160      0.674  0.204393   
## 
##             zscorevolume  PCT_Change    HL_PCT  
## Date                                            
## 2022-05-16      0.257471    0.025432  0.027316  
## 2022-05-17     -0.142751    0.002593  0.011841  
## 2022-05-18     -0.101756   -0.002806  0.013121  
## 2022-05-19      0.357435    0.013669  0.024205  
## 2022-05-20     -0.175773    0.000681  0.012798

These statistical values can be used with many others to get a lot of valuable information about specific stocks. This improves the decision making

*** MOVING AVERAGE ***

we are going to derive the different moving averages . It is the arithmetic mean of all the values of the past n days. Of course this is not the only key statistic that we can derive, but it is the one we are going to use now. We can play around with other functions as well.

What we are going to do with this value is to include it into our data frame and to compare it with the share price of that day.

For this, we will first need to create a new column. Pandas does this automatically when we assign values to a column name. This means that we don’t have to explicitly define that we are creating a new column.

##              Close  5d_ma  20d_ma  50d_ma  100d_ma  200d_ma  5d_ema  20d_ema  \
## Date                                                                           
## 2022-05-12  59.125  61.20   64.63   66.88    65.01    64.12   60.91    63.73   
## 2022-05-13  59.669  60.55   64.16   66.71    64.98    64.08   60.49    63.34   
## 2022-05-16  61.247  60.42   63.72   66.55    64.97    64.05   60.75    63.14   
## 2022-05-17  61.483  60.53   63.35   66.38    64.96    64.03   60.99    62.98   
## 2022-05-18  61.123  60.53   62.98   66.17    64.94    64.02   61.04    62.81   
## 2022-05-19  61.846  61.07   62.71   66.02    64.93    64.01   61.31    62.71   
## 2022-05-20  61.728  61.49   62.46   65.84    64.93    64.01   61.45    62.62   
## 
##             50d_ema  100d_ema  200d_ema  
## Date                                     
## 2022-05-12    65.25     65.24     65.00  
## 2022-05-13    65.03     65.13     64.95  
## 2022-05-16    64.88     65.05     64.91  
## 2022-05-17    64.75     64.98     64.88  
## 2022-05-18    64.60     64.91     64.84  
## 2022-05-19    64.50     64.85     64.81  
## 2022-05-20    64.39     64.78     64.78

Here we define a three new columns with the name 20d_ma, 50d_ma, 100d_ma,200d_ma . We now fill this column with the mean values of every n entries. The rolling function stacks a specific amount of entries, in order to make a statistical calculation possible. The window parameter is the one which defines how many entries we are going to stack. But there is also the min_periods parameter. This one defines how many entries we need to have as a minimum in order to perform the calculation. This is relevant because the first entries of our data frame won’t have a n entries previous to them. By setting this value to zero we start the calculations already with the first number, even if there is not a single previous value. This has the effect that the first value will be just the first number, the second one will be the mean of the first two numbers and so on, until we get to a b values.

By using the mean function, we are obviously calculating the arithmetic mean. However, we can use a bunch of other functions like max, min or median if we like to.

*** Standard Deviation ***

The variability of the closing stock prices determinies how vo widely prices are dispersed from the average price. If the prices are trading in narrow trading range the standard deviation will return a low value that indicates low volatility. If the prices are trading in wide trading range the standard deviation will return high value that indicates high volatility.

## Date
## 2022-05-12    1.327123
## 2022-05-13    1.434649
## 2022-05-16    1.275987
## 2022-05-17    1.002367
## 2022-05-18    0.900315
## 2022-05-19    1.003591
## 2022-05-20    1.061102
## Name: Std_dev, dtype: float64

*** Relative Strength Index ***

The relative strength index is a indicator of mementum used in technical analysis that measures the magnitude of current price changes to know overbought or oversold conditions in the price of a stock or other asset. If RSI is above 70 then it is overbought. If RSI is below 30 then it is oversold condition.

##                   RSI
## Date                 
## 2022-05-16  38.761402
## 2022-05-17  40.859801
## 2022-05-18  38.590529
## 2022-05-19  45.437142
## 2022-05-20  44.525662

*** Average True range ***

##                     ATR     20dayEMA    ATRdiff
## Date                                           
## 2022-05-16  1386.618924  1310.991339  75.627584
## 2022-05-17  1339.574715  1313.713566  25.861149
## 2022-05-18  1301.176521  1312.519561 -11.343040
## 2022-05-19  1315.163912  1312.771404   2.392508
## 2022-05-20  1277.652204  1309.426719 -31.774514

*** Wiliams %R ***

Williams %R, or just %R, is a technical analysis oscillator showing the current closing price in relation to the high and low of the past N days.The oscillator is on a negative scale, from −100 (lowest) up to 0 (highest). A value of −100 means the close today was the lowest low of the past N days, and 0 means today’s close was the highest high of the past N days.

##             Williams %R
## Date                   
## 2022-05-12   -97.908402
## 2022-05-13   -81.501251
## 2022-05-16   -43.609654
## 2022-05-17   -35.616740
## 2022-05-18   -32.015915
## 2022-05-19    -5.083767
## 2022-05-20   -14.953020

Readings below -80 represent oversold territory and readings above -20 represent overbought.

*** ADX ***

ADX is used to quantify trend strength. ADX calculations are based on a moving average of price range expansion over a given period of time. The average directional index (ADX) is used to determine when the price is trending strongly.

0-25: Absent or Weak Trend

25-50: Strong Trend

50-75: Very Strong Trend

75-100: Extremely Strong Trend

##                   ADX
## Date                 
## 2022-05-12  45.635258
## 2022-05-13  48.916370
## 2022-05-16  45.346117
## 2022-05-17  40.582942
## 2022-05-18  37.243538
## 2022-05-19  32.970814
## 2022-05-20  28.360043
##                    CCI
## Date                  
## 2022-05-12 -176.743800
## 2022-05-13 -158.874327
## 2022-05-16  -78.345207
## 2022-05-17  -34.840581
## 2022-05-18  -41.087559
## 2022-05-19  -17.083135
## 2022-05-20   15.230386
##                  ROC
## Date                
## 2022-05-12 -7.860493
## 2022-05-13 -7.800114
## 2022-05-16 -3.237171
## 2022-05-17 -3.012951
## 2022-05-18 -2.182854
## 2022-05-19 -1.366761
## 2022-05-20 -1.875755

*** MACD ***

Moving Average Convergence Divergence (MACD) is a trend-following momentum indicator that shows the relationship between two moving averages of a security’s price. The MACD is calculated by subtracting the 26-period Exponential Moving Average (EMA) from the 12-period EMA.

##             MACD_IND
## Date                
## 2022-05-16 -0.319088
## 2022-05-17 -0.176713
## 2022-05-18 -0.087045
## 2022-05-19  0.034198
## 2022-05-20  0.115129
## Date
## 2019-10-09           NaN
## 2019-10-10           NaN
## 2019-10-11           NaN
## 2019-10-14           NaN
## 2019-10-15           NaN
##                  ...    
## 2022-05-16     97.777133
## 2022-05-17    110.744047
## 2022-05-18    119.437622
## 2022-05-19    124.929270
## 2022-05-20    129.674345
## Name: STC, Length: 675, dtype: float64

*** Bollinger Bands ***

Bollinger Bands are a type of statistical chart characterizing the prices and volatility over time of a financial instrument or commodity.

## 65910.0 62460.0 59010.0

In case we choose another value than zero for our min_periods parameter, we will end up with a couple of NaN-Values . These are not a number values and they are useless. Therefore, we would want to delete the entries that have such values.

We do this by using the dropna function. If we would have had any entries with NaN values in any column, they would now have been deleted

3: Predicting the movement of stock

To predict the movement of the stock we use 5 lag returns as the dependent variables. The first leg is return yesterday, leg2 is return day before yesterday and so on. The dependent variable is whether the prices went up or down on that day. Other variables include the technical indicators which along with 5 lag returns are used to predict the movement of stock using logistic regression.

3.1: Creating lag returns

3.2: Creating returns dataframe

3.2: create the lagged percentage returns columns

##                Today      Lag1      Lag2      Lag3      Lag4      Lag5  \
## Date                                                                     
## 2022-05-09 -1.638901  0.326938  0.345672 -1.429180  0.153248 -2.195714   
## 2022-05-10 -1.478740 -1.638901  0.326938  0.345672 -1.429180  0.153248   
## 2022-05-11  0.236213 -1.478740 -1.638901  0.326938  0.345672 -1.429180   
## 2022-05-12 -3.241908  0.236213 -1.478740 -1.638901  0.326938  0.345672   
## 2022-05-13  0.920085 -3.241908  0.236213 -1.478740 -1.638901  0.326938   
## 2022-05-16  2.644589  0.920085 -3.241908  0.236213 -1.478740 -1.638901   
## 2022-05-17  0.385325  2.644589  0.920085 -3.241908  0.236213 -1.478740   
## 2022-05-18 -0.585528  0.385325  2.644589  0.920085 -3.241908  0.236213   
## 2022-05-19  1.182861 -0.585528  0.385325  2.644589  0.920085 -3.241908   
## 2022-05-20 -0.190796  1.182861 -0.585528  0.385325  2.644589  0.920085   
## 
##                 Lag6      Lag7      Lag8      Lag9     Lag10     Lag11  \
## Date                                                                     
## 2022-05-09  0.853995 -1.040960 -0.606990 -0.059744 -2.155373 -0.898666   
## 2022-05-10 -2.195714  0.853995 -1.040960 -0.606990 -0.059744 -2.155373   
## 2022-05-11  0.153248 -2.195714  0.853995 -1.040960 -0.606990 -0.059744   
## 2022-05-12 -1.429180  0.153248 -2.195714  0.853995 -1.040960 -0.606990   
## 2022-05-13  0.345672 -1.429180  0.153248 -2.195714  0.853995 -1.040960   
## 2022-05-16  0.326938  0.345672 -1.429180  0.153248 -2.195714  0.853995   
## 2022-05-17 -1.638901  0.326938  0.345672 -1.429180  0.153248 -2.195714   
## 2022-05-18 -1.478740 -1.638901  0.326938  0.345672 -1.429180  0.153248   
## 2022-05-19  0.236213 -1.478740 -1.638901  0.326938  0.345672 -1.429180   
## 2022-05-20 -3.241908  0.236213 -1.478740 -1.638901  0.326938  0.345672   
## 
##                Lag12     Lag13     Lag14     Lag15  
## Date                                                
## 2022-05-09 -1.722577 -0.527118 -1.695859  1.386497  
## 2022-05-10 -0.898666 -1.722577 -0.527118 -1.695859  
## 2022-05-11 -2.155373 -0.898666 -1.722577 -0.527118  
## 2022-05-12 -0.059744 -2.155373 -0.898666 -1.722577  
## 2022-05-13 -0.606990 -0.059744 -2.155373 -0.898666  
## 2022-05-16 -1.040960 -0.606990 -0.059744 -2.155373  
## 2022-05-17  0.853995 -1.040960 -0.606990 -0.059744  
## 2022-05-18 -2.195714  0.853995 -1.040960 -0.606990  
## 2022-05-19  0.153248 -2.195714  0.853995 -1.040960  
## 2022-05-20 -1.429180  0.153248 -2.195714  0.853995

3.3: “Direction” column (+1 or -1) indicating an up/down day

***3.4: Create the dependent and independent variables ***

##                 Lag1      Lag2      Lag3      Lag4      Lag5      Lag6  \
## Date                                                                     
## 2022-05-16  0.920085 -3.241908  0.236213 -1.478740 -1.638901  0.326938   
## 2022-05-17  2.644589  0.920085 -3.241908  0.236213 -1.478740 -1.638901   
## 2022-05-18  0.385325  2.644589  0.920085 -3.241908  0.236213 -1.478740   
## 2022-05-19 -0.585528  0.385325  2.644589  0.920085 -3.241908  0.236213   
## 2022-05-20  1.182861 -0.585528  0.385325  2.644589  0.920085 -3.241908   
## 
##                 Lag7      Lag8      Lag9     Lag10  usdinr  usdsilver  20d_ma  \
## Date                                                                            
## 2022-05-16  0.345672 -1.429180  0.153248 -2.195714  77.811     21.551   63.72   
## 2022-05-17  0.326938  0.345672 -1.429180  0.153248  77.454     21.750   63.35   
## 2022-05-18 -1.638901  0.326938  0.345672 -1.429180  77.800     21.544   62.98   
## 2022-05-19 -1.478740 -1.638901  0.326938  0.345672  77.400     21.908   62.71   
## 2022-05-20  0.236213 -1.478740 -1.638901  0.326938  77.850     21.674   62.46   
## 
##             20d_ema  50d_ma  50d_ema  100d_ma  100d_ema  200d_ma  200d_ema  \
## Date                                                                         
## 2022-05-16    63.14   66.55    64.88    64.97     65.05    64.05     64.91   
## 2022-05-17    62.98   66.38    64.75    64.96     64.98    64.03     64.88   
## 2022-05-18    62.81   66.17    64.60    64.94     64.91    64.02     64.84   
## 2022-05-19    62.71   66.02    64.50    64.93     64.85    64.01     64.81   
## 2022-05-20    62.62   65.84    64.39    64.93     64.78    64.01     64.78   
## 
##             250d_ma  250d_ema   Std_dev        RSI  Williams %R  MACD_IND  \
## Date                                                                        
## 2022-05-16    65.09     65.09  1.275987  38.761402   -43.609654 -0.319088   
## 2022-05-17    65.05     65.05  1.002367  40.859801   -35.616740 -0.176713   
## 2022-05-18    65.00     65.00  0.900315  38.590529   -32.015915 -0.087045   
## 2022-05-19    64.96     64.96  1.003591  45.437142    -5.083767  0.034198   
## 2022-05-20    64.92     64.92  1.061102  44.525662   -14.953020  0.115129   
## 
##             VolChange  CloseChange    zscore        CCI       ROC         DX  \
## Date                                                                           
## 2022-05-16  -0.079284     0.026446  0.154454 -78.345207 -3.237171  27.954347   
## 2022-05-17  -0.207302     0.003853  0.178956 -34.840581 -3.012951  21.270850   
## 2022-05-18   0.026787    -0.005855  0.141580 -41.087559 -2.182854  23.675143   
## 2022-05-19   0.292217     0.011829  0.216644 -17.083135 -1.366761  18.479739   
## 2022-05-20  -0.262588    -0.001908  0.204393  15.230386 -1.875755  14.989544   
## 
##                   SAR  Open-Close  Open-Open  
## Date                                          
## 2022-05-16  61.671093       0.059      0.328  
## 2022-05-17  58.560000       0.077      1.596  
## 2022-05-18  58.625800      -0.188     -0.029  
## 2022-05-19  58.690284      -0.111     -0.283  
## 2022-05-20  58.823553      -0.160      0.674

3.5: Create training and test sets

##                 Lag1      Lag2      Lag3      Lag4      Lag5      Lag6  \
## Date                                                                     
## 2022-05-16  0.920085 -3.241908  0.236213 -1.478740 -1.638901  0.326938   
## 2022-05-17  2.644589  0.920085 -3.241908  0.236213 -1.478740 -1.638901   
## 2022-05-18  0.385325  2.644589  0.920085 -3.241908  0.236213 -1.478740   
## 2022-05-19 -0.585528  0.385325  2.644589  0.920085 -3.241908  0.236213   
## 2022-05-20  1.182861 -0.585528  0.385325  2.644589  0.920085 -3.241908   
## 
##                 Lag7      Lag8      Lag9     Lag10  usdinr  usdsilver  20d_ma  \
## Date                                                                            
## 2022-05-16  0.345672 -1.429180  0.153248 -2.195714  77.811     21.551   63.72   
## 2022-05-17  0.326938  0.345672 -1.429180  0.153248  77.454     21.750   63.35   
## 2022-05-18 -1.638901  0.326938  0.345672 -1.429180  77.800     21.544   62.98   
## 2022-05-19 -1.478740 -1.638901  0.326938  0.345672  77.400     21.908   62.71   
## 2022-05-20  0.236213 -1.478740 -1.638901  0.326938  77.850     21.674   62.46   
## 
##             20d_ema  50d_ma  50d_ema  100d_ma  100d_ema  200d_ma  200d_ema  \
## Date                                                                         
## 2022-05-16    63.14   66.55    64.88    64.97     65.05    64.05     64.91   
## 2022-05-17    62.98   66.38    64.75    64.96     64.98    64.03     64.88   
## 2022-05-18    62.81   66.17    64.60    64.94     64.91    64.02     64.84   
## 2022-05-19    62.71   66.02    64.50    64.93     64.85    64.01     64.81   
## 2022-05-20    62.62   65.84    64.39    64.93     64.78    64.01     64.78   
## 
##             250d_ma  250d_ema   Std_dev        RSI  Williams %R  MACD_IND  \
## Date                                                                        
## 2022-05-16    65.09     65.09  1.275987  38.761402   -43.609654 -0.319088   
## 2022-05-17    65.05     65.05  1.002367  40.859801   -35.616740 -0.176713   
## 2022-05-18    65.00     65.00  0.900315  38.590529   -32.015915 -0.087045   
## 2022-05-19    64.96     64.96  1.003591  45.437142    -5.083767  0.034198   
## 2022-05-20    64.92     64.92  1.061102  44.525662   -14.953020  0.115129   
## 
##             VolChange  CloseChange    zscore        CCI       ROC         DX  \
## Date                                                                           
## 2022-05-16  -0.079284     0.026446  0.154454 -78.345207 -3.237171  27.954347   
## 2022-05-17  -0.207302     0.003853  0.178956 -34.840581 -3.012951  21.270850   
## 2022-05-18   0.026787    -0.005855  0.141580 -41.087559 -2.182854  23.675143   
## 2022-05-19   0.292217     0.011829  0.216644 -17.083135 -1.366761  18.479739   
## 2022-05-20  -0.262588    -0.001908  0.204393  15.230386 -1.875755  14.989544   
## 
##                   SAR  Open-Close  Open-Open  
## Date                                          
## 2022-05-16  61.671093       0.059      0.328  
## 2022-05-17  58.560000       0.077      1.596  
## 2022-05-18  58.625800      -0.188     -0.029  
## 2022-05-19  58.690284      -0.111     -0.283  
## 2022-05-20  58.823553      -0.160      0.674

***3.6: Create model ***

3.7: train the model on the training set

## LogisticRegression(max_iter=1000000)

3.8: make an array of predictions on the test set

3.9: output the hit-rate and the confusion matrix for the model

## 
## Train Accuracy: 94.68%
## Test Accuracy: 80.39%
## [[100   3]
##  [ 67 187]]

***3.10: Predict movement of stock for tomorrow. ***

##             y_test  y_pred
## Date                      
## 2022-05-16       1       1
## 2022-05-17       1       1
## 2022-05-18      -1      -1
## 2022-05-19       1       1
## 2022-05-20      -1       1
## Hourly Technical Indicators:
##         name       s3       s2       s1  pivot_points       r1       r2  \
## 0    Classic  61301.0  61407.0  61539.0       61645.0  61777.0  61883.0   
## 1  Fibonacci  61407.0  61498.0  61554.0       61645.0  61736.0  61792.0   
## 2  Camarilla  61605.0  61626.0  61648.0       61645.0  61692.0  61714.0   
## 3   Woodie's  61313.0  61413.0  61551.0       61651.0  61789.0  61889.0   
## 4   DeMark's      NaN      NaN  61472.0       61612.0  61710.0      NaN   
## 
##         r3  
## 0  62015.0  
## 1  61883.0  
## 2  61735.0  
## 3  62027.0  
## 4      NaN
##   technical_indicator   value      signal
## 0             RSI(14)  51.430     neutral
## 1          STOCH(9,6)  98.944  overbought
## 2        STOCHRSI(14)  13.399    oversold
## 3         MACD(12,26)  72.950         buy
## 4             ADX(14)  35.890     neutral
##   period  sma_value sma_signal   ema_value ema_signal
## 0      5   61708.20        buy  61775.2371       sell
## 1     10   61898.40       sell  61820.9143       sell
## 2     20   61917.70       sell  61776.7570       sell
## 3     50   61483.46        buy  61429.7448        buy
## 4    100   60807.36        buy  61393.7202        buy
## Daily Technical Indicators:
##         name       s3       s2       s1  pivot_points       r1       r2  \
## 0    Classic  59468.0  60000.0  60944.0       61476.0  62420.0  62952.0   
## 1  Fibonacci  60000.0  60564.0  60912.0       61476.0  62040.0  62388.0   
## 2  Camarilla  61481.0  61616.0  61752.0       61476.0  62022.0  62158.0   
## 3   Woodie's  59672.0  60102.0  61148.0       61578.0  62624.0  63054.0   
## 4   DeMark's      NaN      NaN  61210.0       61609.0  62686.0      NaN   
## 
##         r3  
## 0  63896.0  
## 1  62952.0  
## 2  62293.0  
## 3  64100.0  
## 4      NaN
##   technical_indicator     value      signal
## 0             RSI(14)    41.339        sell
## 1          STOCH(9,6)    95.308  overbought
## 2        STOCHRSI(14)    97.166  overbought
## 3         MACD(12,26) -1430.173        sell
## 4             ADX(14)    39.844         buy
##   period  sma_value sma_signal   ema_value ema_signal
## 0      5   61500.20        buy  61402.4908        buy
## 1     10   61024.00        buy  61587.0327        buy
## 2     20   62463.80       sell  62615.8889       sell
## 3     50   65843.96       sell  64243.0909       sell
## 4    100   64931.08       sell  64701.1812       sell
## Lag1            1.182861
## Lag2           -0.585528
## Lag3            0.385325
## Lag4            2.644589
## Lag5            0.920085
## Lag6           -3.241908
## Lag7            0.236213
## Lag8           -1.478740
## Lag9           -1.638901
## Lag10           0.326938
## usdinr         77.850000
## usdsilver      21.674000
## 20d_ma         62.460000
## 20d_ema        62.620000
## 50d_ma         65.840000
## 50d_ema        64.390000
## 100d_ma        64.930000
## 100d_ema       64.780000
## 200d_ma        64.010000
## 200d_ema       64.780000
## 250d_ma        64.920000
## 250d_ema       64.920000
## Std_dev         1.061102
## RSI            44.525662
## Williams %R   -14.953020
## MACD_IND        0.115129
## VolChange      -0.262588
## CloseChange    -0.001908
## zscore          0.204393
## CCI            15.230386
## ROC            -1.875755
## DX             14.989544
## SAR            58.823553
## Open-Close     -0.160000
## Open-Open       0.674000
## Name: 2022-05-20 00:00:00, dtype: float64
## Bullish at daily: Buy at  62040.0
## Bullish hourly: Buy at  61736.0
## Pridiction: Buy at  62040.0

pd.set_option('display.max_columns', None)

dfusd2 = df[['Open','High','Low','Close','Volume']]
diffdf = dfusd2.diff()
dfusd2['Pos'] = np.where((dfusd2['Volume'] > dfusd2['Volume'].shift(1)) & ((dfusd2['Close'] > dfusd2['Close'].shift(1))),"Long","Short")



short = dfusd2[dfusd2['Open'] == dfusd2['High']]
lng = dfusd2[dfusd2['Open'] == dfusd2['Low']]


short['gain'] = short['High'] - short['Close']
## C:/Users/slaxm/AppData/Local/r-miniconda/envs/r-reticulate/python.exe:1: SettingWithCopyWarning: 
## A value is trying to be set on a copy of a slice from a DataFrame.
## Try using .loc[row_indexer,col_indexer] = value instead
## 
## See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
lng['gain'] = lng['Close'] - lng['Low']

dfusd2 = dfusd2.reset_index()

dfusd2['weekday'] = dfusd2['Date'].dt.day_name()

print(dfusd2.tail(5))
##            Date    Open    High     Low   Close  Volume    Pos    weekday
## 2127 2021-11-01  23.990  24.135  23.905  24.091      85   Long     Monday
## 2128 2021-11-02  24.120  24.120  23.475  23.524      69  Short    Tuesday
## 2129 2021-11-03  23.565  23.655  23.070  23.251     373  Short  Wednesday
## 2130 2021-11-04  23.635  24.105  23.530  23.930     567   Long   Thursday
## 2131 2021-11-05  23.840  24.260  23.840  24.176     139  Short     Friday
short = short.reset_index()
lng = lng.reset_index()

short['weekday'] = short['Date'].dt.day_name()
lng['weekday'] = lng['Date'].dt.day_name()

1.2.2: Determining Average monthly closing prices

##               Open    High     Low   Close  Volume
## Date                                              
## 2015-01-01  15.695  15.903  15.695  15.813       0
## 2015-01-02  15.790  15.815  15.535  15.734      13
## 2015-01-04  15.782  15.790  15.633  15.740       0
## 2015-01-05  15.795  16.260  15.630  16.213   47134
## 2015-01-06  16.160  16.740  16.115  16.637   43728
## ...            ...     ...     ...     ...     ...
## 2021-11-01  23.990  24.135  23.905  24.091      85
## 2021-11-02  24.120  24.120  23.475  23.524      69
## 2021-11-03  23.565  23.655  23.070  23.251     373
## 2021-11-04  23.635  24.105  23.530  23.930     567
## 2021-11-05  23.840  24.260  23.840  24.176     139
## 
## [2132 rows x 5 columns]
##               Open    High     Low   Close  Volume Currency  VolChange  \
## Date                                                                     
## 2015-01-01  15.695  15.903  15.695  15.813       0      USD   9.110300   
## 2015-01-02  15.790  15.815  15.535  15.734      13      USD   9.110300   
## 2015-01-04  15.782  15.790  15.633  15.740       0      USD  -1.000000   
## 2015-01-05  15.795  16.260  15.630  16.213   47134      USD   9.110300   
## 2015-01-06  16.160  16.740  16.115  16.637   43728      USD  -0.072262   
## ...            ...     ...     ...     ...     ...      ...        ...   
## 2021-11-01  23.990  24.135  23.905  24.091      85      USD   9.110300   
## 2021-11-02  24.120  24.120  23.475  23.524      69      USD  -0.188235   
## 2021-11-03  23.565  23.655  23.070  23.251     373      USD   4.405797   
## 2021-11-04  23.635  24.105  23.530  23.930     567      USD   0.520107   
## 2021-11-05  23.840  24.260  23.840  24.176     139      USD  -0.754850   
## 
##             CloseChange  
## Date                     
## 2015-01-01     0.000325  
## 2015-01-02    -0.004996  
## 2015-01-04     0.000381  
## 2015-01-05     0.030051  
## 2015-01-06     0.026152  
## ...                 ...  
## 2021-11-01     0.010529  
## 2021-11-02    -0.023536  
## 2021-11-03    -0.011605  
## 2021-11-04     0.029203  
## 2021-11-05     0.010280  
## 
## [2132 rows x 8 columns]
## Date
## 2015-01-31    17.175923
## 2015-02-28    16.798125
## 2015-03-31    16.242333
## 2015-04-30    16.372692
## 2015-05-31    16.823577
##                 ...    
## 2021-07-31    25.754880
## 2021-08-31    23.983519
## 2021-09-30    23.283500
## 2021-10-31    23.424038
## 2021-11-30    23.794400
## Freq: M, Name: Close, Length: 83, dtype: float64
## meanprice is  18.136616322701688

1.2.3: Technical moving averages

##   period  sma_value sma_signal   ema_value ema_signal
## 0      5   61500.20        buy  61402.4908        buy
## 1     10   61024.00        buy  61587.0327        buy
## 2     20   62463.80       sell  62615.8889       sell
## 3     50   65843.96       sell  64243.0909       sell
## 4    100   64931.08       sell  64701.1812       sell

1.2.4: Determining the z-scores

## Date
## 2021-11-01    1.563054
## 2021-11-02    1.414214
## 2021-11-03    1.342550
## 2021-11-04    1.520790
## 2021-11-05    1.585367
## Name: zscore, dtype: float64
## Date
## 2021-11-01   -0.652813
## 2021-11-02   -0.653255
## 2021-11-03   -0.644856
## 2021-11-04   -0.639497
## 2021-11-05   -0.651321
## Name: zscorevolume, dtype: float64

1.2.5: Determining the daily support and resistance levels

##                    PP         R1         S1         R2         S2         R3  \
## Date                                                                           
## 2021-11-01  24.043667  24.182333  23.952333  24.273667  23.813667  24.412333   
## 2021-11-02  23.706333  23.937667  23.292667  24.351333  23.061333  24.582667   
## 2021-11-03  23.325333  23.580667  22.995667  23.910333  22.740333  24.165667   
## 2021-11-04  23.855000  24.180000  23.605000  24.430000  23.280000  24.755000   
## 2021-11-05  24.092000  24.344000  23.924000  24.512000  23.672000  24.764000   
## 
##                    S3  
## Date                   
## 2021-11-01  23.722333  
## 2021-11-02  22.647667  
## 2021-11-03  22.410667  
## 2021-11-04  23.030000  
## 2021-11-05  23.504000

1.2.6: Fibonnacci retracement levels

## Retracement levels for rising price
##                                         0
## 0                        {'min': [23.84]}
## 1            {'level5(61.8)': [24.00044]}
## 2                 {'level4(50)': [24.05]}
## 3            {'level3(38.2)': [24.09956]}
## 4  {'level2(23.6)': [24.160880000000002]}
## 5                       {'zero': [24.26]}
## Retracement levels for falling price
##                               0
## 0             {'zero': [24.26]}
## 1  {'level2(23.6)': [23.93912]}
## 2  {'level3(38.2)': [24.00044]}
## 3       {'level4(50)': [24.05]}
## 4  {'level5(61.8)': [24.09956]}
## 5              {'min': [23.84]}

As we can see, we now have a data frame with all the entries from start date to end date. We have multiple columns here and not only the closing stock price of the respective day. Let’s take a quick look at the individual columns and their meaning.

Open: That’s the share price the stock had when the markets opened that day.

Close: That’s the share price the stock had when the markets closed that day.

High: That’s the highest share price that the stock had that day.

Low: That’s the lowest share price that the stock had that day.

Volume: Amount of shares that changed hands that day.

1.3:Reading individual values

Since our data is stored in a Pandas data frame, we can use the indexing we already know, to get individual values. For example, we could only print the closing values using print (df[ 'Close' ])

Also, we can go ahead and print the closing value of a specific date that we are interested in. This is possible because the date is our index column.

print (df[ 'Close' ][ '2020-07-14' ])
## 19.53

But we could also use simple indexing to access certain positions.

print (df[ 'Close' ][ 5 ])
## 16.544

Here we printed the closing price of the fifth entry.

2:Graphical Visualization

Even though tables are nice and useful, we want to visualize our financial data, in order to get a better overview. We want to look at the development of the share price.

Actually plotting our share price curve with Pandas and Matplotlib is very simple. Since Pandas builds on top of Matplotlib, we can just select the column we are interested in and apply the plot method. The results are amazing. Since the date is the index of our data frame, Matplotlib uses it for the x-axis. The y-values are then our adjusted close values.

2.1:CandleStick Charts

The best way to visualize stock data is to use so-called candlestick charts . This type of chart gives us information about four different values at the same time, namely the high, the low, the open and the close value. In order to plot candlestick charts, we will need to import a function of the MPL-Finance library.

import mplfinance as fplt

We are importing the candlestick_ohlc function. Notice that there also exists a candlestick_ochl function that takes in the data in a different order. Also, for our candlestick chart, we will need a different date format provided by Matplotlib. Therefore, we need to import the respective module as well. We give it the alias mdates .

import matplotlib.dates as mdates

2.2: Preparing the data for CandleStick charts

Now in order to plot our stock data, we need to select the four columns in the right order.

df1 = df[[ 'Open' , 'High' , 'Low' , 'Close' ]]

Now, we have our columns in the right order but there is still a problem. Our date doesn’t have the right format and since it is the index, we cannot manipulate it. Therefore, we need to reset the index and then convert our datetime to a number.

df1.reset_index( inplace = True )
df1[ 'Date' ] = df1[ 'Date' ].map(mdates.date2num)

For this, we use the reset_index function so that we can manipulate our Date column. Notice that we are using the inplace parameter to replace the data frame by the new one. After that, we map the date2num function of the matplotlib.dates module on all of our values. That converts our dates into numbers that we can work with.

2.3:Plotting the data

Now we can start plotting our graph. For this, we just define a subplot (because we need to pass one to our function) and call our candlestick_ohlc function.

One candlestick gives us the information about all four values of one specific day. The highest point of the stick is the high and the lowest point is the low of that day. The colored area is the difference between the open and the close price. If the stick is green, the close value is at the top and the open value at the bottom, since the close must be higher than the open. If it is red, it is the other way around.

***2.4:Analysis and Statistics ***

Now let’s get a little bit deeper into the numbers here and away from the visual. From our data we can derive some statistical values that will help us to analyze it.

PERCENTAGE CHANGE

One value that we can calculate is the percentage change of that day. This means by how many percent the share price increased or decreased that day.

The calculation is quite simple. We create a new column with the name PCT_Change and the values are just the difference of the closing and opening values divided by the opening values. Since the open value is the beginning value of that day, we take it as a basis. We could also multiply the result by 100 to get the actual percentage.

##         PCT_Change
## count  2132.000000
## mean     -0.000241
## std       0.014965
## min      -0.151258
## 25%      -0.005561
## 50%       0.000021
## 75%       0.005694
## max       0.079603
##              Close
## Date              
## 2021-11-05  24.176

*** HIGH LOW PERCENTAGE ***

Another interesting statistic is the high low percentage. Here we just calculate the difference between the highest and the lowest value and divide it by the closing value.

By doing that we can get a feeling of how volatile the stock is.

##               Open    High     Low   Close  Volume  ... Open-Open    zscore  \
## Date                                                ...                       
## 2021-11-01  23.990  24.135  23.905  24.091      85  ...     0.040  1.563054   
## 2021-11-02  24.120  24.120  23.475  23.524      69  ...     0.130  1.414214   
## 2021-11-03  23.565  23.655  23.070  23.251     373  ...    -0.555  1.342550   
## 2021-11-04  23.635  24.105  23.530  23.930     567  ...     0.070  1.520790   
## 2021-11-05  23.840  24.260  23.840  24.176     139  ...     0.205  1.585367   
## 
##             zscorevolume  PCT_Change    HL_PCT  
## Date                                            
## 2021-11-01     -0.652813    0.004210  0.009547  
## 2021-11-02     -0.653255   -0.024710  0.027419  
## 2021-11-03     -0.644856   -0.013325  0.025160  
## 2021-11-04     -0.639497    0.012481  0.024028  
## 2021-11-05     -0.651321    0.014094  0.017373  
## 
## [5 rows x 15 columns]

These statistical values can be used with many others to get a lot of valuable information about specific stocks. This improves the decision making

*** MOVING AVERAGE ***

we are going to derive the different moving averages . It is the arithmetic mean of all the values of the past n days. Of course this is not the only key statistic that we can derive, but it is the one we are going to use now. We can play around with other functions as well.

What we are going to do with this value is to include it into our data frame and to compare it with the share price of that day.

For this, we will first need to create a new column. Pandas does this automatically when we assign values to a column name. This means that we don’t have to explicitly define that we are creating a new column.

##              Close  5d_ma  20d_ma  50d_ma  100d_ma  ...  5d_ema  20d_ema  \
## Date                                                ...                    
## 2021-10-29  23.966  24.20   23.61   23.33    24.03  ...   24.12    23.75   
## 2021-10-31  23.840  24.04   23.67   23.33    24.01  ...   24.03    23.76   
## 2021-11-01  24.091  24.05   23.74   23.31    23.98  ...   24.05    23.79   
## 2021-11-02  23.524  23.91   23.78   23.29    23.96  ...   23.87    23.76   
## 2021-11-03  23.251  23.73   23.81   23.26    23.93  ...   23.67    23.72   
## 2021-11-04  23.930  23.73   23.88   23.25    23.91  ...   23.75    23.74   
## 2021-11-05  24.176  23.79   23.93   23.25    23.89  ...   23.89    23.78   
## 
##             50d_ema  100d_ema  200d_ema  
## Date                                     
## 2021-10-29    23.60     24.07     24.59  
## 2021-10-31    23.61     24.07     24.59  
## 2021-11-01    23.63     24.07     24.58  
## 2021-11-02    23.62     24.06     24.57  
## 2021-11-03    23.61     24.04     24.56  
## 2021-11-04    23.62     24.04     24.55  
## 2021-11-05    23.64     24.04     24.55  
## 
## [7 rows x 11 columns]

Here we define a three new columns with the name 20d_ma, 50d_ma, 100d_ma,200d_ma . We now fill this column with the mean values of every n entries. The rolling function stacks a specific amount of entries, in order to make a statistical calculation possible. The window parameter is the one which defines how many entries we are going to stack. But there is also the min_periods parameter. This one defines how many entries we need to have as a minimum in order to perform the calculation. This is relevant because the first entries of our data frame won’t have a n entries previous to them. By setting this value to zero we start the calculations already with the first number, even if there is not a single previous value. This has the effect that the first value will be just the first number, the second one will be the mean of the first two numbers and so on, until we get to a b values.

By using the mean function, we are obviously calculating the arithmetic mean. However, we can use a bunch of other functions like max, min or median if we like to.

*** Standard Deviation ***

The variability of the closing stock prices determinies how vo widely prices are dispersed from the average price. If the prices are trading in narrow trading range the standard deviation will return a low value that indicates low volatility. If the prices are trading in wide trading range the standard deviation will return high value that indicates high volatility.

## Date
## 2021-10-29    0.230266
## 2021-10-31    0.263288
## 2021-11-01    0.235062
## 2021-11-02    0.231465
## 2021-11-03    0.350269
## 2021-11-04    0.321557
## 2021-11-05    0.327914
## Name: Std_dev, dtype: float64

*** Relative Strength Index ***

The relative strength index is a indicator of mementum used in technical analysis that measures the magnitude of current price changes to know overbought or oversold conditions in the price of a stock or other asset. If RSI is above 70 then it is overbought. If RSI is below 30 then it is oversold condition.

##                   RSI
## Date                 
## 2021-11-01  56.812068
## 2021-11-02  42.723626
## 2021-11-03  37.664385
## 2021-11-04  53.178372
## 2021-11-05  57.490494

*** Average True range ***

##                    ATR    20dayEMA    ATRdiff
## Date                                         
## 2021-11-01  441.138255  483.926902 -42.788647
## 2021-11-02  455.699808  481.238607 -25.538799
## 2021-11-03  464.935536  479.685934 -14.750398
## 2021-11-04  492.725855  480.927831  11.798024
## 2021-11-05  487.531151  481.556719   5.974432

*** Wiliams %R ***

Williams %R, or just %R, is a technical analysis oscillator showing the current closing price in relation to the high and low of the past N days.The oscillator is on a negative scale, from −100 (lowest) up to 0 (highest). A value of −100 means the close today was the lowest low of the past N days, and 0 means today’s close was the highest high of the past N days.

##             Williams %R
## Date                   
## 2021-10-29   -85.178571
## 2021-10-31   -95.530726
## 2021-11-01   -67.486034
## 2021-11-02   -95.847458
## 2021-11-03   -85.634921
## 2021-11-04   -28.033473
## 2021-11-05    -7.058824

Readings below -80 represent oversold territory and readings above -20 represent overbought.

*** ADX ***

ADX is used to quantify trend strength. ADX calculations are based on a moving average of price range expansion over a given period of time. The average directional index (ADX) is used to determine when the price is trending strongly.

0-25: Absent or Weak Trend

25-50: Strong Trend

50-75: Very Strong Trend

75-100: Extremely Strong Trend

##                   ADX
## Date                 
## 2021-10-29  34.584584
## 2021-10-31  30.026062
## 2021-11-01  27.153053
## 2021-11-02  26.381451
## 2021-11-03  28.407961
## 2021-11-04  25.373094
## 2021-11-05  22.059429
##                    CCI
## Date                  
## 2021-10-29   -2.003314
## 2021-10-31  -24.911649
## 2021-11-01   -2.687672
## 2021-11-02  -93.305878
## 2021-11-03 -193.503797
## 2021-11-04  -61.738905
## 2021-11-05    2.435831
##                  ROC
## Date                
## 2021-10-29  3.017538
## 2021-10-31 -0.180044
## 2021-11-01 -1.448149
## 2021-11-02 -2.672735
## 2021-11-03 -4.899996
## 2021-11-04 -2.138797
## 2021-11-05 -1.691607

*** MACD ***

Moving Average Convergence Divergence (MACD) is a trend-following momentum indicator that shows the relationship between two moving averages of a security’s price. The MACD is calculated by subtracting the 26-period Exponential Moving Average (EMA) from the 12-period EMA.

##             MACD_IND
## Date                
## 2021-11-01 -0.006777
## 2021-11-02 -0.051721
## 2021-11-03 -0.097203
## 2021-11-04 -0.079662
## 2021-11-05 -0.050990
## MACD_IND   -0.057271
## dtype: float64

*** Bollinger Bands ***

Bollinger Bands are a type of statistical chart characterizing the prices and volatility over time of a financial instrument or commodity.

## 24.76 23.93 23.1

In case we choose another value than zero for our min_periods parameter, we will end up with a couple of NaN-Values . These are not a number values and they are useless. Therefore, we would want to delete the entries that have such values.

We do this by using the dropna function. If we would have had any entries with NaN values in any column, they would now have been deleted

3: Predicting the movement of stock

To predict the movement of the stock we use 5 lag returns as the dependent variables. The first leg is return yesterday, leg2 is return day before yesterday and so on. The dependent variable is whether the prices went up or down on that day. Other variables include the technical indicators which along with 5 lag returns are used to predict the movement of stock using logistic regression.

3.1: Creating lag returns

3.2: Creating returns dataframe

3.2: create the lagged percentage returns columns

##                Today      Lag1      Lag2      Lag3      Lag4  ...     Lag11  \
## Date                                                          ...             
## 2021-10-21 -1.124974  2.353138  2.287036 -0.545214  1.324989  ...  2.217989   
## 2021-10-22  1.154324 -1.124974  2.353138  2.287036 -0.545214  ...  0.319489   
## 2021-10-26 -1.476543  1.154324 -1.124974  2.353138  2.287036  ... -0.336164   
## 2021-10-27  0.427599 -1.476543  1.154324 -1.124974  2.353138  ...  0.559205   
## 2021-10-28 -0.214956  0.427599 -1.476543  1.154324 -1.124974  ...  0.207432   
## 2021-10-29 -0.716683 -0.214956  0.427599 -1.476543  1.154324  ... -0.841224   
## 2021-11-02 -1.844279 -0.716683 -0.214956  0.427599 -1.476543  ...  2.913743   
## 2021-11-03 -1.160517 -1.844279 -0.716683 -0.214956  0.427599  ...  1.324989   
## 2021-11-04  2.920305 -1.160517 -1.844279 -0.716683 -0.214956  ... -0.545214   
## 2021-11-05  1.027998  2.920305 -1.160517 -1.844279 -0.716683  ...  2.287036   
## 
##                Lag12     Lag13     Lag14     Lag15  
## Date                                                
## 2021-10-21  2.615778 -4.341051  0.218643 -1.111945  
## 2021-10-22  2.217989  2.615778 -4.341051  0.218643  
## 2021-10-26  0.319489  2.217989  2.615778 -4.341051  
## 2021-10-27 -0.336164  0.319489  2.217989  2.615778  
## 2021-10-28  0.559205 -0.336164  0.319489  2.217989  
## 2021-10-29  0.207432  0.559205 -0.336164  0.319489  
## 2021-11-02 -0.841224  0.207432  0.559205 -0.336164  
## 2021-11-03  2.913743 -0.841224  0.207432  0.559205  
## 2021-11-04  1.324989  2.913743 -0.841224  0.207432  
## 2021-11-05 -0.545214  1.324989  2.913743 -0.841224  
## 
## [10 rows x 16 columns]

3.3: “Direction” column (+1 or -1) indicating an up/down day

***3.4: Create the dependent and independent variables ***

3.5: Create training and test sets

##                 Lag1      Lag2      Lag3      Lag4      Lag5  ...         CCI  \
## Date                                                          ...               
## 2021-10-29 -0.214956  0.427599 -1.476543  1.154324 -1.124974  ...   -2.003314   
## 2021-11-02 -0.716683 -0.214956  0.427599 -1.476543  1.154324  ...  -93.305878   
## 2021-11-03 -1.844279 -0.716683 -0.214956  0.427599 -1.476543  ... -193.503797   
## 2021-11-04 -1.160517 -1.844279 -0.716683 -0.214956  0.427599  ...  -61.738905   
## 2021-11-05  2.920305 -1.160517 -1.844279 -0.716683 -0.214956  ...    2.435831   
## 
##                  ROC         DX  Open-Close  Open-Open  
## Date                                                    
## 2021-10-29  3.017538   9.987488      -0.129     -0.140  
## 2021-11-02 -2.672735   1.263201       0.029      0.130  
## 2021-11-03 -4.899996  13.530062       0.041     -0.555  
## 2021-11-04 -2.138797   1.163113       0.384      0.070  
## 2021-11-05 -1.691607   5.690663      -0.090      0.205  
## 
## [5 rows x 34 columns]

***3.6: Create model ***

3.7: train the model on the training set

## LogisticRegression(max_iter=10000)

3.8: make an array of predictions on the test set

3.9: output the hit-rate and the confusion matrix for the model

## 
## Train Accuracy: 87.36%
## Test Accuracy: 78.49%
## [[83 36]
##  [ 1 52]]

***3.10: Predict movement of stock for tomorrow. ***

##             y_test  y_pred
## Date                      
## 2021-10-29      -1      -1
## 2021-11-02      -1      -1
## 2021-11-03      -1      -1
## 2021-11-04       1       1
## 2021-11-05       1      -1
## [-1]

Say Something

Comments

Nothing yet.

Recent Posts

Categories

About

about