Anomaly Detection: Identify When UK Currency Crashed
This article and the upcoming articles in this series will focus on using various machine learning techniques to identify anomalies.
Join the DZone community and get the full member experience.
Join For FreeIn the last piece, Fundamentals of Anomaly Detection, we discussed the different types of anomaly detection techniques. The rulebased anomaly detection techniques are very much tied up with the business rules and are primarily based on the experience of the business users. This article and the upcoming articles in this series will focus on using various machine learning techniques to identify anomalies.
In this piece, we use the exchange rates between the US and other countries' datasets from Kaggle. The dataset provides the exchange rate movement of currencies from the year 1971 to 2017. This macrolevel dataset can help identify when there is an abnormal trend in currency movements. Towards the end, we will validate our findings with the historical financial crisis events to verify whether our observation from the data holds.
For the below example and to validate the insights, we have used the exchange rate movements for pound, which is the United Kingdom's (UK) currency. We have used the fundamental rolling mean absolute error (MAE) technique. There are other time series anomaly detection techniques like LSTM, which we will cover in future articles. You can pick up your currency or your preferred currency and try to get insights on when it crashed. In the next part of the series, we will use these insights to predict when the currency might hit again to prepare ourselves better.
As always, we start by loading the required libraries. The pandas and numpy libraries are the primary data manipulation libraries in Python. Package matplotlib is the most used and most comprehensive library that provides methods to plot data in Python. The command %matplotlib inline
is directive and issued to the Jupyter notebook to display the inline plots and the cell where the plot is executed.
import pandas as pd
import numpy as np
from matplotlib import pyplot as plt
from sklearn.metrics import mean_absolute_error
%matplotlib inline
Next, we load the exchange_rates data. There were some text and other currencies in the original data file that were irrelevant for this study. I removed them prior to uploading it to a Pandas data frame. We create a time series plot for this data in the following code snippet.
data = pd.read_csv('timeseries\exchange_rates.csv', index_col=['time_period'], parse_dates=['time_period'])
plt.figure(figsize=(15, 7))
plt.plot(data.UK)
plt.title('Timeseries')
plt.grid(True)
plt.show()
As we can see from the above plot, there are some missing values in between the series; we can do some data imputations to address the null values. We can use various methods to fill the gaps like rolling mean, rolling median, and various other interpolate methods. The pandas package has an interpolate function that can be used for this task.
data['UK'].interpolate(method='linear').plot(figsize = (16,6))
data_imputed =data['UK'].interpolate(method='linear')
data_imputed =pd.DataFrame(data_imputed)
In the following piece of code, we derive the anomalies and mark them in the timeseries plot. The logic is outlined below:
 Window parameter is used to input the window size. The window size is required to calculate the moving average (also known as rolling mean). This is an important parameter for time series calculations as it defines how many previous values to consider for the calculations. When a window slides over each value iteratively, it is known as a sliding window.
 The mean absolute error (MAE) is calculated for each data point. The MAE is the absolute difference between the expected value, i.e. rolling mean and the actual value.
 Next, the confidence interval is estimated. The confidence interval is a range that is used to approximate the actual value of a number within this range.
 Any data value lying outside its confidence interval is designated as an anomaly and is marked in the plot.
def plotTimeSeries(series, window, scale=1.96):
rolling_mean = series.rolling(window=window).mean()
plt.figure(figsize=(15,5))
plt.title("Anomalies in the time series for \n window size = {}".format(window))
plt.plot(rolling_mean, "g", label="Rolling mean trend")
# Calculate confidence intervals based on MAE
mae = mean_absolute_error(series[window:], rolling_mean[window:])
deviation = np.std(series[window:]  rolling_mean[window:])
lower_band = rolling_mean  (mae + scale * deviation)
upper_band = rolling_mean + (mae + scale * deviation)
# Calculate Anomalies
anomalies = pd.DataFrame(index=series.index, columns=series.columns)
anomalies[series<lower_band] = series[series<lower_band]
anomalies[series>upper_band] = series[series>upper_band]
#Plot the confidence intervals and anomalies
plt.plot(upper_band, "r", label="Upper Bond / Lower Bond")
plt.plot(lower_band, "r")
plt.plot(anomalies, "ro", markersize=8)
plt.plot(series[window:], label="Actual values")
plt.legend(loc="upper left")
plt.grid(True)
We invoke the function plotTimeSeries with the data and 360 days as a sliding time window.
plotTimeSeries(data_imputed, 360)
The anomalies — i.e. potential pound currency crash — are shown in red. The years around 1976, 1983, and 2008 are marked as anomalies. Now, if we go back to history and check the events for these years, we can see that there was a currency crash.
1976 IMF Crisis — A financial crisis in the United Kingdom in 1976
Early 1980s Recession — A severe global economic downturn that affected much of the developed world in the early 1980s
2008 Financial Crisis — Many economists consider it as the most severe economic crisis since the Great Depression of the 1930s
Though I knew about the 2008 financial crisis, I was ignorant of the other two before this analysis.
These insights from the data equipped me with specific facts and figures that I was unaware of, and this kind of analysis is termed as datadriven insights. I can utilize this information to dig more into these crises, or I might try to predict when the next financial crisis will occur.
In the next part of this series, we will see some more sophisticated methods for time series anomalies and predict the next currency crash. Until then, happy reading and data crunching for knowledge, insights, and fun.
You can find the code along with the data on my GitHub repo.
Opinions expressed by DZone contributors are their own.
Trending

RAML vs. OAS: Which Is the Best API Specification for Your Project?

DevOps Midwest: A Community Event Full of DevSecOps Best Practices

How To Manage Vulnerabilities in Modern CloudNative Applications

Five Java Books Beginners and Professionals Should Read
Comments