Selecting Optimal Parameters for XGBoost Model Training
Explore how to select optimal parameters for XGBoost model training.
Join the DZone community and get the full member experience.
Join For FreeThere is always a bit of luck involved when selecting parameters for Machine Learning model training. Lately, I have worked with gradient boosted trees and XGBoost in particular. We are using XGBoost in the enterprise to automate repetitive human tasks. While training ML models with XGBoost, I created a pattern to choose parameters, which helps me to build new models quicker. I will share it in this post, and hopefully you will find it useful too.
I'm using the Pima Indians Diabetes Database for the training. CSV data can be downloaded from here.
This is the Python code that runs XGBoost training step and builds a model. Training is executed by passing pairs of train/test data, which helps to evaluate training quality ad-hoc during model construction:
{
"cells": [
{
"cell_type": "code",
"execution_count": 149,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[0]\tvalidation_0-error:0.231518\tvalidation_0-logloss:0.688982\tvalidation_1-error:0.30315\tvalidation_1-logloss:0.689593\n",
"Multiple eval metrics have been passed: 'validation_1-logloss' will be used for early stopping.\n",
"\n",
"Will train until validation_1-logloss hasn't improved in 15 rounds.\n",
"[1]\tvalidation_0-error:0.206226\tvalidation_0-logloss:0.685218\tvalidation_1-error:0.216535\tvalidation_1-logloss:0.686122\n",
"[2]\tvalidation_0-error:0.196498\tvalidation_0-logloss:0.681505\tvalidation_1-error:0.220472\tvalidation_1-logloss:0.682881\n",
"[3]\tvalidation_0-error:0.196498\tvalidation_0-logloss:0.67797\tvalidation_1-error:0.220472\tvalidation_1-logloss:0.679601\n",
"[4]\tvalidation_0-error:0.180934\tvalidation_0-logloss:0.674278\tvalidation_1-error:0.208661\tvalidation_1-logloss:0.676067\n",
"[5]\tvalidation_0-error:0.177043\tvalidation_0-logloss:0.670627\tvalidation_1-error:0.212598\tvalidation_1-logloss:0.673761\n",
"[6]\tvalidation_0-error:0.175097\tvalidation_0-logloss:0.667069\tvalidation_1-error:0.216535\tvalidation_1-logloss:0.671441\n",
"[7]\tvalidation_0-error:0.18677\tvalidation_0-logloss:0.663582\tvalidation_1-error:0.212598\tvalidation_1-logloss:0.668586\n",
"[8]\tvalidation_0-error:0.180934\tvalidation_0-logloss:0.660353\tvalidation_1-error:0.23622\tvalidation_1-logloss:0.665983\n",
"[9]\tvalidation_0-error:0.161479\tvalidation_0-logloss:0.656739\tvalidation_1-error:0.228346\tvalidation_1-logloss:0.662987\n",
"[10]\tvalidation_0-error:0.167315\tvalidation_0-logloss:0.653582\tvalidation_1-error:0.228346\tvalidation_1-logloss:0.660091\n",
"[259]\tvalidation_0-error:0.122568\tvalidation_0-logloss:0.34313\tvalidation_1-error:0.220472\tvalidation_1-logloss:0.475866\n",
"[260]\tvalidation_0-error:0.124514\tvalidation_0-logloss:0.34261\tvalidation_1-error:0.220472\tvalidation_1-logloss:0.476068\n",
"[261]\tvalidation_0-error:0.120623\tvalidation_0-logloss:0.342156\tvalidation_1-error:0.216535\tvalidation_1-logloss:0.476165\n",
"[262]\tvalidation_0-error:0.120623\tvalidation_0-logloss:0.341714\tvalidation_1-error:0.216535\tvalidation_1-logloss:0.476143\n",
"[263]\tvalidation_0-error:0.124514\tvalidation_0-logloss:0.341209\tvalidation_1-error:0.216535\tvalidation_1-logloss:0.476063\n",
"[264]\tvalidation_0-error:0.120623\tvalidation_0-logloss:0.340779\tvalidation_1-error:0.220472\tvalidation_1-logloss:0.47595\n",
"[265]\tvalidation_0-error:0.120623\tvalidation_0-logloss:0.340297\tvalidation_1-error:0.212598\tvalidation_1-logloss:0.475858\n",
"[266]\tvalidation_0-error:0.120623\tvalidation_0-logloss:0.339908\tvalidation_1-error:0.212598\tvalidation_1-logloss:0.476057\n",
"[267]\tvalidation_0-error:0.120623\tvalidation_0-logloss:0.339312\tvalidation_1-error:0.220472\tvalidation_1-logloss:0.476228\n",
"[268]\tvalidation_0-error:0.120623\tvalidation_0-logloss:0.338874\tvalidation_1-error:0.216535\tvalidation_1-logloss:0.476266\n",
"[269]\tvalidation_0-error:0.120623\tvalidation_0-logloss:0.338543\tvalidation_1-error:0.216535\tvalidation_1-logloss:0.476202\n",
"[270]\tvalidation_0-error:0.120623\tvalidation_0-logloss:0.33821\tvalidation_1-error:0.216535\tvalidation_1-logloss:0.47607\n",
"[271]\tvalidation_0-error:0.120623\tvalidation_0-logloss:0.337716\tvalidation_1-error:0.212598\tvalidation_1-logloss:0.476229\n",
"[272]\tvalidation_0-error:0.118677\tvalidation_0-logloss:0.337295\tvalidation_1-error:0.212598\tvalidation_1-logloss:0.47612\n",
"[273]\tvalidation_0-error:0.118677\tvalidation_0-logloss:0.336927\tvalidation_1-error:0.212598\tvalidation_1-logloss:0.476152\n",
"[274]\tvalidation_0-error:0.118677\tvalidation_0-logloss:0.33651\tvalidation_1-error:0.212598\tvalidation_1-logloss:0.476127\n",
"[275]\tvalidation_0-error:0.120623\tvalidation_0-logloss:0.336017\tvalidation_1-error:0.216535\tvalidation_1-logloss:0.476117\n",
"[276]\tvalidation_0-error:0.120623\tvalidation_0-logloss:0.335497\tvalidation_1-error:0.212598\tvalidation_1-logloss:0.476063\n",
"[277]\tvalidation_0-error:0.116732\tvalidation_0-logloss:0.335159\tvalidation_1-error:0.216535\tvalidation_1-logloss:0.476113\n",
"[278]\tvalidation_0-error:0.114786\tvalidation_0-logloss:0.334812\tvalidation_1-error:0.216535\tvalidation_1-logloss:0.476143\n",
"[279]\tvalidation_0-error:0.114786\tvalidation_0-logloss:0.334481\tvalidation_1-error:0.216535\tvalidation_1-logloss:0.476163\n",
"[280]\tvalidation_0-error:0.116732\tvalidation_0-logloss:0.333843\tvalidation_1-error:0.216535\tvalidation_1-logloss:0.476359\n",
"Stopping. Best iteration:\n",
"[265]\tvalidation_0-error:0.120623\tvalidation_0-logloss:0.340297\tvalidation_1-error:0.212598\tvalidation_1-logloss:0.475858\n",
"\n",
"CPU times: user 690 ms, sys: 310 ms, total: 1 s\n",
"Wall time: 799 ms\n"
]
}
],
"source": [
"%%time\n",
"\n",
"model = xgb.XGBClassifier(max_depth=12,\n",
" subsample=0.33,\n",
" objective='binary:logistic',\n",
" n_estimators=300,\n",
" learning_rate = 0.01)\n",
"eval_set = [(train_X, train_Y), (test_X, test_Y)]\n",
"model.fit(train_X, train_Y.values.ravel(), early_stopping_rounds=15, eval_metric=[\"error\", \"logloss\"], eval_set=eval_set, verbose=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.7"
}
},
"nbformat": 4,
"nbformat_minor": 2
}Key parameters in XGBoost (the ones that would affect model quality greatly), assuming you already selected max_depth (more complex classification task, deeper the tree), subsample (equal to evaluation data percentage), objective (classification algorithm):
- n_estimators — the number of runs XGBoost will try to learn
- learning_rate — learning speed
- early_stopping_rounds — overfitting prevention, stop early if no improvement in learning
With matpotlib library, we can plot training results for each run (from XGBoost output). This helps to understand if the iteration that was chosen to build the model was the best one possible. Here, we are using sklearn library to evaluate model accuracy and then plotting training results with matpotlib:
{
"cells": [
{
"cell_type": "code",
"execution_count": 150,
"metadata": {},
"outputs": [],
"source": [
"# make predictions for test data\n",
"y_pred = model.predict(test_X)\n",
"predictions = [round(value) for value in y_pred]"
]
},
{
"cell_type": "code",
"execution_count": 151,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Accuracy: 78.74%\n"
]
}
],
"source": [
"# evaluate predictions\n",
"accuracy = accuracy_score(test_Y, predictions)\n",
"print(\"Accuracy: %.2f%%\" % (accuracy * 100.0))"
]
},
{
"cell_type": "code",
"execution_count": 152,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEICAYAAACuxNj9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd4FWX2wPHvSQ/pCYFAEnqQXiOoKKBIERewCzbEgr27lt+6q6Kurrtrb4uKXbHgKhYWsWBBBQJSQws9hBJCQgiQkHJ+f8yg10hIILm5KefzPPdJ7sw7M2e4ISfvvE1UFWOMMeZw/HwdgDHGmLrPkoUxxphKWbIwxhhTKUsWxhhjKmXJwhhjTKUsWRhjjKmUJQtjjDGVsmRh6hQRCReRDSJygce2CBHZJCLneGxLFZFPRSRXRPJEJF1EHhKRGHf/pSJSKiIF7mudiFzj5dgHi0hmJWVeFZEHfR2HMUfKkoWpU1S1AJgIPCki8e7mR4E0Vf0AQEROAGYDc4BOqhoNjABKgJ4ep/tJVcNVNRw4B3hURHrXzp0Y07BYsjB1jqp+AXwGPCUig4HzgOs8ijwKvKKqD6vqdveYTap6r6rOruCcC4EVQOeD20RktIgsd2sms0XEc19nd1ueW2a0x76Rbk1mj4hsEZHbRSQMmAG09KjNtDyS+xaRE0Rkvojsdr+e4LGvrYh8517zSxF5VkTePJLzu+eJEpHXRSRbRDaKyD0i4ufu6yAi37rX3yki77rbRUQeF5Ed7r4lItLtSK9t6jdLFqauugUYDHwA3K6qWwHcX8rHA9OO5GQicizQEUhz33cE3gFuBuKBz4FPRCRIRAKBT4AvgGbADcBbInKMe7qXgatUNQLoBnytqnuB04Csg7UZVc06gvhicRMkEAc8BnwmInFukbeBee6++4CLj+T+PTwNRAHtgEHAJcAEd98DOPccAyS5ZQGGAQNx/v2igfOBnKO8vqmnLFmYOklVc4HlQBPgQ49dMTg/t9sObhCRR90awF4Rucej7HHu9gKcX7RvAGvcfecDn6nqLFUtBv4FhAInAMcB4cAjqnpAVb8GPgXGuccWA11EJFJVc91aS3WdDqxR1TdUtURV3wFWAqNEpBVwLPA3N54fgOlHegER8ce577tVdY+qbgD+zW+JpxhoDbRU1UL3Oge3RwCdAFHVFQeTt2k8LFmYOklELgLaAF8C//DYlQuUAS0OblDVO9x2i/8CAR5lf1bVaLfNIgHoCvzd3dcS2OhxjjJgM5Do7tvsbjtoo7sP4GxgJLDRfWxzfPXu9o/xlLtmS2CXqu7z2Lf5KK7RFAgqdx3P+7oDEGCe++jtMgA3WT4DPAtsF5HJIhJ5FNc39ZglC1PniEgz4HHgSuAq4DwRGQjgPu6ZC5x1JOd02zamAaPcTVk4f0UfvKYAycAWd1/ywWf5rlbuPlR1vqqOwXlE9RHw3sHLHElM5fwunnLX3ArEikgTj33JR3GNnfxWeyh/DVR1m6peqaotcf7dnxORDu6+p1S1L07C7Qj8+Siub+oxSxamLnoG+EhVv3Efd9wBvCgiwe7+O4DLROQuN7EgIklA24pO6D77PxPn0RY4v+BPF5EhbhvFbUAR8CNOMtoL3CEigW4j+yhgqtumcaGIRLmPr/KBUvec24E4EYmq5P78RSTE4xWE02bSUUQuEJEAETkf6AJ8qqobcdpa7nOvfzy/Jb0KlbtGCE6N7D3gIXG6I7cGbgXedMuf6/47glODU6BURI4Vkf7uv9NeoNDjnk1joar2sledeQFn4PyVHV1u+1fAQx7v++P8gs1zX8uAh4A4d/+lOL/QCtzXDpwG7WYe5zgTSAd2A98CXT32dXW37XbLnOluDwL+h/PLNB+YD5zocdwUnMbfPJxn/+Xv71WcX8Kerx/cfScCC9xrLih33vbA98Ae999iMvByBf+Ggw9xDQU64LT5vAlk4zzK+hvg5x73KE4towBYC0x0tw8BlrjbdwJvAeG+/lmxV+2+xP1hMMbUI2631pWqeq+vYzGNgz2GMqYecB8FtRcRPxEZAYzBaS8xplYEVF7EGFMHJOB0IY4DMoFrVPUX34ZkGhN7DGWMMaZS9hjKGGNMpRrMY6imTZtqmzZtfB2GMcbUKwsWLNipqvGVlfNqsnAb4p4E/IGXVPWRcvsfB0523zbB6dYY7e4bDxycuuFBVX3tcNdq06YNaWlpNRm+McY0eCJSfuaAQ/JasnDnoXkWGIrTIDdfRKaravrBMqp6i0f5G4De7vexwL1AKk7/8AXusbneitcYY0zFvNlm0Q/IUNV1qnoAmIrT3a8i43AGTQEMB2ap6i43QczCWa/AGGOMD3gzWSTy+8nOMvltwrLfcacdaAt8fSTHishEEUkTkbTs7OwaCdoYY8wfebPNQg6xraJ+umOBD1T14HwzVTpWVSfjTHtAamqq9QE2xlRZcXExmZmZFBYW+jqUWhESEkJSUhKBgYFHdbw3k0Umv58ZMwlnzp9DGcvvV0LLxJnfxvPY2TUYmzGmkcvMzCQiIoI2bdrgTDrccKkqOTk5ZGZm0rZthfNtHpY3H0PNB1Lc5SCDcBLCHxZscVcfiwF+8tg8ExgmIjEiEoOzUtdML8ZqjGlkCgsLiYuLa/CJAkBEiIuLq1Ytyms1C1UtEZHrcX7J+wNTVHW5iEwC0lT1YOIYB0xVj6HkqrpLRB7ASTgAk1R1l7diNcY0To0hURxU3Xv16jgLVf0cZxppz21/K/f+vgqOnYIz3bN3Fe0he8bDhB9/GaHNO3j9csYYUx81+uk+Nm7dTpNfXmLre7f5OhRjTCOSk5NDr1696NWrFwkJCSQmJv76/sCBA1U6x4QJE1i1apWXI3U0mOk+jlbrNh2YFnshZ+dMoSD9C8K7DPN1SMaYRiAuLo5FixYBcN999xEeHs7tt9/+uzIHFx7y8zv03/WvvPKK1+M8qNHXLAC6nX03G7Q5hdP/DKXFvg7HGNOIZWRk0K1bN66++mr69OnD1q1bmThxIqmpqXTt2pVJkyb9WvbEE09k0aJFlJSUEB0dzV133UXPnj05/vjj2bFjR43G1ehrFgDHJDXj5VY3cfnm/2P3d88RdfJNvg7JGFOL7v9kOelZ+TV6zi4tI7l3VNejOjY9PZ1XXnmFF154AYBHHnmE2NhYSkpKOPnkkznnnHPo0qXL747ZvXs3gwYN4pFHHuHWW29lypQp3HXXXdW+j4OsZuEadsalfF/Wg6DvH4W9O30djjGmEWvfvj3HHnvsr+/feecd+vTpQ58+fVixYgXp6el/OCY0NJTTTjsNgL59+7Jhw4YajclqFq7kuDCmd7uT45dfzO7P7yPq3Gd8HZIxppYcbQ3AW8LCwn79fs2aNTz55JPMmzeP6OhoLrrookOOlwgKCvr1e39/f0pKSmo0JqtZeDh/5FCmMoyI5W/BtmW+DscYY8jPzyciIoLIyEi2bt3KzJm+GZ9sycJD0/Bg8vvfzm5twt5p10NJka9DMsY0cn369KFLly5069aNK6+8kgEDBvgkjgazBndqaqrWxOJHBUUlPPDIQ/xDH0N7jkXOeAEa0ShPYxqLFStW0LlzZ1+HUasOdc8iskBVUys71moW5YQHB9D51Et4rPgcZPFU+P7fvg7JGGN8zpLFIVzQvzXToy7k68BB6NcPwoY5vg7JGGN8ypLFIQQF+HHnaZ25fs94CpokwUdXQ2HN9sE2xpj6xJJFBUZ0S6BTqwRuLroa3Z0JM//P1yEZY4zPWLKogIjwl9M789XetqQljYdf3oCVn1d+oDHGNECWLA6jb+tYRnRN4MqNp1Ic3xU+udFGdxtjGiVLFpW4e2Qn9pX588+w26BwN3xyEzSQ7sbGGN+piSnKAaZMmcK2bdu8GKnDkkUlWseFce3g9kxeGcKmXrfByk+d9ovSmh1Kb4xpXA5OUb5o0SKuvvpqbrnlll/fe07dURlLFnXIVQPbkxAZws2bBqD9roKfn4Pv/+XrsIwxDdRrr71Gv3796NWrF9deey1lZWWUlJRw8cUX0717d7p168ZTTz3Fu+++y6JFizj//POPuEZypGwiwSoIDfLnlqEp3DltKdMH3MSYrtkw50nocwlEtvR1eMaY6ppxF2xbWrPnTOgOpz1yxIctW7aM//73v/z4448EBAQwceJEpk6dSvv27dm5cydLlzpx5uXlER0dzdNPP80zzzxDr169ajb+crxasxCRESKySkQyROSQE6uLyHkiki4iy0XkbY/tpSKyyH1N92acVXFO32R6JEXxwKfp5J/4FygrhWlXQvEfZ380xpij9eWXXzJ//nxSU1Pp1asX3377LWvXrqVDhw6sWrWKm266iZkzZxIVFVWrcXmtZiEi/sCzwFAgE5gvItNVNd2jTApwNzBAVXNFpJnHKfarqndT5RHw9xP+fmZ3znh2DtfP2MUro57G/6OJ8N+JcM4r4Ofv6xCNMUfrKGoA3qKqXHbZZTzwwAN/2LdkyRJmzJjBU089xbRp05g8eXKtxeXNmkU/IENV16nqAWAqMKZcmSuBZ1U1F0BVa3YdwBrWLTGKB87oxners/lPXl8Y9hCkfwxf3ufr0IwxDcSpp57Ke++9x86dTjf9nJwcNm3aRHZ2NqrKueeey/3338/ChQsBiIiIYM+ePV6Py5ttFonAZo/3mUD/cmU6AojIHMAfuE9V/+fuCxGRNKAEeERVPyp/ARGZCEwEaNWqVc1GX4Fx/Vrx1YrtPP/NWsbeMZHYXWvhx6cgZSi0HVgrMRhjGq7u3btz7733cuqpp1JWVkZgYCAvvPAC/v7+XH755agqIsI//vEPACZMmMAVV1xBaGgo8+bNO6KeVEfCa1OUi8i5wHBVvcJ9fzHQT1Vv8CjzKVAMnAckAd8D3VQ1T0RaqmqWiLQDvgaGqOraiq5XU1OUV8Wa7XsY/sR3jOvXiodObw//OclZ++KaORBSu88RjTFHx6Yod9SFKcozgWSP90lA1iHKfKyqxaq6HlgFpACoapb7dR0wG+jtxViPSErzCC49oS1vzd1EWlYhnPkfyN8Cn99hA/aMMQ2SN5PFfCBFRNqKSBAwFijfq+kj4GQAEWmK81hqnYjEiEiwx/YBwB9XKPeh24Z1JDE6lLs+XEpRQm8YdCcsmQo/2drdxpiGx2vJQlVLgOuBmcAK4D1VXS4ik0RktFtsJpAjIunAN8CfVTUH6Aykichid/sjnr2o6oKw4AAePLMbGTsKeH72Whh4B3QZA1/8FVZ+5uvwjDFV0FBWCq2K6t6rLataTTe+8wv/W7aNz286kQ7R/vDq6ZC9EibMgJZ1puevMaac9evXExERQVxcHNLAl05WVXJyctizZw9t27b93b6qtllYsqimnQVFnPrYt3SID+e9q47Hb+8OeGkIlJXAxNkQkVDrMRljKldcXExmZiaFhY1jYG1ISAhJSUkEBgb+bntVk4VN91FNTcOD+cvIzvz5gyW8PW8TFx3XGsZNhZeHwgeXwSUfg39g5ScyxtSqwMDAP/yVbSpmEwnWgHP6JjGgQxwPf76CTTn7IKEbjHoSNs6Br+73dXjGGFNtlixqgIjw6Dk98fMTbn73F0pKy6DHeXDsFfDj07bCnjGm3rNkUUMSo0N58IxuLNyUx7PfuGMHh//dmXnykxuhINu3ARpjTDVYsqhBY3olcmbvRJ76eg3LtuyGgGBnwF5hPrw/3hnlbYwx9ZAlixp23+iuxIUFcccHSyguLYPmXeGM55z2i09uthHexph6yZJFDYsKDeSBM7qRvjWfyd+tczZ2PwcG3w2L34YfHvNtgMYYcxQsWXjB8K4JjOyewJNfriFjhzt18KA7ofu58NUkWP6HCXSNMaZOs2ThJfeP7kaTYH9ue3+J0ztKBEY/A8n94b9XQeYCX4dojDFVZsnCS+Ijgpk0phuLN+cx+Xv3cVRgCJz/FoQ3g3fGQt7mw5/EGGPqCEsWXjSqRwtGdk/giVlrWLXNfRwVHg8XvA8lhfD2+VDk/RWujDGmuixZeJGI8MCYbkSEBHD7+4ud3lEAzTrBea85Ew6+PwFKDvg2UGOMqYQlCy+LCw/mwTO6sXTLbl6Y7bHQX/tT4E+PQcYseO9iZyyGMcbUUZYsasFp3VswqmdLnvp6DelZHkmh76Vw+mOwZhb8ZyDsWu+zGI0x5nAsWdSSSaO7EhUa9PvHUQDHXg4TPofCPHh9NOzO9F2QxhhTAUsWtSQmLIi/n+kM1nv2m4zf72x1HFz8X9ifB6+Ngj3bfBOkMcZUwJJFLRrWNYEzeyfyzNcZztxRnlr2houmwZ7t8OY5ULj70CcxxhgfsGRRy+4d1YXYMOdx1IGSst/vTO4H578B2SvgnQuguHGs4GWMqfu8mixEZISIrBKRDBG5q4Iy54lIuogsF5G3PbaPF5E17mu8N+OsTdFNgnj4rO6s3LaHp79e88cCHYbAGS/Axh/go6uhrOyPZYwxppZ5bVlVEfEHngWGApnAfBGZrqrpHmVSgLuBAaqaKyLN3O2xwL1AKqDAAvfYXG/FW5uGdG7OOX2TeG72WoZ2aU6PpOjfF+hxLuzJgll/g/DmMPxh8LNKoDHGd7z5G6gfkKGq61T1ADAVGFOuzJXAsweTgKrucLcPB2ap6i533yxghBdjrXV//VMX4sODue29xew/UPrHAifcCMddC3NfgM9vs6nNjTE+5c1kkQh4Tn6U6W7z1BHoKCJzRORnERlxBMciIhNFJE1E0rKz69dKdFGhgfzz3B5kZBdw7/Rlfywg4qy0N+AmSJsCPz1T+0EaY4zLm8lCDrGt/J/HAUAKMBgYB7wkItFVPBZVnayqqaqaGh8fX81wa99JKfFcf3IH3kvLZNqCQ4yvEIEh90GXMfDFPTDnKathGGN8wpvJIhNI9nifBGQdoszHqlqsquuBVTjJoyrHNgg3DUmhf9tY7vlo2W9rX3jy84OzXoIuZ8Csv8LH10FpSe0Haoxp1LyZLOYDKSLSVkSCgLHA9HJlPgJOBhCRpjiPpdYBM4FhIhIjIjHAMHdbgxPg78dT43rTJMifa99aeOj2i4AgOOcVGHgHLHoLpl0GpcW1H6wxptHyWrJQ1RLgepxf8iuA91R1uYhMEpHRbrGZQI6IpAPfAH9W1RxV3QU8gJNw5gOT3G0NUvPIEJ4Y24s1Owr428eHaL8Ap4Zxyl9g2EOQ/jF8YLPVGmNqj2gDeQaempqqaWlpvg6jWh77YhVPfZ3Bv87tyTl9kyou+NNzMPNuOOZ0OPdVp+ZhjDFHQUQWqGpqZeWs834dctOpHTmuXSx//WgZa7YfZlGk46+F0x6FVZ/Be5dA8f7aC9IY0yhZsqhD/P2Ep8b2JizYab/Yd+AwDdn9r4KR/4LVM5zpzbcuqb1AjTGNjiWLOqZZZAhPnN+bjOwC/vbx8sMX7nelM1ttUQFMGQ7p5fsPGGNMzbBkUQedmNKUG05J4YMFmbyftvnwhdufAhNnQ/Ouzop73z5qYzGMMTXOkkUdddOQFKf94uNlrD5c+wVARHMY/yn0GAvfPATvj4cDe2snUGNMo2DJoo462H4RHhzAdRWNv/AUGAJnvgBDH4AVnziPpXZvqZ1gjTENniWLOqxZZAiPn9+LjOwCJn1aSfsFONODDLgRLngPdm2Al4ZA1i9ej9MY0/BZsqjjTkqJ55pB7Xln3mamL67ijCcpQ+HymeAXAFNGwE/P2roYxphqsWRRD9wytCN9WkVz5wdLWLipikt6NO8KV34D7U6Gmf8Hr/0Jcjd4NU5jTMNlyaIeCPT344WL+9IsMpjLX51PZu6+qh0YHg/j3oExz8G2pfDcCbDwdestZYw5YpYs6olmESG8OqEfJaXKtW8tpLC4kgbvg0Sg94VwzY+Q2Aem3wDvXgR7c7wbsDGmQbFkUY+0bRrGv8/ryZLM3dz/SXrlB3iKToZLpju9pVbPhOePh4wvvROoMabBsWRRzwzrmsA1g9vzzrxNlQ/YK8/Pz+ktNfEbCI2BN8+GGXfa3FLGmEpZsqiHbhvakRPax3HPR8tYsLGKDd6eEro7o777X+Os8T15sM0tZYw5LEsW9VCAvx9Pj+tNi6gQJrwyj1XbKhnhfSiBoXDaI3DRh7A/D148xVm21brYGmMOwZJFPRUXHsybV/QnJNCfK16fT+7eo1wIqcMQp/G743Bn2dbXR8Ou9TUbrDGm3rNkUY8lxTThhYv7sj2/iEumzGP3/qNcajUsDs5/E0Y/A1mL4PkBsPSDmg3WGFOvWbKo5/q0iuH5C/uwcls+46fMY0/hUSYMEehzMVz3MyR0g2mXw+tnOMnDGNPoWbJoAIZ0bs4zF/Rh2ZbdXPrKfPYWHWbRpMpEJTkz2A7/O2xdBJMHwbQrbPS3MY2cV5OFiIwQkVUikiEidx1i/6Uiki0ii9zXFR77Sj2226o+lRjeNYGnxvVm0eY8rnw9jQMl1WioDgiC46+DmxbDibfCik/h6VT4+kEoOcq2EWNMvSbqpakfRMQfWA0MBTKB+cA4VU33KHMpkKqq1x/i+AJVDa/q9VJTUzUtLa3acdd30xZkctv7ixndsyWPn98Lfz+p/knzs+DL+2DJuxDR0kkk/a8C/8Dqn9sY41MiskBVUysr582aRT8gQ1XXqeoBYCowxovXM8DZfZO4c0Qnpi/O4q5pSygrq4E/BiJbwlmT4cJp0DQFvviL0wi+7tvqn9sYUy94M1kkAp5DjDPdbeWdLSJLROQDEUn22B4iImki8rOInHGoC4jIRLdMWnZ2dg2GXr9dM7g9Nw5J4f0Fmfz142XUWO0x5VQYPx3GvQulRU432/cvhbwjHElujKl3Kk0WItJeRILd7weLyI0iEl2Fcx/q+Uf531qfAG1UtQfwJfCax75WbtXoAuAJEWn/h5OpTlbVVFVNjY+Pr0JIjcctp6Zw9aD2vDV3E4/8b2XNnvyYEXDtXBj8f7BqBjxzLMx+xKYNMaYBq0rNYhpQKiIdgJeBtsDbVTguE/CsKSQBv1u9R1VzVLXIffsi0NdjX5b7dR0wG+hdhWsal4hw54hjuOi4Vvzn23U8P3ttzV4gMAQG3wnXp8Exp8Hsh+G545zxGWVVnBHXGFNvVCVZlKlqCXAm8ISq3gK0qMJx84EUEWkrIkHAWOB3vZpExPM8o4EV7vYYj9pMU2AAcITTrBoRYdLobozu2ZJ//G8lb83dWPMXiU6Gc19xZrQNbOKMz3h+ACz/yKYOMaYBCahCmWIRGQeMB0a52yrtBqOqJSJyPTAT8AemqOpyEZkEpKnqdOBGERkNlAC7gEvdwzsD/xGRMpyE9ohnLypTdX5+wr/P60lBUQn3fLSMAD/h/GNb1fyF2g2Cq+dA+kfOI6n3x0Nyfxj5L2jRo+avZ4ypVZV2nRWRLsDVwE+q+o6ItAXOV9VHaiPAqrKus4dXWFzKxDcW8N3qbB49pwfnpSZXftDRKiuFxe/AF/fA/lzocCoMuBnanOiMFDfG1BlV7Tp7ROMsRCQGSFbVOjeftSWLyhWVlHLFa2n8tDaHZy/sw/CuCd694P48SHsZfn4e9mZDQg848RboeqYlDWPqiBobZyEis0UkUkRigcXAKyLyWE0EaWpXcIA/z13Yh26JUVzz5gI+XJjp3QuGRsNJt8HNS2HUk1BSBB9McF4FO7x7bWNMjapKA3eUquYDZwGvqGpf4FTvhmW8JSIkkLev7M/x7eO444MlfLu6FsanBIZC30vh2p/glL8604c82Qs+uBy2L/f+9Y0x1VaVZBHg9lo6D/jUy/GYWtAkKIDnL+pLSvMIrnwtjZnLt9XOhf38YeDtTtLofjZkzIIXTnQaxK27rTF1WlWSxSScHk1rVXW+iLQD1ng3LONtkSGBvHNlf7q0jOTatxZ6/5GUp6YpMPppuHERdD/PGaPxZE/49p9QYCPxjamLvDaRYG2zBu6js7eohIlvOI3eL1zUl2HebvQuTxVWfgbzJsP6byEoAlInQOdRkJgKfjaLvjHeVJMN3Eki8l8R2SEi20Vkmogk1UyYxtfCggOYfHEq3ZOiue7thXz0y5baDUAEOv/JmXPqunnOeI2fn4OXh8JTPZ2eVDk1PPrcGHPEqjLOYhbO9B5vuJsuAi5U1aFeju2IWM2ienbvL+aqN9L4ed0u7jqtE1cNbIf4qnvr/jxYMwvmvgBb3M80ZRh0Pxc6j3amGjHG1IgaG2chIotUtVdl23zNkkX1FZWUctt7i/l0yVauOLEt/zeyM341sR7G0VKFXetg2TSY/zIUbIPwBGc9jR7nQ0Rz38VmTANRk+tZ7BSRi0TE331dBORUP0RT1wQH+PPU2N6MP741L/2wnj9/sISSUh/O7yQCce1h0B1w6wq45GNo1glm/RX+3RFeG21rahhTS6oyN9RlwDPA4zhTjP8ITPBmUMZ3/PyE+0Z3JSYsiCe+XEN+YTFPj+tNSKC/rwODdoOd17ZlsPJTWPCas6ZGi56QfBy0HQgpQyEg2LexGtMAHVVvKBG5WVWf8EI8R80eQ9W8137cwL3Tl9O/bSwvjU8lIqSOLaNaXAjzX4LV/4MtC6B4H/gFQkwb6Djc6VGV1M96VBlzGF6ZG8rj5JtU1QtTlx49Sxbe8fGiLdz23mI6tYjg1Qn9aBpeR/9qLy12ut5u+AG2LXUeT5UVQ1gz6DQS2pwEiX0gpq3NS2WMB28ni82q6sVpS4+cJQvv+WblDq55awEtokJ5/bJ+JMc28XVIlSvc7fSoWvGJ87V4r7M9vjP0v8ppIA+qB/dhjJdZzcLUqLQNu7js1fkEBfgz+ZK+9GkV4+uQqq60GHakw+Z5sPB12LYEQqKd2W9b9ISWvSChpz2uMo1StZOFiOzhj2tmg7O2dqiqVqVxvNZYsvC+jB17uOzVNLblF/LPc3owpleir0M6cqqw6SdnDEfG13Bgj7M9vLkzhXrzLs54joTuvo3TmFri1ZpFXWTJonbk7j3A1W8uYO76Xdw+rCPXndzBd4P3qksV8jbCxp9g7dewYwVkr3TaOoIjnXaOnudDynAbCGgaLEsWxmuKSkq584MlfLQoi6FdmvOvc3sSFVrHekodrf25sPQD57HVys+dgYDBUdD2JIjv5PSwallDJwXgAAAcOElEQVSnxqMaUy2WLIxXqSqvzNnA3z9fQcvoUCZf0pdOCZG+DqtmlZU6PayWvAdbFkJOBmgptDvZaevwD4To1tD6BGfwoDH1UJ1IFiIyAngS8AdeKr9ut4hcCvwTODh73TOq+pK7bzxwj7v9QVV97XDXsmThGws27uKaNxdSUFTCv87tycjuLXwdkvfsz3XGdSx6B3I3gJbxa7NeUj/oMsaZwyq+oy+jNOaI+DxZiIg/sBoYCmQC84FxqpruUeZSIFVVry93bCyQBqTi/G9cAPRV1dyKrmfJwne25xdyzZsLWLgpj2sHt+e2Ycfg78s5pWpLWakzd9XqmbDgFafmAc7jqqRjf3vFd7KeVqbOqmqyqLRHUwW9onbj/DK/TVXXVXBoPyDj4H4RmQqMAdIrKO9pODBLVXe5x84CRgDvVOFYU8uaR4bwzsTjuPfj5Tw3ey2rtu3hibG96t6I75rm5+8s5NQ0BU64HvKzYNmHzqOrlZ/CL+5EzaGxTjfd5l3hmJEQ2YBrX6bBqkr318eALJxpygUYCyQAq4ApwOAKjksENnu8zwT6H6Lc2SIyEKcWcouqbq7g2D/00xSRicBEgFat6tSwj0YnOMCfh8/qTpeWkdz/STpnP/8jL11yLK3iGtHAt8iWTtI44frfZszNnO/UPBa9DSX74bNbnQWeEvtAhyHOfFbNu4N/neqJbswfVGWK8rmq2r/ctp9V9TgRWayqPSs47lxguKpe4b6/GOinqjd4lIkDClS1SESuBs5T1VNE5M9AsKo+6Jb7K7BPVf9dUZz2GKrumJOxk2vfWoifwORLUjm2TayvQ/K9sjLnMdWqz2F3Jmyc4/S4AggMg6S+0KIXRCVBVLJTW4lp4zSiG+NFNfYYCigTkfOAD9z353jsO1ymyQQ8pwRJwqmh/HawqudU5y8C//A4dnC5Y2dXIVZTBwzo0JSPrhvA5a/N5+KX5/LchX04pVMjX3vCz89p+PZs/M7PcgYIbvrZec19AUoP/LbfP8hZWrbNiU7X3aR+Nt7D+ExVahbtcHo0He9u+gm4BacHU19V/aGC4wJwHi0NccvOBy5Q1eUeZVqo6lb3+zOBO90aSyxOo3Yft+hC91q7KorTahZ1T05BEZdMmcfyrHwmDmzHHcOPIcDfGnorVFYG+3ZC3ibYuQa2L3NqIFsXOz2v/IMhprVT42jZBxL7Ol12A4IhMtEmSDRHpcZqFm4D9agKdh8yUbjHlYjI9cBMnK6zU1R1uYhMAtJUdTpwo4iMBkqAXcCl7rG7ROQBnAQDMOlwicLUTXHhwUy75gQe/Cydyd+tY8XWfJ4Z14eoJvZo5ZD8/CC8mfNK8vi/uz/PqXls/MFJJNmrnckRPSv2oTFOsik9AM06Q3J/SO4H7U929hlTTVWpWSQBTwMDcH46fwBuUtVM74dXdVazqNumztvEXz9eRlJME168pC8dmkX4OqT6rWiPU+PI2+x8vyPdXfRJnCnatyxwGtQDm0D7U5wEEpcCUYkQ285pjDeGml2DexZOTyi3HyAXAReq6tBqR1mDLFnUffM37OLqNxZQVFLG38/qzuie9gvLa0qLIesXp/vupp8hZ60z+vyg5t2cGkdACASGQrtB0OoEiG4FweG+i9vUuppMFotUtVdl23zNkkX9sCVvPze8vZCFm/I4LzWJ+0Z3pUmQdRv1upIi5xHW7kzYugjWfw/F+6GkEPblOBMqHuQXCGHx0KKH0ysrOMrplXWgwNkuftAkzum5FZnozNhrXX/rrZpMFl8Cr/LbgLhxwARVHVLdIGuSJYv6o7i0jCe+XM1zs9fSrmkYT4/rQ5eWDWxeqfpE1XmMlb0Scjc6C0flZznrfuxaD6VFlZxAIKypkzTi2kPn0U4ProiEWgnfVE9NJotWwDM4vaEU+BG4UVU31USgNcWSRf0zJ2Mnt7y7iLz9xfxlZGcuOb51/Z3uvCErLXZqJoFNYP8up2fW3mzYvQXyM2HPNijYDgU7nLaSgu3OcdGtnIb2pH5Oo31UstNm0iTOxo/UId5eKe9mVX3iqCLzEksW9VNOQRG3v7+Yb1ZlM7RLcx49uwcxYUG+DsscrbJSp61k81zntWmuM817eREtnJpI8T4nGYXFOwklLN5JJvtzne8TujlJJ6Kls72sBALs56Mm2bKqpt4oK1OmzFnPP/63kqbhwTxxfi/6t4vzdVimJqjCnq1uu8gm5/u9O51Ze/fudBrTxc/5fm+2UzvZlwMhkVCYzyHH/QY2cZbFDY2B0Ghnoar9uc65Q6MhrJkz5iQo/Lc2lrA4Z44uFPZsd75Gt3bGrSBQlO8MgmwS57TNBIQ4cRTsgL07oCDbic8/yHnk5h/klIlOdtp44js6yS80tt4lM28ni82qmlx5ydpjyaL+W5q5mxveWcimXfu44ZQUbjilgw3ia4zKypwxJwf2OasX7slyugjvz3V+SRfmOWNPfv26G0KinDaSfTnOdlVn+74c5/uDy+cCzhR3cPgJKMoJjnSSRGmJkzTKip1a1KHOERzpJKmIBKe2FOT2LvMPdOLft8vp7hwQ5CS+wFDna/F+5x735zrjZfwCnGP8Ap1JK8G5l4PXPPi9qtMRYfhDR/TP/Ou/Rg1O93EoDWPFJFOndE+K4tMbT+JvHy3jya/W8NPaHJ4Y24uW0aG+Ds3UpoPTuQc1cebMom/1z1lS5PySFoEmTZ3aRu56p3eYiPMLvni/0yYTHOGUbxLr1FLC4v84zYqq8whtzzanbM4ap3a0b5czCr9gh9N2s3WJc15wOgqUFjs1ouBI531xoXOe4n1O0giNcV7+wU5PtaI9HolJ3FH68lu+O7itifdr4hXWLCqYmhycMENVtU71lbOaRcPy4cJM7vloGUEBfjx6dg+GdbWeNcZ4Q1VrFhXW8VU1QlUjD/GKqGuJwjQ8Z/VJ4rMbTyIpJpSJbyzgbx8vo7C4tPIDjTFeYQ+ETZ3VtmkY0645gctPbMvrP23kjGfnkLFjT+UHGmNqnCULU6cFB/jz1z914ZVLj2XHniJGPT2H13/aQFmZNZsZU5ssWZh64eROzZhx00kc2zaWv328nHEv/syGnXt9HZYxjYYlC1NvNI8M4bUJx/Lo2T1I35rPiCe/44Vv11JUYm0ZxnibJQtTr4gI5x2bzKxbBnFSSjyPzFjJsMe/44vl2ziaMUPGmKqxZGHqpYSoEF68JJXXL+tHkL8fE99YwEUvz2XVNmsAN8YbLFmYem1gx3hm3HQS94/uyrIt+fzp6e95Zc56awA3poZZsjD1XoC/H+NPaMPs2wdzUko893+SzqhnfmDx5jxfh2ZMg+HVZCEiI0RklYhkiMhdhyl3joioiKS679uIyH4RWeS+XvBmnKZhiAkL4uXxqTw1rjc7C4o487k5/P3zFew/YA3gxlSX10Zii4g/8CwwFMgE5ovIdFVNL1cuArgRmFvuFGvr2mp8pu4TEUb3bMngY+J5+POVTP5uHTOXb+O+UV0ZfEy8rZdhzFHyZs2iH5ChqutU9QAwFRhziHIPAI8ChV6MxTQykSGBPHxWd96+sj9+Ikx4dT4XvjSXldvyfR2aMfWSN5NFIrDZ432mu+1XItIbSFbVTw9xfFsR+UVEvhWRk7wYp2nATmjflJk3D+S+UV1YsTWfUU//wBNfrmbfgRJfh2ZMveLNZHGo+v6vXVRExA94HLjtEOW2Aq1UtTdwK/C2iPxhkWYRmSgiaSKSlp2dXUNhm4YmKMCPSwe05ctbBzGsawJPfLmGwf+czdR5mygpLfN1eMbUC95MFpmA5wJJSUCWx/sIoBswW0Q2AMcB00UkVVWLVDUHQFUXAGuBjuUvoKqTVTVVVVPj4+O9dBumoYgLD+bZC/rwwdXHkxQTyl0fLmXkU9/zY8ZOX4dmTJ3nzWQxH0gRkbYiEgSMBaYf3Kmqu1W1qaq2UdU2wM/AaFVNE5F4t4EcEWkHpADrvBiraURS28Qy7ZoTeP7CPuwvLuWCl+Zy3dsLycrb7+vQjKmzvNYbSlVLROR6YCbgD0xR1eUiMglIU9Xphzl8IDBJREqAUuBqVd3lrVhN4yMinNa9BSd3asZ/vl3Hc7Mz+HrFDq4/pQOXn9iWkEB/X4doTJ1yVGtw10W2Up6pjs279vHgZ+nMXL6dxOhQbh/ekTE9E/Hzs662pmGr9kp5xjQmybFN+M/Fqbx1RX9iwgK55d3F/OnpH/h+jXWcMAYsWRjzOwM6NGX6dSfy5Nhe5BcWc/HL87j45bksz9rt69CM8SlLFsaU4+cnjOmVyFe3DeKe0zuzdMtu/vT0D9z67iIyc/f5OjxjfMLaLIypxO79xTw/ey2vzFmPAuOPb81Vg9rTNDzY16EZU21VbbOwZGFMFWXl7eexWav5cGEmQQF+XNCvNVcNakfzyBBfh2bMUbNkYYyXrMsu4Nlv1vLRoi34i3B230SuHdyB5Ngmvg7NmCNmycIYL9uUs48XvlvLBwsyQeHSAW24bnAHopoE+jo0Y6rMkoUxtSQrbz///mI1H/6SSWRIINcMbs9Fx7UmPNhrY16NqTGWLIypZelZ+Tzyv5V8tzqbyJAAxp/QhgkD2hIbFuTr0IypkCULY3xk8eY8npudwczl2wkL8mfiwPZcObAtTYKspmHqHksWxvjYmu17eGzWamYs20aziGBuGdqRs/skERRgw5tM3WHJwpg6YsHGXTz02QoWbsojITKECQPaMK5/KyJDrCHc+J4lC2PqEFVl9upsXvxuHT+uzSEsyJ+x/VoxYUAbkmKsy63xHUsWxtRRy7bs5qXv1/HJkq0AjOzegokntaN7UpSPIzONkSULY+q4rLz9vPrjBt6eu4mCohKObxfHxIHtGNQx3qZGN7XGkoUx9UR+YTHvztvMlDnr2bq7kJRm4Vw5sB1jerUkOMAWYTLeZcnCmHrmQEkZny3N4j/frmPltj00iwjm0gFtuLB/a6JCrTHceIclC2PqKVXl+zU7efH7dXy/ZidhQf6cf2wrLjvRGsNNzbNkYUwDsDxrNy99v55PFmehwOndWzBxYDu6JVpjuKkZdWJZVREZISKrRCRDRO46TLlzRERFJNVj293ucatEZLg34zSmruraMorHz+/Fd3eczGUD2vD1yh386ekfGDv5J2Ys3UpJaZmvQzSNhNdqFiLiD6wGhgKZwHxgnKqmlysXAXwGBAHXq2qaiHQB3gH6AS2BL4GOqlpa0fWsZmEag/zCYt6Zu4nXf9rIlrz9JESGcEH/Voztl0yzCFtXwxy5ulCz6AdkqOo6VT0ATAXGHKLcA8CjQKHHtjHAVFUtUtX1QIZ7PmMatciQQK4a1J7v7jiZFy9JJaV5OI/NWs2AR77m1vcW2Vrhxmu8ObNZIrDZ430m0N+zgIj0BpJV9VMRub3csT+XOzax/AVEZCIwEaBVq1Y1FLYxdZ+/nzC0S3OGdmnOuuwCXvtxA+8vyOTDhVs4rl0sl5/YjiGdmtl4DVNjvFmzONRP6a/PvETED3gcuO1Ij/11g+pkVU1V1dT4+PijDtSY+qxdfDj3j+nGT3cP4e7TOrEpZx9Xvp7GKf+ezWs/bmBvUYmvQzQNgDeTRSaQ7PE+CcjyeB8BdANmi8gG4DhgutvIXdmxxphyokKdR1Tf3nEyT4/rTXSTIO6dvpzjH/6Khz9fQVbefl+HaOoxbzZwB+A0cA8BtuA0cF+gqssrKD8buN1t4O4KvM1vDdxfASnWwG3MkVmwMZcpP6xnxrKtiAgju7fg8hPb0is52tehmTqiqg3cXmuzUNUSEbkemAn4A1NUdbmITALSVHX6YY5dLiLvAelACXDd4RKFMebQ+raOoW/rGDJz9/HajxuYOm8znyzOoldyNIOPiWdEtwQ6JUT6OkxTD9igPGMakYKiEt5P28y0hZmkZ+VTptCxeTije7ZkWNcEUpqFI2KN4o2JjeA2xhzWzoIiZizdyvTFWczfkAs4iePC/q05s0+iLc7USFiyMMZU2dbd+/lqxQ7eS9vMkszdBAX4ccoxzRjVsyVDOjcjJNBmv22oLFkYY47Kksw8Ply4hc+WbiV7TxFhQf6M7N6Cs/ok0b9trI3daGAsWRhjqqW0TJm7Lof//rKFGcu2UVBUQmJ0KGf1SeTM3om0iw/3dYimBliyMMbUmP0HSvkifRsfLtzC92uyKVPo3SqaMT1bclr3FjSPtHmp6itLFsYYr9ieX8jHi7bw4cItrNy2BxFIbR3Dad1acFr3BFpEhfo6RHMELFkYY7wuY8cePl+6jc+XbmXltj2AM7bjtG4JjOzegpbRljjqOksWxphatTa7gBlLt/LZ0m2s2JoPQK/kaE7v7tQ4bJW/usmShTHGZ9bv3MvnS7fy+dKtLM9yEkfPpChGdm/ByO4tSI61xFFXWLIwxtQJG3buZcYy51HV0i3OehvdE53EMbxrc9o2DbNR4z5kycIYU+dsytnHjGVOjWNxppM4WkaFcEKHpozomsAJHeJoEuTNZXZMeZYsjDF12uZd+5i9agc/rcvhhzU7yS8sIdBf6N0qhlM6NWNYl+Y2lqMWWLIwxtQbB0rKmLs+hx8ydvL96p2kuw3k7ePDOCklnh5JUfRIiqZd0zAbQV7DLFkYY+qtLXn7+TJ9O7PSt7NgYy77i50VCiKCA+jpTq8+tEtzWseF+TjS+s+ShTGmQSgtUzJ2FLA4M4+lmbuZuz6H1dsLAEhpFs6Qzs0Z2LEpfVrF2ISHR8GShTGmwdqUs48vV2znyxXbmbd+FyVlSlCAH6mtYxjUMZ7TurWgVZx1z60KSxbGmEYhv7CY+et38ePaHOZk7Px1JHmnhAgGH9OMwcfE07d1DIH+fj6OtG6yZGGMaZQ279rH/5Zt48sVTntHSZkSHhzAgA5xDD6mGYM6xts0JB4sWRhjGr09hcXMycjh29U7+HZVNlm7CwFnRcBBHeM5pVNzUts07lpHnUgWIjICeBLwB15S1UfK7b8auA4oBQqAiaqaLiJtgBXAKrfoz6p69eGuZcnCGHM4qsqaHQV8uyqb2at3MH99LgdKywgJ9KN7YhS9kqPp2zqG/m3jiAkL8nW4tcbnyUJE/IHVwFAgE5gPjFPVdI8ykaqa734/GrhWVUe4yeJTVe1W1etZsjDGHIm9RSV8vyabeetzWbQ5l2VZ+RwoKUMEOidEckL7OPq3i6Nfm1iimjTc9cirmiy8Oa6+H5ChquvcgKYCY4Bfk8XBROEKAxrGMzFjTJ0XFhzAiG4tGNGtBeAMDFy6JY8fM3L4cW0Or/+0kZd+WI8IdEqIpH/bWI5rF0u/tnHENqKax0HeTBaJwGaP95lA//KFROQ64FYgCDjFY1dbEfkFyAfuUdXvD3HsRGAiQKtWrWoucmNMoxMU4Eff1rH0bR3LDUNSKCwuZdHmPOau28Xc9TlMnb+JV3/cADjjO1LbxHJsmxhSW8eSHBva4CdD9OZjqHOB4ap6hfv+YqCfqt5QQfkL3PLjRSQYCFfVHBHpC3wEdC1XE/kdewxljPGmAyVlLMnMY+76XczfsIsFG3LZU1QCQLOIYFLdxJHaJoYuLSIJqCeN5nXhMVQmkOzxPgnIOkz5qcDzAKpaBBS53y8QkbVAR8CygTHGJ4IC/EhtE0tqm1jAGVm+evse0jbmkrZhF2kbcvl86TYAQgP96d0qmtTWMfRtE0ufVtFEhNTvdg9vJov5QIqItAW2AGOBCzwLiEiKqq5x354OrHG3xwO7VLVURNoBKcA6L8ZqjDFHxN9P6Nwiks4tIrn4uNYAbN29n7QNuSzYmMv8Dbt45psMyhT8BI5JiCS1dYxTA2kTS2I9G+vhtWShqiUicj0wE6fr7BRVXS4ik4A0VZ0OXC8ipwLFQC4w3j18IDBJREpwutVeraq7vBWrMcbUhBZRoYzqGcqoni0BKCgqYdGmPNI2OjWPDxdm8sbPG92yIU5NpXUMfVvH0LlFJP51eEZdG5RnjDG1pKS0jJXb9jiPrTbmkrYhl235zkDB8OAAerdyxnoc2yaWXsnRhAV7fyEon4+zqG2WLIwx9Y2qsiVv/6+PrdI25LJq+x7UfXSV0iyCnsnOWh49k6I5JiGCoICabTi3ZGGMMfVQfmExCzfm8sumPBZn5rF4cx65+4oBCPL3o3PLSHokRtEjKYqeydG0jw+v1uMrSxbGGNMAqCqZuftZnJnHkszdLMnMY9mWfArcbrtNgvw5pVMznrmgz1Gdvy50nTXGGFNNIkJybBOSY5vwpx5Ow3lZmbJuZ4GbPHYTFuz9RZ8sWRhjTD3j5yd0aBZBh2YRnNUnqXauWStXMcYYU69ZsjDGGFMpSxbGGGMqZcnCGGNMpSxZGGOMqZQlC2OMMZWyZGGMMaZSliyMMcZUqsFM9yEi2cDGapyiKbCzhsKpSxrqfUHDvbeGel/QcO+tPt9Xa1WNr6xQg0kW1SUiaVWZH6W+aaj3BQ333hrqfUHDvbeGel+e7DGUMcaYSlmyMMYYUylLFr+Z7OsAvKSh3hc03HtrqPcFDffeGup9/craLIwxxlTKahbGGGMqZcnCGGNMpRp9shCRESKySkQyROQuX8dTXSKyQUSWisgiEUlzt8WKyCwRWeN+jfF1nJURkSkiskNElnlsO+R9iOMp9zNcIiJHt75kLang3u4TkS3u57ZIREZ67LvbvbdVIjLcN1FXTkSSReQbEVkhIstF5CZ3e73/3A5zb/X+c6syVW20L8AfWAu0A4KAxUAXX8dVzXvaADQtt+1R4C73+7uAf/g6zircx0CgD7CssvsARgIzAAGOA+b6Ov6juLf7gNsPUbaL+3MZDLR1f179fX0PFdxXC6CP+30EsNqNv95/boe5t3r/uVX11dhrFv2ADFVdp6oHgKnAGB/H5A1jgNfc718DzvBhLFWiqt8Bu8ptrug+xgCvq+NnIFpEWtROpEeugnuryBhgqqoWqep6IAPn57bOUdWtqrrQ/X4PsAJIpAF8boe5t4rUm8+tqhp7skgENnu8z+TwPwD1gQJfiMgCEZnobmuuqlvB+aEHmvksuuqp6D4ayud4vfs4ZorHo8J6eW8i0gboDcylgX1u5e4NGtDndjiNPVnIIbbV977EA1S1D3AacJ2IDPR1QLWgIXyOzwPtgV7AVuDf7vZ6d28iEg5MA25W1fzDFT3Etvp2bw3mc6tMY08WmUCyx/skIMtHsdQIVc1yv+4A/otT9d1+sHrvft3huwirpaL7qPefo6puV9VSVS0DXuS3Rxb16t5EJBDnl+lbqvqhu7lBfG6HureG8rlVRWNPFvOBFBFpKyJBwFhguo9jOmoiEiYiEQe/B4YBy3DuabxbbDzwsW8irLaK7mM6cInbu+Y4YPfBxx71Rbln9WfifG7g3NtYEQkWkbZACjCvtuOrChER4GVghao+5rGr3n9uFd1bQ/jcqszXLey+fuH0yFiN01vhL76Op5r30g6nB8ZiYPnB+wHigK+ANe7XWF/HWoV7eQenWl+M81fa5RXdB06V/1n3M1wKpPo6/qO4tzfc2Jfg/KJp4VH+L+69rQJO83X8h7mvE3EetSwBFrmvkQ3hczvMvdX7z62qL5vuwxhjTKUa+2MoY4wxVWDJwhhjTKUsWRhjjKmUJQtjjDGVsmRhjDGmUpYsjDHGVMqShTHGmEr9P8TKErkgD7DEAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEICAYAAAB1f3LfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzsnXd8FVX2wL8nvSeQhF6SANIRMIAUJSIWLFjWupYVC7qru+y6a9nmrnUtv3VX175r31XWsipWbIAiIgREqkgILdQktBRC2v39ceflTcLLey8hDXK+n8983sydO3fOzHtvzpxz7j1XjDEoiqIoSmMJaW0BFEVRlCMbVSSKoijKYaGKRFEURTksVJEoiqIoh4UqEkVRFOWwUEWiKIqiHBaqSBSlAYhIlojkNWP7T4nIH13bPxWRnSJSLCLJzmdGM5x3lYhkNXW7SvtAFYnSIEQkTkQ2isiPXWXxIrJZRC5wlWWKyHsiskdE9orIahG5V0Q6OPuvEpEq58FYLCK5IvLTZpY9KCUgIqNF5ANH7t0iskhEpjWnbB6MMTcYY+525AgHHgZONcbEGWMKnc/cwzmHiLwgIvfUOe9gY8zcw2m3nnPNFZEy1/dcLCLvNvV5lNZFFYnSIIwxxcB04BERSXWKHwSyjTFvAIjIOGAu8BUwwBiTBJwOVALHupr72nkwxgEXAA+KyIiWuRLfiMhY4HNgHtAXSAZ+CkxpBXE6A1HAqlY4d1Nyk+d7dpazfVUSkbBgyvzR0PpKE2GM0UWXBi/AC8CrQBZQCHR17ZsP/CPA8VcB8+uULQJ+7Nqein2I7sUqpoGufQOdsr1OnamufWcAq4EiYCvwGyAWOABUA8XO0s2HXPOBx/3InQXkubZvB9Y751oNnOfa1xerkPYBBcB/nXIB/gbscvYtB4a47us9wDFACWAcWT939hugr7MeDfwV2OS0Mx+Idva9Duxwyr8ABjvl04EKoNxp912nfCMw2VmPBP4ObHOWvwOR7usHfu3Ivx2Y5ud+zQWu9XcvgdscWV/2VebUvQ7IAXYDs9zfnXNPbgTWARta+7/RHpdWF0CXI3MBOjgPkQL3g8R5YFcBWQGOvwqXIgFGYZXCMc6250F6ChAO3Oo8SCKc7Rzgd872JOdB3t85djtwgkvOkc56Fi4l4EOmGEf2k/zUqdUGcCHQDWvdX+zI3NXZ9yrwe2dfFDDBKT8NWAIkYZXKQNcxLwD3OOtpzkMyzHU+tyJ53HlQdwdCgXGuB/7VQDxepbDM1UbNOVxlG/EqkruAhUAnIBVYANztuv5Kp044VmmXAh3quV9z8a9IKoEHHDmj6ymbhP2djXTK/gF8UeeefAJ0xFGkurTw86C1BdDlyF2AT52HSKKrrIfzxx7gKnsQqyRKgD84ZVc5D4y92Ddj4zwgxNn/R+A1VxshWOsiCzgB+7Ya4tr/KvBnZ30zcD2QUEfeLPwrku51ZfdRJ1Aby4BznPWXgGeAHnXqTAJ+AI53X4Ozr+Yhjx9F4tyPA8CxQXxPSc5xiXXP4aqzEa8iWQ+c4dp3GrDRdf0H6si0Czi+nnPPdX4je12LWymVA1F17m/dsmeBB13bcVirKs11Tya19v+hPS8aI1EahYhcjn3QfYp9e/SwB+s+6uopMMbcamyc5C3A7cNeaIxJMjZG0gUYDNzn7OuGddl42qgGtmAf9t2ALU6Zh03OPoAfYd+UN4nIPCfuEQyHyB4IEblSRJY5gfm9wBAgxdl9K9biWOT0irrauZbPgcewFsVOEXlGRBKCPadDCtbKWe9DplARuV9E1ovIfqyS8BwTDLXuvbPezbVdaIypdG2XYh/u9fEL53v2LH907cs3xpTVqV+3rO5voRjrTu3uqrPFz/mVZkYVidJgRKQT1sd/HfbN/yIRORHAGFMCfAOc35A2jTE7gTcBTyB2G9DbdU4BemKtkm1ATxFx/357Ofswxiw2xpyDdc28DbzmOU0AGUqBr7GKKCAi0hv4J3ATkOwoy5VY5YExZocx5jpjTDfsfXpCRPo6+x41xhyHVZ7HALcEc04XBUAZ0MfHvh8D5wCTgUSswscjFwHuA3XuPfbebmugfMHiS5a6ZXV/C7HYThBbA7SjtBCqSJTG8BjwtjFmjjFmO/bN+58iEunsvxW4WkRud5QOItIDSK+vQRFJBs7D20PpNeBMETnZ6Qb7a+Ag1l//DdZNdquIhDvjH84GZopIhIhcJiKJxpgKYD827gGwE0gWkUQ/13YrcJWI3OLIhIgcKyIzfdSNxT7A8p1607AWieeaLnSuG6y1Y4AqERklImOc6yrBKoQqGoBjjT0HPCwi3RwrZKzzHcQ796oQG/e5r87hOwF/Y1FeBf4gIqkikgLcAfy7IfI1Ma8A00RkuHN99wHfGGM2tqJMigtVJEqDEJFzgQm43qCNMf/C9rS5w9mej40DnAj84Lh8PsL6y//ham6sZ2wBsAb7QP6508Za4HKnfgFWUZxtjCk3xpRje3RNcfY9AVxpjPneafcKYKPj1rnBaQdn/6tAruOKcrtrPNeywJF9klNvNzbO8YGPuquxvaa+xj6ch2K7PHsYBXzjXN8sYIYxZgOQgLVk9mBdNoXA//m43YH4DbACWIztzfQA9j/9ktPuVmxPsoV1jnsWGOTcg7d9tHsPkI3tTbYCWOqUNZbH6owjWdKQg40xn2FjZm9iO1L0AS45DHmUJsYT2FQURVGURqEWiaIoinJYqCJRFEVRDgtVJIqiKMphoYpEURRFOSzaRYKzlJQUk5aW1tpiKIqiHFEsWbKkwBiTGqheu1AkaWlpZGdnt7YYiqIoRxQisilwLXVtKYqiKIeJKhJFURTlsFBFoiiKohwW7SJGoiiKEiwVFRXk5eVRVlY3KfHRS1RUFD169CA8PLxRx6siURRFcZGXl0d8fDxpaWnYpNNHN8YYCgsLycvLIz293ryqflHXlqIoiouysjKSk5PbhRIBEBGSk5MPywJTRaIoilKH9qJEPBzu9aoi8cd3/4XFz7a2FIqiKG2aZlUkInK6iKwVkRwRud3H/htEZIUzVel8ERnk2vdb57i1InJasG02KSvfgKUvNespFEVR3BQWFjJ8+HCGDx9Oly5d6N69e812eXl5UG1MmzaNtWvXNrOkXpot2C4iodg5qU/BTnq0WERmOZMBeXjFGPOUU38q8DBwuqNQLsFOQ9oN+FREjnGOCdRmE15EKJgGTVynKIpyWCQnJ7Ns2TIA/vznPxMXF8dvfvObWnWMMRhjCAnxbQs8//zzzS6nm+a0SEYDOcaYXGdGu5nYeaRrMMbsd216pi3FqTfTGHPQmVEux2kvYJtNSkgoVFc3W/OKoijBkpOTw5AhQ7jhhhsYOXIk27dvZ/r06WRmZjJ48GDuuuuumroTJkxg2bJlVFZWkpSUxO23386xxx7L2LFj2bVrV5PL1pzdf7sDW1zbecCYupVE5EbgZiACO72p51j39KB5ThnBtNlkSIhaJIrSjrnz3VWs3rY/cMUGMKhbAn86e3Cjjl29ejXPP/88Tz31FAD3338/HTt2pLKykpNOOokLLriAQYMG1Tpm3759TJw4kfvvv5+bb76Z5557jttvb9qoQHNaJL66ARwyr68x5nFjTB/gNuAPAY4Nqk0AEZkuItkikp2fnx+kyHUICYVqVSSKorQN+vTpw6hRo2q2X331VUaOHMnIkSNZs2YNq1cf6uWPjo5mypQpABx33HFs3LixyeVqToskD+jp2u4BbPNTfybwZBDHBtWmMeYZ4BmAzMzMxk1MrzESRWnXNNZyaC5iY2Nr1tetW8cjjzzCokWLSEpK4vLLL/c5FiQiIqJmPTQ0lMrKyiaXqzktksVAPxFJF5EIbPB8lruCiPRzbZ4JrHPWZwGXiEikiKQD/YBFwbTZpKhFoihKG2X//v3Ex8eTkJDA9u3bmT17dqvJ0mwWiTGmUkRuAmYDocBzxphVInIXkG2MmQXcJCKTgQpgD/AT59hVIvIasBqoBG40xpoGvtpsrmsgJEwViaIobZKRI0cyaNAghgwZQkZGBuPHj281WcSYxnl9jiQyMzNNoya2eucmyPkUfv190wulKEqbZM2aNQwcOLC1xWhxfF23iCwxxmQGOlZHtvtDXVuKoigBUUXiDw22K4qiBEQViT/UIlEURQmIKhJ/SCgYHdmuKIriD1Uk/lCLRFEUJSCqSPyhKVIURVECoorEH2qRKIrSwjRFGnmA5557jh07djSjpF50znZ/aK8tRVFamGDSyAfDc889x8iRI+nSpUtTi3gIqkj8EeIE242Bdjb1pqIobY8XX3yRxx9/nPLycsaNG8djjz1GdXU106ZNY9myZRhjmD59Op07d2bZsmVcfPHFREdHs2jRolo5t5oaVST+kFD7aaq964qitB8+vB12rGjaNrsMhSn3N/iwlStX8tZbb7FgwQLCwsKYPn06M2fOpE+fPhQUFLBihZVz7969JCUl8Y9//IPHHnuM4cOHN638PlBF4g/P7GPVVdY6URRFaSU+/fRTFi9eTGamzVhy4MABevbsyWmnncbatWuZMWMGZ5xxBqeeemqLy6aKxB81FonGSRSlXdIIy6G5MMZw9dVXc/fddx+yb/ny5Xz44Yc8+uijvPnmmzzzzDMtKpv22vKHxwrRnluKorQykydP5rXXXqOgoACwvbs2b95Mfn4+xhguvPBC7rzzTpYuXQpAfHw8RUVFLSKbWiT+CHFuT3XTTwSjKIrSEIYOHcqf/vQnJk+eTHV1NeHh4Tz11FOEhoZyzTXXYIxBRHjggQcAmDZtGtdee22LBNs1jbw/Fj4FH90Gt26AmI5NL5iiKG0OTSPvRdPINwXq2lIURQmIKhJ/iHN7NNiuKIpSL6pI/KEWiaK0S9qDy9/N4V5vsyoSETldRNaKSI6I3O5j/80islpElovIZyLS2yk/SUSWuZYyETnX2feCiGxw7Wu+0Tba/VdR2h1RUVEUFha2G2VijKGwsJCoqKhGt9FsvbZEJBR4HDgFyAMWi8gsY8xqV7VvgUxjTKmI/BR4ELjYGDMHGO600xHIAT52HXeLMeaN5pK9BrVIFKXd0aNHD/Ly8sjPz29tUVqMqKgoevTo0ejjm7P772ggxxiTCyAiM4FzgBpF4igMDwuBy320cwHwoTGmtBll9Y07RYqiKO2C8PBw0tPTW1uMI4rmdG11B7a4tvOcsvq4BvjQR/klwKt1yu513GF/E5FIX42JyHQRyRaR7Ea/WahFoiiKEpDmVCS+0uX6dDqKyOVAJvBQnfKuwFBgtqv4t8AAYBTQEbjNV5vGmGeMMZnGmMzU1NSGSw/aa0tRFCUImlOR5AE9Xds9gG11K4nIZOD3wFRjzME6uy8C3jLGVHgKjDHbjeUg8DzWhdY8qEWiKIoSkOZUJIuBfiKSLiIRWBfVLHcFERkBPI1VIrt8tHEpddxajpWCiAhwLrCyGWR3Tqa9thRFUQLRbMF2Y0yliNyEdUuFAs8ZY1aJyF1AtjFmFtaVFQe8bvUCm40xUwFEJA1r0cyr0/R/RCQV6zpbBtzQXNegFomiKEpgmjVpozHmA+CDOmV3uNYn+zl2Iz6C88aYSU0oon+015aiKEpAdGS7P9wTWymKoig+UUXiD00jryiKEhBVJP7QYLuiKEpAVJH4Q4PtiqIoAVFF4g+1SBRFUQKiisQfNRaJ9tpSFEWpD1Uk/tAUKYqiKAFRReIPjZEoiqIERBWJPzRGoiiKEhBVJP5Qi0RRFCUgqkj8oSlSFEVRAqKKxB9qkSiKogREFYk/tNeWoihKQFSR+EMtEkVRlICoIvGH9tpSFEUJiCoSf6hFoiiKEhBVJP6oSSOvikRRFKU+/CoSEQkRkXGNbVxETheRtSKSIyK3+9h/s4isFpHlIvKZiPR27asSkWXOMstVni4i34jIOhH5rzMffPOgwXZFUZSA+FUkxphq4K+NaVhEQoHHgSnAIOBSERlUp9q3QKYxZhjwBvCga98BY8xwZ5nqKn8A+Jsxph+wB7imMfIFhbq2FEVRAhKMa+tjEfmRiEgD2x4N5Bhjco0x5cBM4Bx3BWPMHGNMqbO5EOjhr0FHhklYpQPwInBuA+UKHg22K4qiBCQYRXIz8DpQLiL7RaRIRPYHcVx3YItrO88pq49rgA9d21Eiki0iC0XEoyySgb3GGM/ct/W2KSLTneOz8/PzgxDXB2qRKIqiBCQsUAVjTHwj2/ZlwRifFUUuBzKBia7iXsaYbSKSAXwuIisAXwrMZ5vGmGeAZwAyMzN91gmIWiSKoigBCahIAERkKnCisznXGPNeEIflAT1d2z2AbT7angz8HphojDnoKTfGbHM+c0VkLjACeBNIEpEwxyrx2WaToRNbKYqiBCSga0tE7gdmAKudZYZTFojFQD+nl1UEcAkwy11BREYATwNTjTG7XOUdRCTSWU8BxgOrjTEGmANc4FT9CfBOELI0Du21pSiKEpBgLJIzgOFODy5E5EVsb6tDuvO6McZUishNwGwgFHjOGLNKRO4Cso0xs4CHgDjgdSeWv9npoTUQeFpEqrHK7n5jzGqn6duAmSJyjyPHsw264oYgYpWJxkgURVHqJSjXFpAE7HbWE4Nt3BjzAfBBnbI7XOuT6zluATC0nn252B5hLYOEqkWiKIrih2AUyV+Ab0VkDjaAfiLw22aVqi0REqoWiaIoih/8KhJn3MZ84HhgFFaR3GaM2dECsrUNJFQntlIURfGDX0VijDEi8rYx5jjqBMrbDWqRKIqi+CWYAYkLRWRUs0vSVpEQjZEoiqL4IZgYyUnA9SKyCSjBureMkx/r6EctEkVRFL8Eo0imNLsUbZmQMKiuDFxPURSlnRIo2B4CvG+MGdJC8rQ9tPuvoiiKXwIF26tF5DsR6WWM2dxSQrUpQkL9p0jZuxliUyE8uuVkCpaSAtixHDpmQIe01pam/VG621qzcZ1aW5LAbF0KZXttTLDHaIiI8f5+6iMkDHoeD2HNNyWQcmQQjGurK7BKRBZhYyQA1Jkj5OjFX7DdGPj7UMjIgiubL1NLo/nfdbD+c4jvCjevsSP1lZbjzWuhtACu/6K1JfHPjpXwz5O82+NnwCl3eX8//jj1Xhh3U/PKp7R5glEkdza7FG0Zf8H2SifHZO7cFhMnaMpLYeN8iOsCRdth12roPLi1pWo/lJfCxi+hqhyKdkJ859aWqH5yPrWfl70B8x6AnM9h4u3293PspXDcVb6Pe+cmWP+ZKhKlfkUiIgOMMd8bY+aJSKQ7M6+IHN8y4rUB/MVIKkp9l7cFtiy0D7Gs2+G9X1plp4qk5fDcf4ANX8CwC1tXHn9smAepA6DfKbD9O/j8bvj+fSv/kB9Br3r+7n1PhiUv2heqsMiWlVlpU/izSF4BRjrrX7vWAZ6os3304s8iKS/xXd5aHCyGiFjrwsqda33YQy+Erx+z22NvtPXKS2y9+qiuhn2bgxvRH9/10PhQoPbbMvu2QtVBCI2ERD/zsBkD+7ZAeAzEphy6P3cuhITb/T98CD2OC3zuhB423hDM/U/sBaEBHAoVByAs6lCXpjGwdxNEJtjvadPXcNxP7L6Mk6wimXOvlb/X2Prbz8iCb56CLYsg/QSoLAeMPc4jf2SC7/sTDOUl9v6pS7bN4++XKPWs+9o+Klm0YTeDKwyx9f2h3RZJ2T6ICjqfZdNTuhv+Pgym3A8jLodNC6B7JkTGQfpEWP5f+wDZtRqeOgGu/RS61/Mu8OX/2QdJMPQeD9NceTm3LoF/nWLjAl2OsM5+338AMy/1bl86E/rX0/t96Yvw7gxrsf5yxaFKZ9MC6JEJcZ1h5Zt2CcSgc+GiF4O7/6OnwxkP1b//YBE8PAjO+hsMvaD2vgWPwid32If0eU9D5QH7GwHoNhyiO8CeDdB7gv391Efv8fb6c+daRfLW9TYmlHaCV/767k8gSnfDI8fCmQ+3bWtOAfwrElPPuq/to5In5+bwh6IK+nSqzyIp9q7v2QRdW3GM5oZ5UF5kXRIjLofCHBh8nt3XaaCVtXgn5K+1rrq1H9avSNZ+CJ0Gw/hf+D/nuo9h1VtwYI99+IB9GJsq29vnSFMkP3wIkYlwxoPwwS3ww0f1K5K1H9lPUwX5aw59UHru/4m31t+GmxVvQM5nUFUZ+P5nP2dl86dICtfDwf2w7dtDFUnBOvtZUWof+BIKaeNtWUgoXPE25H9fv0vLQ1QCdD/O/vaqbocfZlultH+7lX/g2TDvft/3JxC5c63825epIjkC8KdIeojIo1jrw7OOs93AX8WRSXREKJXGT6+tcpdFsmdj6yoST8B/43z7NndgDyT1tmWerr97NkKJM3/9hnnYiSnrcGCP/fOeeCsce4n/cyb1sm/aG+fbh0ZNu865jjRy59k362MvgdXv2G1fVFXaaz5milU+da+1bJ/3/id0DXwfwbqgcj6xvaQC3f+DRfDBb2D3BuiY7ruORyZf30NJPqQOtK65/O9td1+3Nd1tuF2CISPLWlDrP4cKx9VbuM4G64+7yiqSxvwWjuTfUTvEX66tW4AlQLZr3bN9a/OL1vpEh4dRaaT+GInbtbVnQ8sIVRdj7JI7DyLi7FvcqrfsPo8C8aVI8rKhbP+h7W2cb33bGRMDn7t7JoTHWiVmDBzYa11bnnP5k9ezuMuCoe7xTbXszrVxA4+LJ32i/U73bDy07tYl1vobdpFVAO46YK1TaNjYnfQTAbHxiUD33yOjv96CbkVS9/6W5FsFlzbBbgfzXddHxkQr75x7rfxh0d7yuM7e+9NQPNfmOdbXd6a0Geq1SIwxL7akIG2R6IgQq0jqtUhcwfZP7oCiHXD6Xxp/ws0L4Y2r4brPIb5L4Pob58NL50J1hd2eeJvtvrnU+eo8D7LEnoDYB1yxM6OxqbJ+/P6n124zd65VDt0zA58/LMK6RBb/yy4eIuJg5yp4ZDicdi8MONOWv3czZLsmtIzrApf8B144yw7cu/Idr4slmOttDjwP1Yws+/nIsfVUFFsnqZf93u7vbd2HFz5PTQixIYokpqO1aLd/F/j+p/SD+G72u8qcZsvWvAef/BGu/9LGNWoUySZ47nTbi2zUdXDm/0FxPiT3s1bHDx95FVNj6DHKxlq2fwfdRtjBuRu/svKHhNj7U1eRvHQO9Jlkx6vM/7u9jlHXeuUvLbDHRMRZ+Y2x/4tV//O2kdwPbvzGuuKUVifYGRIbhYicDjyCnWr3X8aY++vsvxm4FqgE8oGrjTGbRGQ48CSQAFQB9xpj/usc8wIwEdjnNHOVMWZZc8gfExFGhT+LxKNIzvg/WPRPO27gcFj5P9i/FdZ9AiOvCFx/9SzbM+vE30BohP0zrv3A/qnB+yALj4KEbvbPWbYPkvvCvjz7Bz5EkcyzD/NgRyufcpf1k3uI7mDPv+w/dnvFG1aRVFdZN1iPUdB3sh3bsuQFmP0761cPjYA17/pXJO7rbQ4SukFqf7ue2h/OftTK6Yvkvvbh3yHNxorA9vRa9bZ9oELDswmc+Tfr3uo63P/9F0eJ/fCR7eEVEgIrXrdW1aav4JjTvA/v8iKrRMKi7P2f8qC1SGJTYOSV1qXlsUwaQ1gkXPCc/c77nQIR8fY37JG/Q1ptRbJ3i7dH4fgZ1oWVO8da9x75i5zpjoZeYH8jpbvtS0S3kfbaCtfDitdsHM5zr5VWpdkUiYiEAo8DpwB5wGIRmeWaex3snOuZxphSEfkp8CBwMVAKXGmMWSci3YAlIjLbGLPXOe4WY8wbzSW7h6hwGyMx1VW+u6l5XFuDz7dBbE/PqMZ2V/T4hTfMC06RbJgHvcfZsSIeMrJgxwqISoLoJG+55w9dXQGJPayVUtc1sm+r9W/XNwDNF50G2sXNPFcQeMMX9mG3Y7lNwTF6unUJVRyAZa/Clm+gy1CISfFef0Out7kQ8XaJ9YdHWcR1sdZMzmf24Vz3/gdDj+OC6yYM9lzfvQI7V0DnofY+g30ROOY066aLiLeKBGDMDfDV361SqTxg07ZExMLwHzdMRl/0n1K7Q0HqMd71DmnWYvP8L+rGPjyfW77xyl+8w97PfqdZRbJrFZTsgjHX25eIop1WkeTOU0XSRghmPpLGMhrIMcbkGmPKgZnAOe4Kxpg5xhhPoGEh0MMp/8EYs85Z3wbsAlKbUVafxESEUmVCqA5kkUTE2D/Mwf02yNoY9m+3gc+QcPsHCeQD9tTPyKpdnu5s130b9iiSknyI7WSPy19j/5QePH/yum02FM+5Q8Ktm2LXKq/S8rhRwqO9vYIysuyya3VtedzUd72tjedaMyZa2UoLbK+r5s5tVhMnmWeVyYHdzm9nrrX+9m62HQfAft9jrrfry19zylro71T3f+H5HezdbMed7HWl8AsJt9ZJ7jx7Pz0dCTydHjz3NL4zdBrUNjNKtFMCWiQikgpcB6S56xtjrg5waHdgi2s7Dxjjp/41wIc+zj8aiADWu4rvFZE7gM+A292j7l3HTQemA/Tq1SuAqL6JiQilihCqqyrx6Yktd6ZnCYuqHdCO6djwk3neKDOvhkVP226bn91pe+8secG+1bmpLLOfdQOlvcfaP2SH3rXLk3pD0TabO6z/md7jNsyzFsJHv4Vv/20tg06DGi6/G8+98FzLS+dARZlt150qJCPLnj89C2KTa8vjYfGz1o9e3/W2NjWKJMv7cC/eAb38/dSbgISukNIf5t5vLQ3w3u+/D7Mxp/QTraszY6J126X0h9Vv27qxLZRI0tNz8Mlx9ndZtN26MavKbaeF6krvtkd+cOJPzrG5c+xnB1cPtfSJsOR5+7uqKIWXpsKBfdbqOfVuGFTrnbVl2P6djeVUlluX30UvtptsEsG4tt4BvgQ+xcYrgsWXf8fna7aIXA5kYmMf7vKuwMvAT4ypGRX4W2AHVrk8A9wG3HXIiYx5xtlPZmZmo7p4RIVbRWKq6pmPpKLUO5LcrUjqG5/hj9y5EN3Rjj5f9LR9QOTOtX+ydR/bLprJfWsfk9DNujXcRMTC2Y9AyjG1y7s6QWNTDXGp0GWYjWfkzoOBU+3YhJR+MG6G9bkfDt1G2MD/6OttpwHPuIXB59auN/JKqyDST7RBU488bkWy5Hn72Xey7+vm5R48AAAgAElEQVRtbTKy4MRb7D2MjIOT/2Svd+SVzX/uU+6y3ZTBupOOvdSOyvc8yIb8yL441PTOyvI+qBs72ryhpJ9oY3eervISYmX95A6vBTz5z3Ysy+BzrfwhYTDgLGvppxzj7QnotvIysuCbJyFvkY2p7Fhhv4PNX8PSl1tHkax803YOGHyujVltnK+KxEWMMea2RrSdB/R0bfcAttWtJCKTsQMaJtbJ55UAvA/8wRhT8zpujPFEPw+KyPNAM0VeXRZJfb2E3KlAPBZAY7o6GmOVRsZE207HPrButt3nCeROub92UNsfIy47tMwdxI5NtQ/u9BPtebd8Yx/oJ/3h0OB7YwgNg5N+Z9dPuLn+erEp3nrglcfjTy8psA+ISX+wD+u2SESslc+Dv+ttavqffuj3dfYjtbc9Li2oo0hayLUVlQBn/rV22e5cq0g8LqsBZ3qVRF350ydCwQ+2B5fb0k9zjaov2mlfQi58AT68DZa94ijTFk5vnzsXeo6G8/9pBwa3ozEwwbx6viciZzSi7cVAPxFJF5EI4BJglruCiIwAngamGmN2ucojgLeAl4wxr9c5pqvzKcC5wMpGyBYU0eGhVGOD7T6pKLVdHwEi461baO+mhp+oMMe6nTyukRr3jWPURSXanjyHQ2S8/eOB162RPhH251lrREJtILs18cizO9due9x9GSfVf4wSPGnjrUUALadIfJHY08qxeYH93SX0qL9uRpb97JBWuxNLZLxNQZM71y5pJ9iXo4yJdmDk1uxmE98npbth+3Irr8dD0Y4USTAWyQzgdyJSDnhezY0xJsHfQcaYShG5CZiN7f77nDFmlYjcBWQbY2YBDwFxwOtWL7DZmefkIuBEIFlErnKa9HTz/Y8TtxFgGXBD8JfbMKIjQtlHCNG718CLU61r5ZwnvK6fuskJO6TZ0cZ1qTgA7//a9jZa9E/7BuZOP+EJGmZkeT+zn7MunuX/9f5JDpduI+wfzJM/yXO+1W9DzzH27bE1yciyn/+9HGKSrZsgMuHwlahiiUq0Vm3BD607GVVouFUe+zZDUk//ySfTJlil46vzQkaWHTcFXkvQU3/Wz2HCr6y7eOcqmNgIi7ZwPXx4q3W1nfV3+PRPsP8Qp4qlbB9gvC+DHkXy/q9tj04Px5xeO+1+cT68c6N9KQ0Jg1Pu9Lqhv5tpsxiMvs5ew+JnbfftQMk6W4GAEhlj4hvbuDHmA+CDOmV3uNYn13Pcv4F/17NvUmPlaSjR4aHs9hhtHn/upD968wZ5spN66DwIVr1jU2i4v+xt39pxFaYavnvV+lJvdvWCzp1rB255eqn0ORmGXmTdPtEdvQP6DpcLnoM599n++GBnTjzuKsj/AY5vNn0cPG55qqtsN+XR17bJP84Ry4Rf2d5xrc3o62zvtgEBnB3RSTbe1tXHwNBhF1u3bEiYN0VPdAcY9wv7P/vyr9ZLsHUJjJne8KSqy1/zztXy6Z/tS13nodYaqktEHAy5wOt+7pBmx/nsWm07OcQk2x5q8x60XbE9v+k171g3ds8xNovykhfhrIftvjn3Wetm9HV2fFL2szZtTs/RDbuOFiCof6iITMVaCABzjTHvNZ9IbYeYCNftiU21XWdLdnkVSUWp/QF5yMiCpS/Z3hvu8QCelBnfzXTacgU6q6tgw5cw2BUcjIyDH/3Trk+pNYbz8OjQG85/2rstcqhPujVpa/IcjQw4s+leTA6H8b8InBTUQ33jhpL7+J6Z9JQ7bSePj253PATGjrYPpLTqkjvXvnQV77RKBIGfzAquV2aHNO80AJf8x3ZkWfmm7dXlfj7kzrWuvqtnwysXe19Yd2+wbvKQMPti6nGZ585rk4okYIxERO7HurdWO8sMp+yoJzo8lIHi9HMf6mQgLSnwVigvre3aqunbP6d2QzW+UqfzWKTLhbRtGRzc1/bGRyjKkUxGlrPi/OcCDXaty8Ei6wbOyHLaMtYqCrZrv8cVl9Dd29uy7vPB8xKZMdHJVjDRxkv35Xnlra60mQI8z5A2OnYmGIvkDGC4p/utiLyIHZHeAsOLW5foiFB6hTidxIb8CBY+YXNVVVXAlw/b7Knu7n2xKdb0XfqS94uPTjrUr+pWRhvm2s/DyXekKEptUgfYpJFl+627ydO7a94D3pkrwcZTRl9nsyuAneRr2X/sf7S60j7ci3basoys4M/vGQOTPtHbSaDu86G82GZ78Awi9rT/zk21nxl7NtbOAPDOTbZ798TbbIYCD+s/t+72pN42953HIpr0h+By9x0GwTqfk4Ddznorzt7UskRHhPKbiuu5sds60j1pQEp22R/l3PvsdkRM7YOO+4lVMjmf2XQkJfk2B1P3TDuau3inbcPDtmX2jaWl+vUrSntAxKbjObDHuqU//RPM/5uNM8R389Yr2WU7w3hcyXPvg83f2JhG90zoebx1YfcY7fVKBEPHDNudvW5XfPfzAey8LX1PdtYHORknnOD8iCvg25dh1xr73Bg41aYayvnM9vJM6mXzlXn46LdQWmjPu3qWt2fehF8FL3cjCUaR/AX4VkTmYHtKnYgdFHjUEx0eyhtVE+nR/1p+GRFrA+slBbXNS3eMBOzbzejr7Hp5CTyQZgdZpfaHc5+wAbR5D3oD8ns22h+doihNiye55zYnp2v2s/Zh/bOvvXXeuMa6kYyxCmXzQquATnPNUBkeBdd+0rBzh0XAT949tNz9fKiLSO2YT3WV7TTgcXMNOgcuftmuPz7GPoc8iqRoh00hBDb56eDzvMqxBQgYIzHGvAocD/zPWcYaY2Y2t2BtgdAQISIshAMVzjiS2FTr2nJPeOSvW25ErH2TAa/PNDYVMPbNAWwg3mMGK4rS9HiyOFSVH+qeypjozBz6vU1o6atOaxESWju5at2R/Zu+hkpnDLf7mdQK11CvIhGRAc7nSKArdqT6FqCbU9YuiIkI5UC5S5HsWuNkXHWmkd25yn8DGVn2s5Yiwbq8DuyxgfbmTvCnKO2ZkBBn4jAOjUVmZNnPOffBV4/aXlK9xrakdP7pkObNMu5+TqRPtFmcZ/8evnjIxkSiO3i9Gy2ck86fa+tmbNLDv/rYZ4AWG8/RmsSEuxRJXCebBA9g8p3wnx9ZP6Y/Bp4Ni57x9i/3BMdKdnknaFJFoijNy5AL7HiuuvPdJPWyc+SscZJuDDjLO2C3LdB7vO3lldzPxm08pE2wGSoWu9xXmVfb9Psbv7RjsFoQfzMkTndWpxhjytz7RCSqWaVqQ0RFhFJa49pyAuKRifZN5s/76jvMS6cBcMs673aNRVLgjIZFFYmiNDeDptrFF9d8YntogbVI2hITb4EJv7SpZNwpYqIS4Ndra8/eGhrurDQmNeLhEUyurQVBlh2VxESEUlbj2vLkqDqh8aOtPYqkeJe3S1/dlO+KorQcIvYhHBre+EnpmpPQcN8ZuUNCvHLXKJHWod6noYh0wc4pEu0kV/Tc4QQgpr7jjjaiw0MpdcdI4PDGfEQl2tTwmxZYH2dMiu+UC4qiKEcI/l6rTwOuwqZ/f9hVXgT8ztcBRyPREWHsP+DEMlL7WyXQz2eKsOAQsWni175vt3UgoqIoRzj+YiQvAi+KyI+MMW+2oExtiujwEHbucyySjCy4Zf3hZ8m95mPb5RBsCgVFUZQjmGCy/74pImcCg4EoV/khsxIejcREhFFa4QTiRJom1XpUQuunbFcURWkigkna+BRwMfBzbJzkQqDdRIeTYsLZU1LPDImKoihKUL22xhljrgT2GGPuBMZSewrdo5ouCVEUH6ykqEyViaIoii+CUSQHnM9SEemGnSUxvflEalt0SbTevJ37ywLUVBRFaZ8EO2d7EnZa3KXARiCoXFsicrqIrBWRHBE5JO28iNwsIqtFZLmIfCYivV37fiIi65zlJ67y40RkhdPmo87c7c1GlwSrSHbsO9icp1EURTliCSZp493GmL1Oz63ewABjzB8DHSciocDjwBRgEHCpiAyqU+1bINMYMwx4A3jQObYj8CdgDDAa+JOIdHCOeRKbuqWfs5we8CoPA49FskMtEkVRFJ8EE2y/0bFIMMYcBEJE5GdBtD0ayDHG5BpjyrFWzDnuCsaYOcYYJyMZC7FjVsCOYfnEGLPbGLMH+AQ4XUS6AgnGmK+NMQZ4CTg3CFkaTecai+RAgJqKoijtk2BcW9cZY/Z6NpwHez0J9WvRHZst2EOeU1Yf1wAfBji2u7MesE0RmS4i2SKSnZ+fH4S4vokKD6VDTLhaJIqiKPUQjCIJccchHJdVRBDH+YpdGJ8VRS4HMrFxGH/HBt2mMeYZY0ymMSYzNTU1CHHrp3NClMZIFEVR6iEYRTIbeE1EThaRScCrwEdBHJdH7W7CPYBtdSuJyGTg98BUx3Xm79g8vO6vettsarokRrFjv7q2FEVRfBGMIrkN+Bz4KXAj8BlwaxDHLQb6iUi6iEQAlwCz3BWcZJBPY5WIayJzZgOnikgHJ8h+KjDbGLMdKBKR4x0r6UrgHZqZrolRbMgv4Ym5OdjQjKIoiuIhmF5b1caYJ40xFxhjfmSMedoYdxL8eo+rBG7CKoU1wGvGmFUicpeIeCYGeAiIA14XkWUiMss5djdwN1YZLQbucsrAKrR/ATnAerxxlWZjfN8UQkR48KO1FJaUN/fpFEVRjij8pZF/zRhzkYiswEccwumy6xdjzAfAB3XK7nCt15tG1xjzHPCcj/JsYEigczclZw3rRlW1YcbMZewtrSAlLrIlT68oitKm8Ze08ZfO51ktIUhbJyHaThyz74CmSlEURXHjT5G8B4wE7jHGBJiY/Ogn0VEk+1WRKIqi1MKfIolwUpOME5Hz6+40xvyv+cRqeySqRaIoiuITf4rkBuAyIAk4u84+A7QrRZKkikRRFMUn/mZInA/MF5FsY8yzLShTm0RjJIqiKL7x12trkjHmc2CPurYgPDSE2IhQVSSKoih18OfamogdiFjXrQXt0LUFNk6iikRRFKU2/lxbf3I+p7WcOG2bhDqKZMmm3QzulkhUeGgrSqUoitK6BJNGfoaIJIjlXyKyVERObQnh2hpui2RPSTkXPvU1L3+9qZWlUhRFaV2CybV1tTFmPzbfVSdgGnB/s0rVRkmMDmdfqVUkO4vKqDawatu+VpZKURSldQlGkXhSt58BPG+M+Q7f6dyPepJivBZJQZHNufX9jqLWFElRFKXVCUaRLBGRj7GKZLaIxAPVzStW28Tt2ioothnv1+cXU17ZLm+HoigKEJwiuQa4HRjlTIsbjnVvtTsSo8M5UFFFeWV1jSKpqDLkFhS3smSKoiitRzCKZCyw1hiz15nJ8A9AuwwMuNOk5Bd7Z0xcq+4tRVHaMcEokieBUhE5Fjuh1SbgpWaVqo3SMdamj9+5v4zC4nJS4iIJEcjZpRaJoijtl2AUSaWx0wKeAzxijHkEiG9esdomx/XuAMDX6wspKD5I18Qo4qPCg8oIvGt/mc6uqCjKUUkwiqRIRH4LXA68LyKh2DhJu6NLYhR9O8UxP6eAguKDpMRFEBcZRtHBSr/H5e0pZez9nzP3h/wWklRRFKXlCEaRXAwcBK4xxuwAumOnyG2XTOibwjcbCtm2t4yUuEjiIsMoCaBINhSUUFVtWL1t/2Gfv7yymuIA51MURWlJgpmzfYcx5mFjzJfO9mZjTFAxEhE5XUTWikiOiNzuY/+Jzkj5ShG5wFV+kjOHu2cpE5FznX0viMgG177hwV/u4TOhbwplFdXsLiknOS6SuKiwgA/27fvKANiyu/Swz3/P+6uZ+th8dZMpitJmCCZFyvEislhEikWkXESqRCRgry3HBfY4MAUYBFwqIoPqVNsMXAW84i40xswxxgw3xgwHJgGlwMeuKrd49htjlgWSpSmZ2D+VjrERAHSMDScuMoziMv+KZKejSDY3QpEUH6xky+5Stuwupara8PGqneTml5BbUNJw4RVFUZoBf9l/PTwGXAK8DmQCVwL9gjhuNJBjjMkFEJGZ2ID9ak8FY8xGZ5+/EX0XAB86Y1hanfDQEG4+5Rj+8PZKkqJtjCRvj3/Rtu+3imRTYcMuoarakPXQHAqK7Sj6Id0T2OG0NX9dAX1S4xpxBYqiKE1LMDESjDE5QKgxpsoY8zyQFcRh3YEtru08p6yhXAK8WqfsXhFZLiJ/E5FIXweJyHQRyRaR7Pz8pg1yXzamF69cN4ZzR3S3FkkA15bHItm+70CDRsFvLCyhoLicy8b04orje7Nyq42xJESFMT+noPEXoCiK0oQEo0hKRSQCWCYiD4rIr4DYII7zlY+rQY59EekKDAVmu4p/CwwARgEdgdt8HWuMecYYk2mMyUxNTW3IaYORi3F9UogIC7ExkgCuLY8VUW1g294DQZ/HM9Dx0tG9+MNZA+nRIZreyTGcOawbC9cXUlmlqVkURWl9glEkVwChwE1ACdAT+FEQx+U5dT30ALY1UL6LgLeMMTUDNYwx243lIPA81oXWasRGhlFSXkV1df06cse+Mvp2sm6o+uIkZRVVVNRRDN9v30+IQN9OcUSGhfLCtNE8edlxTOibQtHBSr7La3yCAQ3WK4rSVATTa2uTMeaAMWa/MeZOY8zNjqsrEIuBfiKS7lg0lwCzGijfpdRxazlWCiIiwLnAyga22aTER9owU0m5tUr+8dk6LnhyQc3+g5VVFJaUMzq9IwCrfHQBfuaL9Qz440ccf99nLM/by7F3fszaHUV8v6OI9JTYmomz+naKY1C3BMb1SUYEvmqke2vb3gMce+fHfLp6Z6OOVxRFcVOvIhGRFU4cwucSqGFjTCXWipkNrAFeM8asEpG7RGSqc45RIpIHXAg8LSKrXOdPw1o08+o0/R8RWQGsAFKAexpywU1NXJRVJJ44yZLNe8jetKemq++u/TYn17E9EhmT3pFn5+fWGneya38Zf/tkHV0SoigsKefFBZvYd6CCj1ft4PsdRQzoknDIOTvERjCkWyLz1zVOkcxZu4v9ZZXc98EadY8pinLY+Ou1ddbhNm6M+QD4oE7ZHa71xViXl69jN+IjOG+MmXS4cjUlcR6LxFEOO5zA+oL1BVzcsRfr820erq6J0dw2ZQDnP7GAZ+dvYNW2fcxeZS2CsBDhoQuHccWzi5i9agcA76/YzubdpVx4nM/bw/i+KTw1bz2j7/2U164fS1qKN2xljOGGfy9h9qqdTBnSheMzknl9yRZeu34sMRFhfJVTQHiokFtQwrvLt3HeCN/n+Ca3kCueXURMZCgvXT2aYT2SmuCOKYpytOHPtRUO9HBcWzUL0Ivgug23CzyKpMgJuO90AutfrivAGMMTc9aTEhdJZloHRvbqwKmDOvPoZ+uYvWon543ozoyT+/H8tFGM75NCbERojWXz/Y4iQkOEs47t5vO8V09I49enHEPxwUoe+nhtrX2frtnF7FU7GdkriQ9X7uDu91azcut+nv9qI1XVhgXrCzlneHc6xUfy+ff192j7cOUOQkKsorv3/TUaV1EUxSf+FMnfAV/50Q84+xS8rq0ZM5fxlw/WsMeZinfB+kIWrC9k0cbdzDi5LzERtt6tp/en2hjSU2J58IJh/OqUYzihXyohIUL/LjYXZkaqtS4uHtWT9BTfHeQ6xUfx85P7cc2EdN5fvp3Mez5h7tpdnP/EV9zw7yWkp8TyynXHk5YcQ7UxHNsziafmrufz73ext7SCE/qlMKFvCl/lFDDt+UU8NW89AP/8IpcbXl4CwJfr8hmTnszPJ/Xjmw27NVeYoig+8WdZpBljDomFGGOynfiFgtci2by7lH8v3ATAuD7JLFhfyKOfrSMyLIQLM72d1/p2iufpKzJJS44hPLS2Hh/QNYGlm/dy1bg0KqsMPxrp2+Xk5mdZfYmOCOXfX29i+stLKK+s5orje3Pl2N5EhYfyzJWZbC4spVdyDKf9/QtufGUpHWMjmDSgE1XVhv99u5U5a/P5KqeQs4Z15b/ZW8jNL2ZjQQnr80u4ZFQvLh3di2fnb+DBj9Yy0VF6iqIoHvxZJFF+9kU3tSBHKh5FAlBSXgXA+Y4C+GbDbkand6zpdeXhlEGd6df50Ez8AxyLZFDXBK6ekE5iTOAky9ERofwsqy+3TRlAeWU1J/VP5e5zh9S0f0zneCYP6swxneM5f0QPyiuruemkvsRHhTO+bwoA/TrFgcDv3lpJzq5iqg28sGAjABP62fEyvz71GNZs30/mvZ/yzrKtDbxLiqIczfizSBaLyHXGmH+6C0XkGmBJ84p15OBWJB6G90ykX6c41u0qZoLzsA6Gc47tzoHyKkb06tBgOc4e1o39ByqYPKhzvXV+f+ZAhnRP4LIxvQHonBDFvecNYVRaR17P3sI/v9xQU/e17C2kxEXQ31FIZw/rRn7RQZ75Ipc3luRxzvDGJClQFOVoxJ8i+SXwlohchldxZAIRwHnNLdiRQqwPRdI5IYoJ/VJYt6u45q0/GBJjwrl+Yp9GyRESIlwxNs1vnY6xEUwbn16rzKNUfpbVl5mLthAeFkJpeSWl5VWcMqhzjRsrJES49oQMtu49wCvfbOaOd1by2ZpdnDa4C3ecXTcXp6Io7Yl6XVvGmJ3GmHHAncBGZ7nTGDPWmZdEASLCvLcwKjyE2IhQ4qPCuXp8Orec1p9BXQ8dB9IW6RAbwd8vGc5d5wyusUJ8KcEJfVM4WFnNS19vorS8klcWbaKsoqqlxVUUpQ0RzMj2OcaYfzjL5y0h1JFKv07xdE60oaWeHWO48aS+R1Rg+uSBnTlrWLea3mO+3HJjMpIJCxE6J0Ry1zlDKKuoZunmPTX7NxWWcNm/FrJldynXv5zNt659rcHmwlKueym7SeaCURTFNzoepAn4y/lDyUiJpaisktKj4O38yrFppKfE0S3p0D4VcZFh/P7MgfTtFMfwnkmEhghf5RQwro9VOg989D1f5RRy93ur+Xj1To7pHN+omE9Tcf9Ha/hk9U4iwkJ4/McjW00ORTmaUUXSBFw6uldri9CkDOmeyJDuifXud8dZRvRM4qWvN1FYXM5Fo3rywQrr9fzYyePlK0nlFz/k8/AnP1BtDJ0TonjogmEkxdjJwm5+bRnrdxUz/cQ+nDmsa6OvYenmPdz17mqWbdlLjw7RvL98O9edsJfXsrcwZUgXTujXtBmhFaU9E9R8JIpSHz/N6sOgrgnMXLyF617MJjk2gnOGe0fj153Mq6yiitveXM6u/WV0jI3gszU7eXyOzQGaX3SQ/y3dypodRfz2f8vZV1pBY/nvoi2s3VHEpaN78sYN4+gQE841LyzmlW82c9sbyzWuoyhNiCoS5bA4eWBnXr3ueIZ2T6SwpJybJvXl9MFdAAgR7zz1z87fwMVPf815Tyxg+74y/nrRcF6YNprzR/bgxa83sWNfGd/vsJmRbz99AEUHK3liXg7vL9/Ow5/8wMqt+7jl9e84WBlYARhjmJ9TwMRjUvnL+cPokhjFjSf1pbCknM4JkWzbV8Z5TyzgzSV5zXdjFKUdoa4t5bAJCRH+cv5Q/rt4Cz8e04uqasP5I7oTFRHKK99sZsmm3dzz/mr6pMaRHBvBjJP7MbZPMgA3TOzDG0vy+HTNTg44AzrPGd6NlVv38cJXG4kKD6WorILc/GLeW76dgc5gTX9sKixl694D3JDl7Up9xdjebCos5ZLRPZm9aifvLNvK795awbi+yXRN1PG1inI4SHtIxJeZmWmys7NbW4x2x/vLt3PjK0vJSImloPggX9466ZDR+sYYJjwwh2E9EomJCOPLdfks+v1ktuwuZdJf51JRZX+fImAMxEeFkdm7dvBeRLhqXBr7yyp4c0ke+cUHWbl1P3N+k1VvrjJP+xcc14O/nD+s1r6isgru+2ANN03qR3cfHQ4Upb0gIkuMMZmB6qlFojQbvTrGAJBbUMKtp/f3mfJFRBjfN5nZq3bSNTGqputxz44x/HnqYAqKynlq3noOVFRx+fG9yNlVTGFJea02tu0t41f/XUZpeRVJMeGkxkdy/ojupCXH1Ctbz44xXDamNy8v3MS1J2TQJzWuZt8zX+Ty6qItdEuM5ucn92uKW6EoRzWqSJRmw6NIOidEMm1c/e6o8X1TeC07j30HKjjxGG9vKs+o+2+37GHu2nyuGpdeM2Wxm6Wb93D+EwsICxFmTj+e3sm+rZC63DSpL69nb+GGl5fQr7O33TlOav0vcwoOUSQLcwtZt7OI8X1TeGfZNq4Y25vnv9rAdSdk1PQ8U5T2hioSpdlIjAnn3OHdmDK0K9ERofXWy+rficzeHSg+WMmpPnKFXTm2Nx1iIuiT6ltBjOzVgZ9P6kuHmIiglQhASpwdVPn0F+tZt7O4pnxo90RSEyL5eNUOSg5W1qTBKS2v5Oevfkt+0UHSU2LZUFDCu99tI7eghJKDVfx56uCgz60oRxPNGiMRkdOBR4BQ4F/GmPvr7D8RO7fJMOASY8wbrn1V2Ol0ATYbYzzT86YDM4GOwFLgCmNMbV9HHTRGojSU+esKuPzZb3h+2ihO6t+JD1Zs5+WvN/F1biFxkWEUH6ys9Xmwsoqzj+1GeEgIV09Ir3HRLdm0h1cXbSbQ3+yizB6MyUhugStTlOBp9RiJiIQCjwOnAHnYbMKzjDGrXdU2A1cBv/HRxAFjzHAf5Q8AfzPGzBSRp4BrgCebVHil3ZOZ1oGIsBDmryugb2ocM2Z+S1xkGNedkM6IXh1497ttXHtCBo98to5bT+vPHe+s5Jvc3ewuKee7vL188IsTKK+q5qZXllJUVklidP1TAhQUH2T7vgO8oopEOUJpTtfWaCDHGJMLICIzgXOAGkXizMuOiFQH06CICDAJ+LFT9CLwZ1SRKE1MVHgoo9M68lVOAbtLygkR4cMZJ9LFyaV2xlA76v6lq0cD8L+fjQfgnWVbmTFzGTe9upQD5VVs31fGzOnHc7wfJXHPe6t56etNHCiv8usCVJS2SnMOSOwObMZbxJkAABL7SURBVHFt5zllwRIlItkislBEznXKkoG9xpjKQG2KyHTn+Oz8fJ0iVmk44/um8P2OIt5etpWrxqfVKBF/nD2sG1OGdGFh7m6+y9vHZWN6+VUiAOP7pVBeVU32pt1NJbqitCjNaZH4SnvbkIBML2PMNhHJAD4XkRXA/mDbNMY8AzwDNkbSgPMqCgAn9EvhgY8gPjKMn03sG9QxISHCk5cf16DzjEnvSHio8I/PcqisMpw0oFNjxFWUVqM5LZI8oKdruwewLdiDjTHbnM9cYC4wAigAkkTEowAb1KaiNIRBXRMY0SuJW08fENS0x40lJiKMs4d1Y+nmPdz25nLawyBh5eiiORXJYqCfiKSLSARwCTArmANFpIOIRDrrKcB4YLWx/7A5wAVO1Z8A7zS55IqCtS7e+tl4Lj++d7Of6+GLh3PPuUPYVXSQnF3FgQ9QlDZEsykSJ45xEzAbWAO8ZoxZJSJ3iYinK+8oEckDLgSeFpFVzuEDgWwR+Q6rOO539fa6DbhZRHKwMZNnm+saFKUlmdDPzukyP6eglSVRlIahubYUpQ2R9dAcYiPDmHpsN3onxzBpQGfmrN3FpAGdCA/VZN1Ky9Lq40gURWk4Zw7ryuNz1rNqm+1XclL/VOaszef2KQO4YWKfAEcrSuugFomitDFKyyupqDKc+eiX5O05AEBCVBi/O2MgU4d3I2dXMcu27GVcn2T6dopvZWmVo5lgLRK1lRWljRETEUZidDi/P2MgHWMjeOSS4RgDt/9vBTf+ZykXPPU1d7yzitveXBG4MUVpAdS1pShtlClDu3Lq4C6EhginDOrMgx+t5YUFG4kIDeG8Ed2Z9d02tuwupayiin6d1TJRWg+1SBSlDRMaYsf1xkSEMePkfnSKj+T6iRlclNmTqmrDGY98ydmPzWfX/rJWllRpz6giUZQjhA6xEXx520n8+tT+jOydRHR4KEUHKymrqObRz9fVqrt6236qq238M2dXEYXFB2vtL6uoYuXWfS0mu3J0o4pEUY4gIsNCaz6z+qcyoEs8l47uxcxFW9hYUALAx6t2cMajX/L4nByqqg0XPb2Qn/57aa0R8/9euImpj81n294DrXIdytGFxkgU5QjlbxcPp9oYig9W8va3W7nrvdVcfnwvHpy9FrBTBg/pkcjuknIWlexm7tr8mjxeK7fuo9rYwY8XZfb0dxpFCYhaJIpyhBIVHkpMRBid4qO4YWIfPv9+F1e/kE3OrmJuOa0/JeWV/Oa17wDomhjFAx99X+Pu+n5HEWAn8FKUw0UtEkU5CvjFyX05dXBnyiurSYwOJy0llo0FJby+JI+BXRO4YWIGM2Yu453vtnLm0G6sz7f5vL7KKaC62pBbUEJ+0UHCQoWRvTpQXFZJeJhgDBQfrKRzgjeF/o59ZQFT6ldVG/L2lDZo6mPlyEUViaIcBYgIA7sm1Cr71SnH8O7ybZzUP5Wzh3XjmS9y+evHP9A3NZ6KKkNW/1Tmrs3n/z5ey5Pz1tdMB3zeiO4s2bSHDjHhJMVE8MPOIhbcPgkRqZm46x+XjuDsY7vVK89Ds9fy7Pxcvrp9Ep3iA8/johzZqCJRlKOUbknRfPbrLJJjIwgJEW49fQA/eW4Rd75rc6P++pT+/LCjiCfmrqd7UjQPXTiMt7/d+v/t3Xl0VFWewPHvL/tKQhZICGFNAIHIjiKCSrcojA4gjmIr0k6fYehuz3SPjTYOc+webWy7te0ePU6PDOKIMMOINjMqCgqoLELYhJAQE0ICISRkI2Ql+50/6hFCSIWKlaRSxe9zTp16ue+9qvvjFvWrd+979/HeoTwAclvdZ+tyV9grn9nGX17elkF8RNA1Nx1K6BdCRW0Db+/NoaHJ8HVWKfMndOZ+dsodaSJRyoPFhQe2LM9MjGJ6QiR7s0oJ9PVmVGwoT80eyfJNx3jq7hHcNjyKm2L6sDX1PEkDwyitqqekqp6SqjqWrT/MmdIaAH50+1De2pPD/Df2XvN+Cf1CGDcwHGNsNwTbfbJEE8kNQOfaUuoGcqm+if3ZpQwID2RkjO1q+JySaoZGXRnLKCi/RHigH/VNzTQ0NfPQm/vILq5m1qh+PHX3CMbGhZF6rpyiyqsvgjxVVM2qT9IBeGL6EIoq6zh8uox9z87iYk0DfYP9ei5Q1SV09l+l1DUC/byvuZVv6yQCEBtmO4oJxHbNyszEaHJLa3juvtEMsbYdGxcGhF21310jDZ+dOE96QSU/vSuBz08UsiWlgO3pRSxbf5jfPpCkpxp7KD0iUUp1qKK2gXNll64ZzG9PZW0DZdUNDIoM4uyFGmb8/guGRQWTXVJN/z7+/M/Safh4C3HhgYgIjU3NNDYbAny9eyAS1Vl6RKKU6hJ9AnzpE+vYPetDA3wJDbBtGx8RxODIILJLqgn196Gwoo47X/kSgEVT4nlp4c388oPjpOWXs/XnM7ur+qoHdOsFiSJyr4hkiEiWiKxoZ/1METkiIo0i8mCr8vEisk9E0kQkRUQebrXuP0UkR0SOWo/x3RmDUuq7m55gu33wvAkDeO/vp/Hygzczf/wANh48y8YDuXxwJI9vz1dSWdtw1X7VdY2UVtVxsaa+pcwY03JBpepduu2IRES8gTeAu4E84KCIfNjq3usAucAPgeVtdq8BHjfGnBSRAcBhEdlmjLlorX/aGPN+d9VdKdU1ZiZG8V/JucxIjGbq0AimDo1g9pgY210f/3LlfiqZhZVMGhwBwNkLNXzvD19R39QMwPLZI3hyViLLN6VQUlXHO3871SWxKPu6s2trKpBljMkGEJGNwDygJZEYY05b65pb72iMyWy1nC8iRUA0cBGllNuYPTqG1Ysn8f2b+reUhQX6smnZNPZnlxIfEcQTbx8kvaCSMQPC8PYSvswoor6pmWfuHcm+U6W8vjOL+RPi2JZ2npr6RipqG+hjdZ8ZYxBpezWLe3LnWLqzaysOONvq7zyrrFNEZCrgB5xqVbzK6vL6o4j429lvqYgcEpFDxcXFnX1bpVQX8PISZo+Jwcvr6i/IEf1DeXzaEO4cEU2ovw+rd2Uz5lfbmLpqOxsPniUuPJAf3zGcFxck0WwMP9lwhKq6RpoN7D9VCsCO9EJue2knWUVVrgjtKltTC5j22x3XdNE5qrCilimrdrAlpQCAv1t3iJWb3ecOmN2ZSNpLrZ3q4BSRWOBd4AljzOWjlmeBUcAUIAL4ZXv7GmNWG2MmG2MmR0dHd+ZtlVI9REQYFRtK7oUaRsf2oaHJkJZfwYzEKESE+IggHr1lMCl55YhAgK8Xe7JKaGhq5jdb0ikor+Xlbd+6Ogx2nyyhoLyW5OwL19+4Hf+64yQlVXV8cryAitoGdn5bxMcpBTS5yZhQd3Zt5QGtTxofCOQ7urOI9AG2AP9sjNl/udwYU2At1onI21w7vqKUciM3xfbh4OkyXpg/ll2Zxbz6eWbLID3Ak7MS2HToLMP7hRAR7Me6fWdYt+8MALcNj2RbWiFDVmwBbHeU/NX9o9mbVUK/0ABemD/W7vtW1zXy8Op9PDQ5nsenDQHg1c8zeW3HlZuEhQX68tk/zrxq0sr2ZFhTyKzenc0vNh2joamZNxdPYkbilR+xKzcfZ0NyLrcMjeCRqYN45v2UlnEgAD9vL/aeKuHrrFKamg3llxpIyy/n5oHhDv5Luk53JpKDQKKIDAXOAYuAHziyo4j4AZuBdcaYTW3WxRpjCsTWmTgfSO3aaiuletLSmcOYNiyS8fHhjIoJJSrEn3vGxLSsjwrxZ82SKYT4++Dn48U464t1SFQQc8bGsvFALmU1ti6l3SeLef6jEzRav+QfnDSQcfHtfxGv3ZND6rkKTpdk8FdJsVTWNvJvX2QxIzGKiYP6Ul3XyJo9OXyVUcxDU+xfSGmMaZmL7EDOBaJC/AgL9OeFj0/w6c9m4u0lfJNbxobkXCYOCic55wKHz5SR0C+kJc4Qfx+C/L1ZuTmV1btO4efjRX1jM7tPlrhFIunWCxJFZC7wJ8AbWGuMWSUizwOHjDEfisgUbAmjL1ALnDfGjBGRx4C3gbRWL/dDY8xREdmJbeBdgKPAMmNMh52kekGiUjeGrKJKZv9xF4MigqisbaT8UgM+3sLo2D78w/cS+cV7x6iubwSgrrGZcQPDScm7iLc1huPj5cVXz9xJv9AAjDFMfXEHtw6L5PVHJtDcbFj471+zcOJAHrt1cMt7tlx4GR1MdnE1L8wfS2SwHz/ZcAQ/Hy+8BBqbDOFBvnz59F08tiaZo2cv8v6yaUweEtHyOsWVdUxZtR2AO0ZEU1RZR8b5Cvx82h+B8PP24uW/GdeSjH7z8QnWJ59h6tBI1nXRmW294oJEY8wnwCdtyp5rtXwQW5dX2/3WA+vtvOasLq6mUspDJPQLZfXiyQyODKK6volPjxdQVdfIhuRclr57mOgQfx6cZPvK8ffxYvG0IXyTW8bhM2WA7Qv88rT3IsLtCVHsyiymudk2dvNN7kVOFVVx/80DCAuynTl2uVvruftGk1NSzaIp8fh4CasWjCXXmugSYG5SLCH+Prz+yASOnr14VRIBiA7153cLk8gurub+cQOoqW9iR3qh3Vi3pxfy/EcnuGNENFlFVazZk0NsWAC7MosprKi9bndcV9IpUpRSHm/J2gN8lVnMa49M4K87uI9KW+8fzmP5pmPEhQeSFBfG1rTziECwn62bbc7YGCKC/Xh9Zxap/3IPIf49N1nI3qwSHl2TTGiAD41NhgBfL/782CQWrd7Pqw+N44GJ1/xG77RecUSilFK9wUsLk9h+opD7kmI7td+csTFkFlbyv9+cY2vaeUbFhPLjO4dz6HQZeWU1bEjOJcDXiztGRPdoEgHbrAEvLkgivaACgPvHDWDy4L5EBvux52RJlyQSR2kiUUp5vNiwQBZbZ2Z1RrC/D/809yZG9A9l+aZjTE+IYt74OOaNj6O2oYm7XvmSgvJanr5nZNdX2gE/uGXQNWW3JUSx66TtFGlf726dBauFJhKllLqOBRPiyL94ifnjr1xTHeDrzRuPTuRUUZU1rX7v8MCEOD46ls/Gg2dZ3OqkgO6kYyRKKeVBjDE8/OZ+UvPLiQsP5K0lUxgUGfSdXsvRMZKeOe5RSinVI0SElxYmMXt0fxL7h9g9fbgradeWUkp5mGHRIfxp0YQeez89IlFKKeUUTSRKKaWcoolEKaWUUzSRKKWUcoomEqWUUk7RRKKUUsopmkiUUko5RROJUkopp9wQU6SISDFw5jvuHgWUdGF1ehNPjc1T4wLPjc1T4wL3jm2wMSb6ehvdEInEGSJyyJG5ZtyRp8bmqXGB58bmqXGBZ8d2mXZtKaWUcoomEqWUUk7RRHJ9q11dgW7kqbF5alzgubF5alzg2bEBOkailFLKSXpEopRSyimaSJRSSjlFE0kHROReEckQkSwRWeHq+jhDRE6LyHEROSoih6yyCBH5XEROWs99XV1PR4jIWhEpEpHUVmXtxiI2r1ltmCIiE11X847ZievXInLOarejIjK31bpnrbgyROQe19TaMSISLyJfiEi6iKSJyM+scrdutw7i8oh2c5gxRh/tPABv4BQwDPADjgGjXV0vJ+I5DUS1Kfs9sMJaXgH8ztX1dDCWmcBEIPV6sQBzgU8BAW4Fkl1d/07G9WtgeTvbjrY+k/7AUOuz6u3qGDqILRaYaC2HAplWDG7dbh3E5RHt5uhDj0jsmwpkGWOyjTH1wEZgnovr1NXmAe9Yy+8A811YF4cZY3YBF9oU24tlHrDO2OwHwkUktmdq2jl24rJnHrDRGFNnjMkBsrB9ZnslY0yBMeaItVwJpANxuHm7dRCXPW7Vbo7SRGJfHHC21d95dPwB6e0M8JmIHBaRpVZZf2NMAdj+QwD9XFY759mLxRPa8Umre2dtq+5Ht41LRIYAE4BkPKjd2sQFHtZuHdFEYp+0U+bO50pPN8ZMBOYAPxWRma6uUA9x93b8MzAcGA8UAH+wyt0yLhEJAT4Afm6Mqeho03bKem187cTlUe12PZpI7MsD4lv9PRDId1FdnGaMybeei4DN2A6nCy93F1jPRa6rodPsxeLW7WiMKTTGNBljmoH/4Eo3iNvFJSK+2L5sNxhj/mIVu327tReXJ7WbIzSR2HcQSBSRoSLiBywCPnRxnb4TEQkWkdDLy8BsIBVbPEuszZYA/+eaGnYJe7F8CDxunQV0K1B+uSvFHbQZF1iArd3AFtciEfEXkaFAInCgp+vnKBER4C0g3RjzaqtVbt1u9uLylHZzmKtH+3vzA9uZI5nYzqxY6er6OBHHMGxnihwD0i7HAkQCO4CT1nOEq+vqYDz/ja27oAHbL7wf2YsFW1fCG1YbHgcmu7r+nYzrXaveKdi+hGJbbb/SiisDmOPq+l8nttuxdeGkAEetx1x3b7cO4vKIdnP0oVOkKKWUcop2bSmllHKKJhKllFJO0USilFLKKZpIlFJKOUUTiVJKKadoIlFKKeUUTSRKKaWc8v8wLFqLRKUydQAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# retrieve performance metrics\n",
"results = model.evals_result()\n",
"epochs = len(results['validation_0']['error'])\n",
"x_axis = range(0, epochs)\n",
"# plot log loss\n",
"fig, ax = pyplot.subplots()\n",
"ax.plot(x_axis, results['validation_0']['logloss'], label='Train')\n",
"ax.plot(x_axis, results['validation_1']['logloss'], label='Test')\n",
"ax.legend()\n",
"pyplot.ylabel('Log Loss')\n",
"pyplot.title('XGBoost Log Loss')\n",
"pyplot.show()\n",
"# plot classification error\n",
"fig, ax = pyplot.subplots()\n",
"ax.plot(x_axis, results['validation_0']['error'], label='Train')\n",
"ax.plot(x_axis, results['validation_1']['error'], label='Test')\n",
"ax.legend()\n",
"pyplot.ylabel('Classification Error')\n",
"pyplot.title('XGBoost Classification Error')\n",
"pyplot.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.7"
}
},
"nbformat": 4,
"nbformat_minor": 2
}Let's describe my approach to select parameters (n_estimators, learning_rate, early_stopping_rounds) for XGBoost training.
Step 1. Start with what you feel works best based on your experience or what makes sense
Results:
- Stop iteration = 237
- Accuracy = 78.35%
With the first attempt, we already get good results for Pima Indians Diabetes dataset. Training was stopped at iteration 237. The classification error plot shows a lower error rate, around iteration 237. This means the learning rate 0.01 is suitable for this dataset, and early stopping of 10 iterations (if the result doesn't improve in the next 10 iterations) works.
Step 2. Experiment with learning rate and try to set a smaller learning rate parameter and increase the number of learning iterations.
Results:
- Stop iteration = didn't stop, spent all 500 iterations
- Accuracy = 77.56%
A smaller learning rate wasn't working for this dataset. Classification error almost doesn't change and XGBoost log loss doesn't stabilize even with 500 iterations.
Step 3. Try to increase the learning rate.
- n_estimators = 300
- learning_rate = 0.1
- early_stopping_rounds = 10
- Stop iteration = 27
- Accuracy = 76.77%
With increased learning rate, the algorithm learns quicker, and it stops already at iteration Nr. 27. The XGBoost log loss error is stabilizing, but the overall classification accuracy is not ideal.
Step 4. Select the optimal learning rate from the first step and increase early stopping (to give the algorithm more chances to find a better result).
Results:
- Stop iteration = 265
- Accuracy = 78.74%
A slightly better result is produced with 78.74% accuracy — this is visible in the classification error plot.
Resources:
XGBoost
Published at DZone with permission of Andrejus Baranovskis, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments