DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • Learning AI/ML: The Hard Way
  • How to Use Python for Data Science
  • Data Science: Scenario-Based Interview Questions
  • The Art of Separation in Data Science: Balancing Ego and Objectivity

Trending

  • Contextual AI Integration for Agile Product Teams
  • Simplify Authorization in Ruby on Rails With the Power of Pundit Gem
  • Chaos Engineering for Microservices
  • Integrating Security as Code: A Necessity for DevSecOps
  1. DZone
  2. Data Engineering
  3. AI/ML
  4. Perceptron Explained Using Python Example - Data Analytics

Perceptron Explained Using Python Example - Data Analytics

In this post, you will learn about the concepts of Perceptron with the help of Python example. It is very important for data scientists to understand the con...

By 
Ajitesh Kumar user avatar
Ajitesh Kumar
·
Oct. 20, 20 · Presentation
Likes (6)
Comment
Save
Tweet
Share
20.9K Views

Join the DZone community and get the full member experience.

Join For Free

In this post, you will learn about Perceptrons with the help of a Python example. It is very important for data scientists to understand the concepts related to Perceptron as a good understanding lays the foundation of learning advanced concepts of neural networks including deep neural networks (deep learning).

What Is a Perceptron?

Perceptron is a machine learning algorithm which mimics how a neuron in the brain works. It is also called as single layer neural network, as the output is decided based on the outcome of just one activation function which represents a neuron. 

Let's first understand how a neuron works. The diagram below represents a neuron in the brain. The input signals (x1, x2, ...) of different strength (observe weights, w1, w2 ...) is fed into the neuron cell via dendrites. The net input (weighted sum) is processed by the neuron and output signal (observer signal in AXON) is appropriately fired. In case the combined signal strength is not appropriate based on decision function within neuron cell (observe activation function), the neuron does not fire any output signal.

Fig 1. Neuron in Human Brain
Fig 1. Neuron in Human Brain

The perceptron when represented as a line diagram would look like the following:


Fig 2. Perceptron - Single-layer Neural Network
Fig 2. Perceptron - Single-layer Neural Network


Pay attention to some of the following in relation to what's shown in the above diagram representing a neuron:

  • Step 1 - Input signals weighted and combined as net input: Weighted sums of input signal reaches the neuron cell through dendrites. The weighted inputs represent the fact that different input signal may have different strength, and thus, weighted sum. This weighted sum can as well be termed as net input to the neuron cell.
  • Step 2 - Net input fed into activation function: Weighted The weighted sum of inputs or net input is fed as input to what is called as activation function.
  • Step 3A - Activation function outputs binary signal appropriately: The activation function processes the net input based on unit step (heaviside) function and outputs the binary signal appropriately as either 1 or 0. The activation function for perceptron can said to be unit step function.
  •  Step 3B - Learning input signal weights based on prediction vs actuals: A parallel step is neuron sending the feedback to strengthen the input signal strength (weights) appropriately such that it could create output signal appropriately that matches the actual value. The feedback is based on the outcome of the activation function which is a unit step function. Weights are updated based on the gradient descent learning algorithm. Here is my post on gradient descent - Gradient descent explained simply with examples. Here is the equation based on which the weights get updated:
Fig 3. Weight update rule of Perceptron learning algorithm
Fig 3. Weight update rule of Perceptron learning algorithm

Pay attention to some of the following in above equation vis-a-vis Perceptron learning algorithm:

  • Weights get updated by \(\delta w\)
  • \(\delta w\) is derived by taking first order derivative of loss function (gradient) and multiplying the output with negative (gradient descent) of learning rate. The output is what is shown in the above equation - product of learning rate, difference between actual and predicted value (perceptron output) and input value.
  • In this post, the weights are updated based on each training example such that perceptron can learn to predict closer to actual output for next input signal. This is also called as stochastic gradient descent (SGD). Here is my post on stochastic gradient descent. That said, one could also try batch gradient descent to learn the weights of input signals.

Perceptron Python Code Example

In this section, we will look each of the steps described in previous section and understand the implementation with the Python code:

  • Input signals weighted and combined as net input: Input signals get multiplied with weights and the sum of all weighted input signal is taken. This sum is called as net input and would be fed into activation function. Here is the code for net input. Note the dot product of coefficients with input X. Coefficient[0] is multiplied with 1 (bias element).
Python
xxxxxxxxxx
1
 
1
'''
2
Net Input is sum of weighted input signals
3
'''
4
def net_input(self, X):
5
   weighted_sum = np.dot(X, self.coef_[1:]) + self.coef_[0]
6
   return weighted_sum


  • Activation function invoked with net input: Net input is fed into activation function and output is determined based on the outcome of unit step function. Here is the code:
Python
xxxxxxxxxx
1
 
1
'''
2
Activation function is fed the net input and the unit step function is executed to determine the output.
3
'''
4
def activation_function(self, X):
5
    weighted_sum = self.net_input(X)
6
    return np.where(weighted_sum >= 0.0, 1, 0)


  • Prediction based on the activation function output: In Perceptron, the prediction output coincides with (or equal to ) the output of activation function which uses unit step function. This is where the Perceptron is different from ADAptive LInear NEuron also termed as Adaline. The Adaline algorithm implementation will be described in future post. In Adaline algorithm, the activation function is identity function and prediction is done based on unit step function output.
Python
xxxxxxxxxx
1
 
1
'''
2
Prediction is made on the basis of output of activation function
3
'''
4
def predict(self, X):
5
    return self.activation_function(X)


  • Learning weights based on the prediction vs actual: Weights are learned based on comparing the value of actual label and the predicted label (binary) and applying stochastic gradient descent algorithm to learn weights based on each training example. The idea is to improve on making predictions based on the learning from last training example. Here is the fit method which demonstrates the learning of input weights:
Python
xxxxxxxxxx
1
15
 
1
'''
2
Stochastic Gradient Descent 
3
    
4
1. Weights are updated based on each training examples.
5
2. Learning of weights can continue for multiple iterations
6
3. Learning rate needs to be defined
7
'''
8
def fit(self, X, y):
9
    rgen = np.random.RandomState(self.random_state)
10
    self.coef_ = rgen.normal(loc=0.0, scale=0.01, size=1 + X.shape[1])
11
    for _ in range(self.n_iterations):
12
        for xi, expected_value in zip(X, y):
13
            predicted_value = self.predict(xi)
14
            self.coef_[1:] = self.coef_[1:] + self.learning_rate * (expected_value - predicted_value) * xi
15
            self.coef_[0] = self.coef_[0] + self.learning_rate * (expected_value - predicted_value) * 1


Here is how the entire Python code for Perceptron implementation would look like. This implementation is used to train the binary classification model that could be used to classify the data in one of the binary classes. Pay attention to all the methods that are explained previously. Also, pay attention to the score method which is used to measure the accuracy of the model.

Python
xxxxxxxxxx
1
61
 
1
import numpy as np
2
#
3
# Perceptron implementation
4
#
5
class CustomPerceptron(object):
6
    
7
    def __init__(self, n_iterations=100, random_state=1, learning_rate=0.01):
8
        self.n_iterations = n_iterations
9
        self.random_state = random_state
10
        self.learning_rate = learning_rate
11
12
    '''
13
    Stochastic Gradient Descent 
14
    
15
    1. Weights are updated based on each training examples.
16
    2. Learning of weights can continue for multiple iterations
17
    3. Learning rate needs to be defined
18
    '''
19
    def fit(self, X, y):
20
        rgen = np.random.RandomState(self.random_state)
21
        self.coef_ = rgen.normal(loc=0.0, scale=0.01, size=1 + X.shape[1])
22
        for _ in range(self.n_iterations):
23
            for xi, expected_value in zip(X, y):
24
                predicted_value = self.predict(xi)
25
                self.coef_[1:] = self.coef_[1:] + self.learning_rate * (expected_value - predicted_value) * xi
26
                self.coef_[0] = self.coef_[0] + self.learning_rate * (expected_value - predicted_value) * 1
27
    
28
    '''
29
    Net Input is sum of weighted input signals
30
    '''
31
    def net_input(self, X):
32
            weighted_sum = np.dot(X, self.coef_[1:]) + self.coef_[0]
33
            return weighted_sum
34
    
35
    '''
36
    Activation function is fed the net input and the unit step function
37
    is executed to determine the output.
38
    '''
39
    def activation_function(self, X):
40
            weighted_sum = self.net_input(X)
41
            return np.where(weighted_sum >= 0.0, 1, 0)
42
    
43
    '''
44
    Prediction is made on the basis of output of activation function
45
    '''
46
    def predict(self, X):
47
        return self.activation_function(X) 
48
    
49
    '''
50
    Model score is calculated based on comparison of 
51
    expected value and predicted value
52
    '''
53
    def score(self, X, y):
54
        misclassified_data_count = 0
55
        for xi, target in zip(X, y):
56
            output = self.predict(xi)
57
            if(target != output):
58
                misclassified_data_count += 1
59
        total_data_count = len(X)
60
        self.score_ = (total_data_count - misclassified_data_count)/total_data_count
61
        return self.score_


Here is the Python code which could be used to train the model using CustomPerceptron algorithm shown above. Note that SKlean breast cancer data is used for training the model in order to classify / predict the breast cancer.

Python
xxxxxxxxxx
1
25
 
1
import numpy as np
2
from sklearn import datasets
3
from sklearn.model_selection import train_test_split
4
#
5
# Load the data set
6
#
7
bc = datasets.load_breast_cancer()
8
X = bc.data
9
y = bc.target
10
#
11
# Create training and test split
12
#
13
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42, stratify=y)
14
#
15
# Instantiate CustomPerceptron
16
#
17
prcptrn = CustomPerceptron()
18
#
19
# Fit the model
20
#
21
prcptrn.fit(X_train, y_train)
22
#
23
# Score the model
24
#
25
prcptrn.score(X_test, y_test), prcptrn.score(X_train, y_train)

Conclusions

Here is the summary of what you learned about the Perceptron algorithm with help of Python implementation:

  • Perceptron mimics the neuron in the human brain
  • Perceptron is termed as machine learning algorithm as weights of input signals are learned using the algorithm
  • Perceptron algorithm learns the weight using gradient descent algorithm. Both stochastic gradient descent and batch gradient descent could be used for learning the weights of the input signals
  • The activation function of Perceptron is based on the unit step function which outputs 1 if the net input value is greater than or equal to 0, else 0.
  • The prediction is also based on the unit step function.
Perceptron neural network Python (language) Activation function Data science Gradient descent Signal Machine learning Algorithm Net (command)

Published at DZone with permission of Ajitesh Kumar, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • Learning AI/ML: The Hard Way
  • How to Use Python for Data Science
  • Data Science: Scenario-Based Interview Questions
  • The Art of Separation in Data Science: Balancing Ego and Objectivity

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!