DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Please enter at least three characters to search
Refcards Trend Reports
Events Video Library
Refcards
Trend Reports

Events

View Events Video Library

Zones

Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Last call! Secure your stack and shape the future! Help dev teams across the globe navigate their software supply chain security challenges.

Modernize your data layer. Learn how to design cloud-native database architectures to meet the evolving demands of AI and GenAI workloads.

Releasing software shouldn't be stressful or risky. Learn how to leverage progressive delivery techniques to ensure safer deployments.

Avoid machine learning mistakes and boost model performance! Discover key ML patterns, anti-patterns, data strategies, and more.

Related

  • How to Port CV/ML Models to NPU for Faster Face Recognition
  • Unsupervised Learning Methods for Analyzing Encrypted Network Traffic
  • Understanding Neural Networks
  • Understanding the Basics of Neural Networks and Deep Learning

Trending

  • Comparing SaaS vs. PaaS for Kafka and Flink Data Streaming
  • Scalability 101: How to Build, Measure, and Improve It
  • Stateless vs Stateful Stream Processing With Kafka Streams and Apache Flink
  • Immutable Secrets Management: A Zero-Trust Approach to Sensitive Data in Containers
  1. DZone
  2. Data Engineering
  3. AI/ML
  4. Learn the Math for Feedforward Neural Networks

Learn the Math for Feedforward Neural Networks

If you're learning about feedforward neural networks for the first time, understanding the math behind them is a great place to start.

By 
Pranjut Gogoi user avatar
Pranjut Gogoi
·
Jul. 11, 17 · Tutorial
Likes (6)
Comment
Save
Tweet
Share
5.7K Views

Join the DZone community and get the full member experience.

Join For Free

In my last blog, we were discussing the biological motivations of the artificial neural network. This time, we will discuss the artificial neural network in practice. In ANNs, we have different layers of networks to solve a problem. How many layers required to solve certain problems is a different topic — and I will be writing a blog on that soon. However, we can still proceed to implement the network and make it learn how to solve the problem.

As programmers, we understand code better than anyone else and we can pick anything up pretty quickly. You can also learn about ANNs directly by going through the code. However, so far, I feel that knowing the math behind ANN algorithms helps more in understanding the process. So before going to the code, I will be discussing the math behind it. The image below is a feedforward network. There are several network structures in ANNs, but let’s start with this one first.

Image title

As shown in the above image, the network has three layers: input, hidden, and output. In the input layer, we have the inputs as X1, X2, ... Xn. For the middle and hidden layers, we have its outputs as Yh1, Yh2, …Yhn. For the output layer, we have the outputs as Y1, Y2, Y3. Let’s take the targeted outputs as Ŷ1, Ŷ2, … Ŷn. Similarly, we have different weights among different neurons and have named them like W11 between X1 to Yh1; W12 between X1 to Yh2; W13 between X1 to Yh3; etc. We've done it similarly for the output layer neurons, as well. An important thing to note here is that ANN works on real-valued, discrete-valued, and vector-valued inputs.

To summarize, we have the terms as below. If you’re new to neural networks, I would recommend going through them:

  • Inputs = X1, X2, X3

  • Hidden outputs = Yh1, Yh2, Yh3

  • Putputs = Y1, Y2, Y3

  • Targeted outputs = Ŷ1, Ŷ2, Ŷ3

  • Weights to Yh1 = W11, W12, W13

  • Weights to Yh2 = W21, W22, W23

  • Weights to Yh3 = W31, W32, W33

  • Weights to Y1 = W41, W42, W43

  • Weights to Y2 = W51, W52, W53

  • Weights to Y3 = W61, W62, W63

Now, we are all set and ready to implement the network mathematically. Every neuron has an activation function like f(x) = sigmoid(x). The activation function takes a parameter. Our first step is creating the input for the activation function. We do that by multiplying the weights by the input value. The formula looks like this:

  • XWh1 = X1.W11 + X2. W21 + X3. W31

  • XWh2 = X1.W12 + X2. W22 + X3. W32

  • XWh3 = X1.W13 + X2. W23 + X3. W33

The outputs that the hidden layers release are:

  • Yh1 = sigmoid(XWh1)

  • Yh2 = sigmoid(XWh2)

  • Yh3 = sigmoid(XWh3)

The outputs from the hidden layer become the input for the output layer and are multiplied by the weights for the output layer. Therefore, the multiplication is like this:

  • YhWo1= Yh1.W41+Yh2.W51+Yh3.W61

  • YhWo2= Yh1.W42+Yh2.W52+Yh3.W62

  • YhWo3= Yh1.W43+Yh2.W53+Yh3.W63

The final output in the output layer is like this:

  • Y1 = sigmoid(YhWo1)

  • Y2 = sigmoid(YhWo2)

  • Y3 = sigmoid(YhWo3)

If you’re reading about neural networks for the first time, you might be wondering what the Sigmoid function is. Here's the formula for it: 

Pasted image at 2017_06_22 12_07 PM

We can take different activation functions to solve different problems using ANN. When to choose which activation function is, again, a different topic that we will try to cover in another article. But briefly, the Sigmoid function produces an S-shaped curve when put on a graph. When the input for the network is real-valued and differentiable, we use the Sigmoid function so that we can easily find the gradient out of it.

sigmoid

And that’s it! If we implement what we have just mentioned here, our neural network is ready. The next step is to train it. But before we go detail into how to train it, our next step is to see the implementation in Scala — which we are going to see in our next blog!

neural network Network

Published at DZone with permission of Pranjut Gogoi. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • How to Port CV/ML Models to NPU for Faster Face Recognition
  • Unsupervised Learning Methods for Analyzing Encrypted Network Traffic
  • Understanding Neural Networks
  • Understanding the Basics of Neural Networks and Deep Learning

Partner Resources

×

Comments
Oops! Something Went Wrong

The likes didn't load as expected. Please refresh the page and try again.

ABOUT US

  • About DZone
  • Support and feedback
  • Community research
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends:

Likes
There are no likes...yet! 👀
Be the first to like this post!
It looks like you're not logged in.
Sign in to see who liked this post!