Artificial Neural Network

-

                   Artificial Neural Network




An artificial neural network (ANN) is a computational model inspired by the biological neural networks of animal brains. It consists of interconnected nodes (neurons) organized in layers. Each neuron receives input signals, processes them, and generates an output signal.

The XOR problem spurred the development of multi-layer perceptrons (MLPs), showcasing the need for complex models to understand non-linear patterns. This marked a significant advancement in AI. Training neural networks involves iteratively adjusting weights and biases to minimize errors, enabling widespread AI adoption for decision-making, task automation, and personalized recommendations. Inspired by the brain's neural networks, artificial neural networks (ANNs) experienced a resurgence in the 1980s and 1990s, diverging from symbolic AI to offer a more adaptable approach to learning from data. ANNs have since transformed various domains such as image and speech recognition, driving innovations in deep learning and influencing technology's societal impact..

Here's how it works in brief:

  1. Input Layer: The first layer receives input data.
  2. Hidden Layers: Intermediate layers process the data through weighted connections.
  3. Output Layer: The final layer produces the network's output.

During training, the network adjusts the weights of connections between neurons to minimize the difference between actual and desired outputs. This process, called backpropagation, uses optimization algorithms like gradient descent.

Once trained, the ANN can make predictions or classify new data by propagating it forward through the network.


In this example, we’re going to be building a three layer feed forward network, the three being the input layer, hidden layer and output layer as shown above. when we compared to Truth table for LCD we give an input of value which shows an number from 0 to 9.


PatternCount – The number of items/row of training data in your table.
InputNodes – The number of neurons associated with the input data.
Output Nodes – The number of neurons associated with the output data.

For example 
if we wanna get a 8 LCD segments all the input hast to be 1 and if we compare with number 0 
if we change last input to 0 the 8 will change in to 0
Binary is an output.


To run the code, first connect the Arduino in the computer and then upload the code from the link. This code belongs to Ralph Heymsfeld. He has given the rights to use it by anyone.

//Author: Ralph Heymsfeld
//28/06/2018

#include <math.h>

/**********************
 * Network Configuration - customized per network 
 **********************/

const int PatternCount = 10;
const int InputNodes = 7;
const int HiddenNodes = 8;
const int OutputNodes = 4;
const float LearningRate = 0.3;
const float Momentum = 0.9;
const float InitialWeightMax = 0.5;
const float Success = 0.0004;

const byte Input[PatternCount][InputNodes] = {
  { 1, 1, 1, 1, 1, 1, 0 },  // 0
  { 0, 1, 1, 0, 0, 0, 0 },  // 1
  { 1, 1, 0, 1, 1, 0, 1 },  // 2
  { 1, 1, 1, 1, 0, 0, 1 },  // 3
  { 0, 1, 1, 0, 0, 1, 1 },  // 4
  { 1, 0, 1, 1, 0, 1, 1 },  // 5
  { 0, 0, 1, 1, 1, 1, 1 },  // 6
  { 1, 1, 1, 0, 0, 0, 0 },  // 7 
  { 1, 1, 1, 1, 1, 1, 1 },  // 8
  { 1, 1, 1, 0, 0, 1, 1 }   // 9
}; 

const byte Target[PatternCount][OutputNodes] = {
  { 0, 0, 0, 0 },  
  { 0, 0, 0, 1 }, 
  { 0, 0, 1, 0 }, 
  { 0, 0, 1, 1 }, 
  { 0, 1, 0, 0 }, 
  { 0, 1, 0, 1 }, 
  { 0, 1, 1, 0 }, 
  { 0, 1, 1, 1 }, 
  { 1, 0, 0, 0 }, 
  { 1, 0, 0, 1 } 
};

Run the code 

output is 

Training cycle 
each time it changes and reduce the error giving the output and finally It get solved.




Now, we gonna change the code and bring the XOR table



#include <math.h>

/**********************
 * Network Configuration - customized per network 
 **********************/

const int PatternCount = 4;
const int InputNodes = 2;
const int HiddenNodes = 3;
const int OutputNodes = 1;
const float LearningRate = 0.3;
const float Momentum = 0.9;
const float InitialWeightMax = 0.5;
const float Success = 0.0004;

const byte Input[PatternCount][InputNodes] = {
  { 0,0 },  // 0
  { 0,1 },  // 1
  { 1,0 },  // 2
  { 1,1 },  // 3
  
}; 

const byte Target[PatternCount][OutputNodes] = {
  { 0 },  
  { 1 }, 
  { 1 }, 
  { 0 }, 
   
};

and the result you get

Training cycle is very fast when optimising.
You get solved very quick.

we change the pattern count, Input and output nodes from the previous code.

First cycle output

Initial/Untrained Outputs: 

  Training Pattern: 0
  Input 0 0   Target 0   Output 0.27673 
  Training Pattern: 1
  Input 0 1   Target 1   Output 0.27028 
  Training Pattern: 2
  Input 1 0   Target 1   Output 0.26796 
  Training Pattern: 3
  Input 1 1   Target 0   Output 0.26167 

TrainingCycle: 1  Error = 0.60655

  Training Pattern: 0
  Input 0 0   Target 0   Output 0.30410 
  Training Pattern: 1
  Input 0 1   Target 1   Output 0.29860 
  Training Pattern: 2
  Input 1 0   Target 1   Output 0.29505 
  Training Pattern: 3
  Input 1 1   Target 0   Output 0.28964 

TrainingCycle: 1000  Error = 0.00135

  Training Pattern: 0
  Input 0 0   Target 0   Output 0.01486 
  Training Pattern: 1
  Input 0 1   Target 1   Output 0.97724 
  Training Pattern: 2
  Input 1 0   Target 1   Output 0.96900 
  Training Pattern: 3
  Input 1 1   Target 0   Output 0.03147 

TrainingCycle: 2000  Error = 0.00051

  Training Pattern: 0
  Input 0 0   Target 0   Output 0.00857 
  Training Pattern: 1
  Input 0 1   Target 1   Output 0.98563 
  Training Pattern: 2
  Input 1 0   Target 1   Output 0.98122 
  Training Pattern: 3
  Input 1 1   Target 0   Output 0.01961 

TrainingCycle: 2432  Error = 0.00040

  Training Pattern: 0
  Input 0 0   Target 0   Output 0.00748 
  Training Pattern: 1
  Input 0 1   Target 1   Output 0.98719 
  Training Pattern: 2
  Input 1 0   Target 1   Output 0.98339 
  Training Pattern: 3
  Input 1 1   Target 0   Output 0.01742 

Training Set Solved! 
--------

Second Cycle 

Initial/Untrained Outputs: 

  Training Pattern: 0
  Input 0 0   Target 0   Output 0.25949 
  Training Pattern: 1
  Input 0 1   Target 1   Output 0.26417 
  Training Pattern: 2
  Input 1 0   Target 1   Output 0.26299 
  Training Pattern: 3
  Input 1 1   Target 0   Output 0.26827 

TrainingCycle: 1  Error = 0.62150

  Training Pattern: 0
  Input 0 0   Target 0   Output 0.30798 
  Training Pattern: 1
  Input 0 1   Target 1   Output 0.31376 
  Training Pattern: 2
  Input 1 0   Target 1   Output 0.31220 
  Training Pattern: 3
  Input 1 1   Target 0   Output 0.31843 

TrainingCycle: 1000  Error = 0.00131

  Training Pattern: 0
  Input 0 0   Target 0   Output 0.02251 
  Training Pattern: 1
  Input 0 1   Target 1   Output 0.97564 
  Training Pattern: 2
  Input 1 0   Target 1   Output 0.97570 
  Training Pattern: 3
  Input 1 1   Target 0   Output 0.03037 

TrainingCycle: 2000  Error = 0.00052

  Training Pattern: 0
  Input 0 0   Target 0   Output 0.01446 
  Training Pattern: 1
  Input 0 1   Target 1   Output 0.98465 
  Training Pattern: 2
  Input 1 0   Target 1   Output 0.98470 
  Training Pattern: 3
  Input 1 1   Target 0   Output 0.01886 

TrainingCycle: 2476  Error = 0.00040

  Training Pattern: 0
  Input 0 0   Target 0   Output 0.01277 
  Training Pattern: 1
  Input 0 1   Target 1   Output 0.98650 
  Training Pattern: 2
  Input 1 0   Target 1   Output 0.98654 
  Training Pattern: 3
  Input 1 1   Target 0   Output 0.01652 

Training Set Solved !
---------

Network Configuration: Sets up parameters such as the number of patterns (data points), input nodes, hidden nodes, output nodes, learning rate, momentum, initial weight maximum, and success threshold.

Input and Target Data: Defines input and target data patterns. Input patterns represent the input features, while target patterns represent the desired output for each input pattern.

Training Process:

The network goes through multiple training cycles.
For each training cycle, it iterates through the training patterns.
For each pattern, it calculates the output of the network based on the input, compares it with the target output, and adjusts the weights using a training algorithm (likely backpropagation).
It repeats this process until the error falls below a certain threshold (success) or until a maximum number of training cycles is reached.
Output:

It prints out the initial outputs before training.
It then displays the outputs after each training cycle along with the error.
Finally, it indicates when the training set is solved, meaning the network has learned to produce the correct outputs for all input patterns.

The number of training cycles, or iterations, required to train an artificial neural network (ANN) can vary each time it is trained due to several factors:

Random Initialization: ANNs typically start with randomly initialized weights. The initial configuration of weights can influence the convergence of the training process. Different random initializations can lead to different convergence rates, affecting the number of training cycles needed to reach a satisfactory solution.

Learning Rate and Momentum: The learning rate and momentum parameters affect the speed and stability of the training process. Adjusting these parameters can result in different convergence behaviors, impacting the number of training cycles required.

Data Variability: The complexity and variability of the training data can influence the convergence of the network. If the data is more complex or variable, the network may require more training cycles to learn and generalize effectively.

Network Architecture: The architecture of the neural network, including the number of layers, neurons, and connections, can affect the convergence rate. More complex architectures may require more training cycles to converge compared to simpler ones.

Stopping Criteria: Training may stop based on predefined criteria, such as reaching a certain level of accuracy or error threshold. If different stopping criteria are used between training sessions, the number of training cycles may vary.

Noise and Perturbations: Noise or perturbations in the training process, such as fluctuations in the training data or optimization algorithms, can lead to variations in convergence behavior, resulting in different numbers of training cycles required.

Overall, the variability in the number of training cycles highlights the dynamic nature of the training process and the sensitivity of ANNs to various factors involved in learning from data.



Comments

Post a Comment

Popular posts from this blog

Main Project : A Lie Detector

-
Image
In week 12, our teacher introduced us to our main project. He informed us that we now have a basic understanding of Arduino and encouraged us to apply our knowledge creatively to develop something useful. This project will contribute to our final grading, and upon completion, we will be required to create a video demonstrating its functionality. Additionally, he mentioned that the most innovative projects will be showcased at a science fair, where numerous attendees, particularly school students, will have the opportunity to interact with our products. Following the instructions, we formed groups of three individuals. I, Chowdhury Shakline, along with my friends Neelushan Antony and Mahid Ahmed, constituted one such group. After a week of brainstorming various ideas, we collectively decided to create a lie detector. Initially, we deliberated amongst ourselves regarding the necessary kits, the wiring setup with Arduino, and the coding process. Subsequently, we presented our idea to our ...
Read more

Recommender System

-
Image
This week, our teacher introduced us to a fascinating topic: Recommender Systems. Let's provide you with a brief overview of what a recommender system is and how it works. A recommender system is a type of information filtering system that predicts and suggests items or content that a user might be interested in, based on their preferences, past behaviour, or similarities with other users. These systems are widely used in various online platforms such as e-commerce websites, streaming services, social media platforms, and more. Here's how a recommender system typically works: Data Collection : The system gathers data on user interactions, such as purchases, ratings, views, clicks, likes, and other relevant actions. User Profiling : The system creates profiles for individual users based on their preferences, behaviours, and historical interactions with items or content. Item Representation : Similarly, the system analyses items or content available in the system and represents t...
Read more