Although computers weren’t standard in the workplace until the 1980s, early computers were developed decades earlier. In 1944, Colossus was used at Bletchley Park by code breakers trying to decipher German military communications. Colossus was programmable, electronic, and digital, and the first of its kind. The Electronic Numerical Integrator and Computer (ENIAC) followed Colossus in the 1940s. ENIAC was the first programmable electronic computer in the United States in the 1940s.
Albert Turing theorized about a universal computer in 1936 and also built a precursor to modern computers. While John Mauchly and J. Presper Eckert were building ENIAC at the University of Pennsylvania, mathematician Walter Pitts and neurophysiologist Warren McCulloch developed the first artificial neural network (ANN) at the Massachusetts Institute of Technology (MIT). Also known as neural networks (NNs), ANNs are modeled after the human brain.
Walter Pitts and Warren McCulloch developed the Threshold Logic Unit (TLU), or Linear Threshold Unit.
Pitts and McCulloch used electrical circuits to demonstrate how the human brain functions and pave the way for artificial neurons. At the time, Warren and Walter demonstrated the viability of using neurons with Albert Turing’s proposed universal computing machines. McCulloch developed brain theories, and Pitts developed theories about neural activity. Together, they helped pave the way for developing cybernetics and artificial intelligence. They also developed strategies for creating neural nets. The TLU operated by receiving inputs, applying a weighted sum, and evaluating the result. It generated an output of 0 or 1 based on whether the result was above or below a predetermined linear threshold gate.
Donald O. Hebb’s book explored neural pathways.
Canadian Donald O. Hebb’s book, The Organization of Behavior, was published in 1949. Hebb developed the theory of Hebbian learning, which was introduced in The Organization of Behavior. His work advanced neuroscience, education, and cybernetics because it expanded people’s understanding of the human brain and brain functions. Hebb explored how synaptic activity strengthens or weakens synapses.
Hebb’s work laid the foundation for Rosenblatt.
Cornell psychologist Frank Rosenblatt was attempting to reconstruct how a fly’s eye worked. He posited that the eye sparked a chain of thought processes that prompted flies to decide whether to flee in specific situations. This led to the development of the Mark I Perceptron in 1958. Perceptron was based on the McCulloch-Pitts neuron, but Perceptron could learn weights through repetition. It was limited to linear functions, but it advanced the development of NNs and machine learning.
ANNs started solving real problems.
Marcian Hoff and Bernard Widrow took a significant step forward when they used neural networks named ADALINE and MADALINE to solve a real problem. Anyone who’s had a telephone call has benefited from MADALINE’s work because MADALINE eliminated phone line echoes with an adaptive filter, improving the quality of telephone calls.
One key difference between Hoff and Widrow’s creations and earlier iterations is that the weighted sums varied based on the values of the inputs. MADALINE stands for Many ADALINE and uses a three-layer ANN. The layers included an input layer, hidden layer, and output layer.
The hidden layer of an ANN processes data. It’s called a hidden layer because the variables for the mathematical computations performed can vary. When working with more datasets, you may need more hidden layers to process data.
NNs have multiple applications today.
Artificial neural networks have multiple practical applications. Software companies can develop neural network software, which simulates artificial neural networks. This software has multiple applications for machine learning and artificial intelligence and can help artificial intelligence make predictions. In the medical field, deep neural networks (DNNs) process inputs to make a medical diagnosis.
From smartphones to video games to personal computers, speech recognition has become a part of everyday life in modern society. NNs make speech recognition possible, allowing people to operate hands-free with verbal commands.
The history of ANNs stretches back over the past nine decades. Early neural network research paved the way for digital computers, and in recent years, ANNs have been applied to various technological devices to alter and enhance their functions.