Intro to Deep Learning

This is essentially the machine version of “concept learning.

” You know what a concept is (for example an object, idea, event, etc.

) based on the belief that each object/idea/event has common features.

The idea here is that you can be shown a set of example objects with their labels and learn to classify objects based on what you have already been shown.

You simplify what you’ve learned from what you’ve been shown, condense it in the form of an example, and then you take that simplified version and apply it to future examples.

We really just call this “learning from examples.

”Photo by Gaelle Marcel on Unsplash(Dress that baby up a little and it looks like this: concept learning refers to the process of inferring a Boolean-valued function from training examples of its input and output.

)In a nutshell, supervised machine learning is the task of learning a function that maps an input to an output based on example input-output pairs.

It works with labeled training data made up of training examples.

Each example is a pair that’s made up of an input object (usually a vector) and the output value that you want (also called the supervisory signal).

Your algorithm supervises the training data and produces an inferred function which can be used to map new examples.

Ideally, the algorithm will allow you to classify examples that it hasn’t seen before.

Basically, it looks at stuff with labels and uses what it learns from the labeled stuff to predict the labels of other stuff.

Classification tasks tend to depend on supervised learning.

These tasks might includeDetecting faces, identities, and facial expressions in imagesIdentifying objects in images like stop signs, pedestrians, and lane markersClassifying text as spamRecognizing gestures in videosDetecting voices and identifying sentiment in audio recordingsIdentifying speakersTranscribing speech-to-textSemi-supervised learningThis one is more like the way you learned from the combination of what your parents explicitly told you as a child (labeled information) combined with what you learned on your own that didn’t have labels, like the flowers and trees that you observed without naming or counting them.

Photo by Robert Collins on UnsplashSemi-supervised learning does the same kind of thing as supervised learning, but it’s able to make use of both labeled and unlabeled data for training.

In semi-supervised learning, you’re often looking at a lot of unlabeled data and a little bit of labeled data.

There are a number of researchers out there who have found that this process can provide more accuracy than unsupervised learning but without the time and costs associated with labeled data.

(Sometimes labeling data requires a skilled human being to do things like transcribe audio files or analyze 3D images in order to create labels, which can make creating a fully labeled data set pretty unfeasible, especially when you’re working with those massive data sets that deep learning tasks love.

)Semi-supervised learning can be referred to as transductive (inferring correct labels for the given data) or inductive (inferring the correct mapping from X to Y).

In order to do this, deep learning algorithms have to make at least one of the following assumptions:Points that are close to each other probably share a label (continuity assumption)The data like to form clusters and the points that are clustered together probably share a label (cluster assumption)The data lie on a manifold of lower dimension than the input space (manifold assumption).

Okay, that’s complicated, but think of it as if you were trying to analyze someone talking — you’d probably want to look at her facial muscles moving her face and her vocal cords making sound and stick to that area, rather than looking in the space of all images and/or all acoustic waves.

Unsupervised learning (aka Hebbian Learning)Unsupervised learning involves learning the relationships between elements in a data set and classifying the data without the help of labels.

There are a lot of algorithmic forms that this can take, but they all have the same goal of mimicking human logic by searching for hidden structures, features, and patterns in order to analyze new data.

These algorithms can include clustering, anomaly detection, neural networks, and more.

Clustering is essentially the detection of similarities or anomalies within a data set and is a good example of an unsupervised learning task.

Clustering can produce highly accurate search results by comparing documents, images, or sounds for similarities and anomalies.

Being able to go through a huge amount of data to cluster “ducks” or the perhaps the sound of a voice has many, many potential applications.

Being able to detect anomalies and unusual behavior accurately can be extremely beneficial for applications like security and fraud detection.

Photo by Andrew Wulf on UnsplashBack to it!Deep learning architectures have been applied to social network filtering, image recognition, financial fraud detection, speech recognition, computer vision, medical image processing, natural language processing, visual art processing, drug discovery and design, toxicology, bioinformatics, customer relationship management, audio recognition, and many, many other fields and concepts.

Deep learning models are everywhere!There are, of course, a number of deep learning techniques that exist, like convolutional neural networks, recurrent neural networks, and so on.

No one network is better than the others, but some are definitely better suited to specific tasks.

Deep Learning and Artificial Neural NetworksThe majority of modern deep learning architectures are based on Artificial Neural Networks (ANNs) and use multiple layers of nonlinear processing units for feature extraction and transformation.

Each successive layer uses the output of the previous layer for its input.

What they learn forms a hierarchy of concepts where each level learns to transform its input data into a slightly more abstract and composite representation.

Image by ahmedgad on PixabayThat means that for an image, for example, the input might be a matrix of pixels, then the first layer might encode the edges and compose the pixels, then the next layer might compose an arrangement of edges, then the next layer might encode a nose and eyes, then the next layer might recognize that the image contains a face, and so on.

While you may need to do a little fine tuning, the deep learning process learns which features to place in which level on its own!Photo by Cristian Newman on UnsplashThe “deep” in deep learning just refers to the number of layers through which the data is transformed (they have a substantial credit assignment path (CAP), which is the chain of transformations from input to output).

For a feedforward neural network, the depth of the CAPs is that of the network and the number of hidden layers plus one (the output layer).

For a recurrent neural network, a signal might propagate through a layer more than once, so the CAP depth is potentially unlimited!.Most researchers agree that deep learning involves CAP depth >2.

Convolutional Neural NetworksOne of the most popular types of neural networks is convolutional neural networks (CNNs).

The CNN convolves (not convolutes…) learned features with input data and uses 2D convolutional layers, which means that this type of network is ideal for processing (2D) images.

The CNN works by extracting features from images, meaning that the need for manual feature extraction is eliminated.

The features are not trained!.They’re learned while the network trains on a set of images, which makes deep learning models extremely accurate for computer vision tasks.

CNNs learn feature detection through tens or hundreds of hidden layers, with each layer increasing the complexity of the learned features.

(Want to learn more? Check out Introduction to Convolutional Neural Networks by Jianxin Wu and Yann LeCun’s original article, Gradient-Based Learning Applied to Document Recognition.

)Recurrent neural networksWhile convolutional neural networks are typically used for processing images, recurrent neural networks (RNNs) are used for processing language.

RNNs don’t just filter information from one layer into the next, they have built-in feedback loops where the output from one layer might be fed back into the layer preceding it.

This actually lends the network a sort of memory.

Generative adversarial networksIn generative adversarial networks (GANs), two neural networks fight it out.

The generator network tries to create convincing “fake” data while the discriminator tries to tell the difference between the fake data and the real stuff.

With each training cycle, the generator gets better at creating fake data and the discriminator gets sharper at spotting the fakes.

By pitting the two against each other during training, both networks improve.

(Basically, shirts vs.

skins here.

The home team is playing itself to improve its game.

) GANs can be used for extremely interesting applications, including generating images from written text.

GANs can be tough to work with, but more robust models are constantly being developed.

Deep Learning in the FutureThe future is full of potential for anyone interested in deep learning.

The most remarkable thing about a neural network is its ability to deal with vast amounts of disparate data.

That becomes more and more relevant now that we’re living in an era of advanced smart sensors which can gather an unbelievable amount of data every second of every day.

It’s estimated that we are currently generating 2.

6 quintillion bytes of data every single day.

This is an enormous amount of data.

While traditional computers have trouble dealing with and drawing conclusions from so much data, deep learning actually becomes more efficient as the amount of data grows larger.

Neural nets are capable of discovering latent structures within vast amounts of unstructured data, like raw media for example, which are the majority of data in the world.

The possibilities are endless!Still with me?.Stay tuned for part 2 where we’ll take a deeper look at artificial neural networks!Photo by Andrew Seaman on Unsplash.

. More details

Leave a Reply