Lesson Learned: Don’t use the Same number of Input as Output in a Feed Forward Network

My first official “failure post” – although I think I have had a couple already – is potentially an amateur lesson, but I learned a lot about the relationship between the inputs and outputs of a system.

I created a simple network in order to test my code. It was the “Red Light, Green Light” network where the input was a two feature vector that determined whether the green light was on, or the red light. The output was another vector with two features, whichever one would be a larger value would determine if it would “go” or “stop”.

The system would not work.

It kept getting a 50 / 50 success (completely random). I spent a ton of time looking into my logic in the code, and then resorted to putting a version of my logic in an Excel file to see if my understanding of the back propagation was correct, and it was still not working no matter what variations I made.

I hit the drawing board on almost everything and questioned everything, until I realized that the network I had setup was a linear network (2 inputs -> 2 Outputs) and not a multi-variable function network (2 inputs -> 1 output).

I changed my network to a single output with a midpoint (0.5) of the single output determining whether the system should go or stop. The system started working and within a couple of iterations was able to learn the rule.

The lesson learned is that there needs to be more inputs, than outputs in order for a feed forward network to work. By trying to simplify things, I overcomplicated things.

I’ve attached my Excel file with all my different attempts if you care to review all I did and what was happening in all the different iterations of the network.

I’ve attached my Excel file with all my different attempts if you care to review all I did and what was happening in all the different iterations of the network.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s