🔒 Locked

Hebbian learning

Started by apocalypstic Jun 4, 2006 at 5:16 PM 4 replies 1k views

Original Post

Jun 4, 2006 at 5:16 PM

I have been wrestling with this spiked neural network problem for a few days. Im trying to think how one could create a spiked neural network (because it is the most biologically realistic) that automatically finds the structure to learn the XOR function. For example if this was the OR function, we would have two neurons for the inputs and one for the output, and hebbian learning would work just fine for correlating the inputs with the output. However for the XOR function we have somehow got to find a way of automatically creating a neuron that will fire when both inputs are on, and have a negative weight to the output. Does anyone know if there is any algorithm for automatically finding the structure? Hebbian style STDP (Spike timing dependant plasticity) is great for correlating existing neurons, but there has to be some way of automatically creating new neurons when the structure requires it.

apocalypstic

Author

apocalypstic

Jun 4, 2006 at 5:44 PM

Quote:
Original post by Anonymous Poster
I'm not a ANN expert, but Hebbian based learning sucks for task learning.

I cant say I agree with that statement, because Hebbian learning actually exists in the brain. Im sure it does have its downfalls compared to other methodologies, but I chose it because it is the most realistic.

apocalypstic

Author

apocalypstic

Jun 5, 2006 at 1:48 AM

Quote:
Original post by Anonymous Poster
Quote:
Original post by apocalypstic
Quote:
Original post by Anonymous Poster
I'm not a ANN expert, but Hebbian based learning sucks for task learning.

I cant say I agree with that statement, because Hebbian learning actually exists in the brain. Im sure it does have its downfalls compared to other methodologies, but I chose it because it is the most realistic.

Hebbian based learning is mostly used for correlation networks, not task learning. Hebbian network can't learn specific tasks. Yes, Hebbian networks exist in the brain, but mostly for stuff like feature extraction. I base this information on the course in Cognitive Modelling I had this spring.

Anyway can we just stick to the problem at hand - that of automatically finding the neuron structure?

Timkin

149

Timkin

Jun 5, 2006 at 9:37 PM

As with any structural model learning problem, you need a means of quantifying the relative 'goodness' of two models. One common approach is to use a penalised log-likelihood measure for models. There are a variety of such information theoretic measures (AIC, BIC, MML, MDL) and investigating the literature with these terms will lead you in roughly the right direction.

Coupled with these, you need an algorithm for suggesting new models given a previous model. This is a search problem in model space (with the aforementioned measure being an appropriate cost/utility function) and you'll find plenty of literature on 'model learning', 'structural learning', etc.

Of course, I'd be surprised if someone hasn't already tackled the specific problem you're looking at, even in the domain of SNNs. Have you done a thorough literature search in this area?

Cheers,

Timkin

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.