math required for Adaline neural network

You need to understand what a gradient is. How much math do you know? Do you understand what a derivative is?

I understand this conceptually from reading high-level descriptions, but I think I need to get to the point where I can work out a problem and see it through. So the answer is no.

Do you know how to compute the derivative of 1/(1+exp(-x))?

I'd have to say no.

I would say if you understand the concept of gradient descent and how to compute the gradient of the error function as a function of the weights, that's about all you need to know.

And what areas/courses would this mostly be taught under so I can find the books to study in-depth?

The area is called multivariable calculus, but you only need the very basics.

Honestly, I can't help you with your application, but I can tell you how the simple calculus works (which I'm assuming is what Alvaro is referring to).

To give an extremely simple introduction, a derivative is the instantaneous rate of change at a specific point on a line. The derivative of a function f(x) (which is notated f'(x)) is found by using the difference quotient:

(This picture's from the Wikipedia article on derivatives; just replace a with x.)

So if f(x) = x², f'(x) = {the limit as h approaches 0 of} ([x+h]² - x²)/h. This may seem like it would equal 0; however, if the h variable is left in during expansion, you get:

{the limit as h approaches 0 of}(x² + 2xh + h² - x²)/h --> {the limit as h approaches 0 of}(2xh + h²)/h --> {the limit as h approaches 0 of} 2x + h = 2x.

Therefore, the derivative of x² is equal to 2x.

There are a lot of rules for derivatives that make derivation a lot easier, e.g. the power rule. This states that the derivative of a polynomial function's derivative can be found by taking each term, multiplying it by its power, and then reducing the power by 1. Thus the derivative of x³ = 3x², and the derivative of 2x² + 3x = 4x + 3, etc.

Now, for Alvaro's function, (1+e^-x)^-1, there are a few other rules involved; its derivative (I'm pretty sure) is -(1+e^-x)^-2-e^-x.

Hope this helps,

Selenaut