/sci/ - Science & Math » Thread #12739412

13KiB, 633x320, derivative.png

View Same Google iqdb SauceNAO

noob question

Anonymous Mon 22 Feb 00:15:09 2021 No.12739412 View Reply Original Report

Quoted By: >>12739448

What happened here? How did the equation go from division to multiplication? Is this how derivatives work? pls no bully

Anonymous

Anonymous Mon 22 Feb 2021 00:22:45 No.12739448 Report

Quoted By: >>12739470 >>12739496

>>12739412
I don't know the function the derivative is taken of here, but isn't it just saying that
$\frac{1}{\delta h2_{in1}}= 1-Sigmoid(h2_{in1})$ ?

Anonymous

Anonymous Mon 22 Feb 2021 00:28:51 No.12739470 Report

Quoted By:

>>12739448
That looks reasonable, yet I don't know how to reconcile it.. This is from a backprop calculation, the layer before is Relu, the one after is Softmax. Would that explain it?

Anonymous

Anonymous Mon 22 Feb 2021 00:36:44 No.12739496 Report

Quoted By:

>>12739448
No wait, that's the derivative of the sigmoid, but where did the divider go? Does it disappear when you solve derivatives?

Anonymous

Anonymous Mon 22 Feb 2021 00:45:08 No.12739516 Report

Quoted By: >>12739532

h2_out1 = Sigmoid(h2in1)
When you take the derivative wrt to the input, you get equation 2. This is backprop 101.

Anonymous

Anonymous Mon 22 Feb 2021 00:47:37 No.12739532 Report

Quoted By: >>12739545

>>12739516
Just to add that it does not have to be a sigmoid, it could've been other non linear function like relu. In which case, equation 2 would no longer hold.

Anonymous

Anonymous Mon 22 Feb 2021 00:50:43 No.12739545 Report

Quoted By: >>12739557

>>12739532
One last thing.
https://en.wikipedia.org/wiki/Logistic_function#Derivative

Anonymous

View Same Google iqdb SauceNAO EnApdOxXcAEVPv1.jpg, 32KiB, 680x383

Anonymous Mon 22 Feb 2021 00:56:44 No.12739557 Report

Quoted By:

>>12739545
thanks, fren

Capcode	All Only User Posts Only Moderator Posts Only Admin Posts Only Developer Posts
Show Posts	All Only With Images Only Without Images
Deleted Posts	All Only Deleted Posts Only Non-Deleted Posts
Ghost Posts	All Only Ghost Posts Only Non-Ghost Posts
Post Type	All Only Sticky Threads Only Opening Posts Only Reply Posts
Results	All Grouped By Threads
Order	Latest Posts First Oldest Posts First

Your latest searches