11行python代码的神经网络-详细公式推导

ANeuralNetworkin11linesofPython单层神经网络的python实例按照NN的定义，这个应该是单层⽹络，importnumpyasnpdefnonlin(x,deriv=False):if(deriv==True):returnx*(1-x)return1/(1+np.exp(-x))#inputdatasetX=np.array([[0,0,1],[0,1,1],[1,0,1],[1,1,1]])#outputdatasety=np.array([[0,0,1,1]]).T#seedrandomnumberstomakecalculation#deterministic(justagoodpractice)np.random.seed(1)#initializeweightsrandomlywithmean0syn0=2*np.random.random((3,1))-1#bias是0foriterinrange(10000):#forwardpropagationl0=Xl1=nonlin(np.dot(l0,syn0))#howmuchdidwemiss?使用了((y-y')**2)/2的代价函数l1_error=y-l1#multiplyhowmuchwemissedbythe#slopeofthesigmoidatthevaluesinl1l1_delta=l1_error*nonlin(l1,True)#updateweightssyn0+=np.dot(l0.T,l1_delta)#syn0=syn0+np.dot(lo.T,l1_delta)print(OutputAfterTraining:)print(l1)这个例子是使用了[0,0,1],[0,1,1],[1,0,1],[1,1,1]四个输入样本，对应的0,0,1,1四个输出值。需要找到NN的w，也就是程序中的syn0，这个网络只有一个hide层，l1。为此，程序迭代了10000次。这里l1的输出就是y的估值，这里使用的代价函数是ℒ𝑦,𝑦= &'𝑦−𝑦'= &'𝑙1−𝑦'（1）由于刷新syn0（w）的方法是ω= ω− α-ℒ-.（2）α是学习步长，这里取1由（1）式，𝑙1= 11+𝑒1.∙ 34∂ℒ∂ω= ∂12𝑙1−𝑦'∂ω=𝑙1−𝑦∙∂𝑙1∂ω= 𝑙1−𝑦∙𝑙1∙1−𝑙1∙∂ω∙𝑙0∂ω=𝑙1−𝑦∙𝑙1∙1−𝑙1∙𝑙0代入（2）𝑠𝑦𝑛0 =𝑠𝑦𝑛0−𝑙1−𝑦 ∙𝑙1∙1−𝑙1∙𝑙0 1.1.2.双层神经网络的python实例importnumpyasnpdefnonlin(x,deriv=False):if(deriv==True):returnx*(1-x)return1/(1+np.exp(-x))X=np.array([[0,0,1],[0,1,1],[1,0,1],[1,1,1]])y=np.array([[0],[1],[1],[0]])np.random.seed(1)#randomlyinitializeourweightswithmean0syn0=2*np.random.random((3,4))-1syn1=2*np.random.random((4,1))-1forjinrange(60000):#Feedforwardthroughlayers0,1,and2l0=Xl1=nonlin(np.dot(l0,syn0))l2=nonlin(np.dot(l1,syn1))#howmuchdidwemissthetargetvalue?l2_error=y-l2if(j%10000)==0:print(Error:+str(np.mean(np.abs(l2_error))))#inwhatdirectionisthetargetvalue?#werewereallysure?ifso,don'tchangetoomuch.l2_delta=l2_error*nonlin(l2,deriv=True)#howmuchdideachl1valuecontributetothel2error(accordingtotheweights)?l1_error=l2_delta.dot(syn1.T)#inwhatdirectionisthetargetl1?#werewereallysure?ifso,don'tchangetoomuch.l1_delta=l1_error*nonlin(l1,deriv=True)syn1+=l1.T.dot(l2_delta)syn0+=l0.T.dot(l1_delta)print(l2)代价函数ℒ𝑦,𝑦= 12𝑦−𝑦'= 12𝑙2−𝑦'由于刷新syn（w）的方法是ω4= ω4− α∂ℒ∂ω4ω&= ω&− α∂ℒ∂ω&α是学习步长，这里取1𝑙2= 11+𝑒1.∙ 3&∂ℒ∂ω&= ∂12𝑙2−𝑦'∂ω&=𝑙2−𝑦∙∂𝑙2∂ω&= 𝑙2−𝑦∙𝑙2∙1−𝑙2∙∂ω&∙𝑙1∂ω&=𝑙2−𝑦∙𝑙2∙1−𝑙2∙𝑙1𝑠𝑦𝑛1 =𝑠𝑦𝑛1−𝑙2−𝑦∙𝑙2∙1−𝑙2∙𝑙1 𝑙1= 11+𝑒1.∙ 34∂ℒ∂ω4= ∂12𝑙2−𝑦'∂ω4=𝑙2−𝑦∙∂𝑙2∂ω4= 𝑙2−𝑦∙𝑙2∙1−𝑙2∙∂ω&∙𝑙1∂ω4=𝑙2−𝑦∙𝑙2∙1−𝑙2∙ω&∙∂𝑙1∂ω4 =𝑙2−𝑦∙𝑙2∙1−𝑙2∙ω&∙𝑙1∙1−𝑙1∙𝑙0 𝑠𝑦𝑛0 =𝑠𝑦𝑛0−𝑙2−𝑦∙𝑙2∙1−𝑙2∙ω&∙𝑙1∙1−𝑙1∙𝑙0