SupplementaryMaterials: BiologicalCredit AssignmentthroughDynamicInversion ofFeedforwardNetworks

Notethattheaccuracyof δl 1 isnotmeasureddirectlyforReLU because it does not have an explicit inversion. This precludes stability forα = 0 and dl > dl 1 (expanding layer), as the matrix productWlBl will be singular. Forexample,inthenonlinearregression experiment shown in the main text, we initialize the SLDI feedback asB = B1B2, whereB1 and B2 are the feedback matrices for sequential DI. Once again, when the controller has no leak, this will produce the same steady state assequential dynamicinversion. We study a simple case here as an illustration, and leave a more thorough analysis for futurework.