The feed-forward networks with fixed hidden units (FllU-networks) are compared against the category of remaining feed-forward net(cid:173) works with variable hidden units (VHU-networks). Two broad classes of tasks on a finite domain X C R n are considered: ap(cid:173) proximation of every function from an open subset of functions on X and representation of every dichotomy of X. For the first task it is found that both network categories require the same minimal number of synaptic weights. For the second task and X in gen(cid:173) eral position it is shown that VHU-networks with threshold logic hidden units can have approximately lin times fewer hidden units than any FHU-network must have.

In this paper, we propose a compositional nonparametric method in which a model is expressed as a labeled binary tree of $2k+1$ nodes, where each node is either a summation, a multiplication, or the application of one of the $q$ basis functions to one of the $p$ covariates. We show that in order to recover a labeled binary tree from a given dataset, the sufficient number of samples is $O(k\log(pq)+\log(k!))$, and the necessary number of samples is $\Omega(k\log (pq)-\log(k!))$. We further propose a greedy algorithm for regression in order to validate our theoretical findings through synthetic experiments.

The feed-forward networks with fixed hidden units (FllU-networks) are compared against the category of remaining feed-forward networks with variable hidden units (VHU-networks).

The feed-forward networks with fixed hidden units (FllU-networks) are compared against the category of remaining feed-forward networks with variable hidden units (VHU-networks).

The feed-forward networks with fixed hidden units (FllU-networks) are compared against the category of remaining feed-forward networks withvariable hidden units (VHU-networks).