Distribution free uncertainty quantification in neuroscience-inspired deep operators