Lies, Damn Lies, And TOPS/Watt

There are almost a dozen vendors promoting inferencing IP, but none of them gives even a ResNet-50 benchmark. The only information they state typically is TOPS (Tera-Operations/Second) and TOPS/Watt. These two indicators of performance and power efficiency are almost useless by themselves. So what, exactly, does X TOPS really tell you about performance for your application? When a vendor says their ABC-inferencing-engine does X TOPS, you would assume that in one second it will perform X Trillion Operations.