This paper studies the problem of distributed detection (binary hypothesis testing) over a discrete memoryless channel (DMC) under the constraint that an eavesdropping adversary should not be able to determine whether communication is ongoing or not, i.e., communication over the DMC has to remain covert. The main contribution of the paper is an upper bound on the largest possible Stein exponent, showing that it cannot exceed the largest exponent achievable under zero-rate communication over a noise-free link. In interesting special cases, the upper bound is achieved by a local test at the decision center that completely ignores the communication. In these cases, the covertness constraint thus renders communication useless for improving the Stein exponent.