A $k$-server verifiable computation (VC) scheme allows a client to outsource a computation $F(x)$ to $k$ servers, receive a partial result from each server, efficiently reconstruct $F(x)$ from the partial results and then verify its correctness. Such a scheme is $t$-private if no collusion of $t$ servers can learn information about the client's input $x$ and $t$-secure if no collusion of $t$ servers can mislead the client into outputting a wrong value by supplying wrong partial results. In a publicly verifiable scheme, the verifier may be different from the client and unexpectedly learn too much information about $x$ from the $k$ partial results. We say that a publicly verifiable scheme is {\em context-hiding} if the verifier cannot learn more information about $x$ from the partial results than what $F(x)$ trivially implies. In this paper, we formally define the context-hiding property and show that some existing schemes do not satisfy this property. We also design a new $k$-server VC scheme that is information-theoretically $t$-private, computationally $t$-secure and context-hiding.