The field of distributed estimation, computation, testing and learning on graphs has witnessed large advances in theory and wide adoption in practice. However, there do not currently exist any methods for multiple hypothesis testing on graphs that are equipped with provable guarantees on error metrics like the false discovery rate (FDR). In this paper, we consider a novel but natural setting where distinct agents reside on the nodes of an undirected graph, and each agent possesses p-values corresponding to one or more hypotheses local to its node. Each agent must individually decide whether to reject one or more local hypotheses by only communicating with its neighbors, with the joint aim that the global FDR over the entire graph must be controlled at a predefined level. We propose a simple decentralized family of Query- Test-Exchange (QuTE) algorithms and prove that they can control FDR under independence or positive dependence of the p-values. Our algorithm reduces to the Benjamini-Hochberg algorithm when the graph is a star or a clique, and to the Bonferroni procedure when the graph has no edges. We study the power of our procedure using a simulation suite of different levels of connectivity and communication on a variety of graph structures, and also provide an illustrative real-world example.