Talbot-Lau grating interferometry enables the use of clinical x-ray tubes for phase contrast imaging, greatly broadening its utility for both laboratory and preclinical applications. However, phase contrast measurements made in porous or highly heterogeneous media are negatively impacted by low visibility, the interferometer signal amplitude used to calculate relative phase shifts. While this loss in visibility is the source of dark field contrast it presents an additional source of noise in phase images. In this work, we develop a method to use normalized visibility images as the weighting matrix for denoising the corresponding phase contrast images. By using the visibility to guide filtering, the resulting denoised images are locally smoothed in regions of low visibility while maintaining spatial detail in regions of high visibility. This work demonstrates how the complementary properties of the dark field signal in grating interferometry can be leveraged to improve image quality in phase contrast images and presents an application in preclinical lung micro-CT.