Patients with carotid atherosclerotic plaques carry an increased risk of cardiovascular events such as stroke. Ultrasound has been employed as a standard for diagnosis of carotid atherosclerosis. To assess atherosclerosis, the intima contour of the carotid artery lumen should be accurately outlined. For this purpose, we use simultaneously acquired side-by-side longitudinal contrast enhanced ultrasound (CEUS) and B-mode ultrasound (BMUS) images and exploit the information in the two imaging modalities for accurate lumen segmentation. First, nonrigid motion compensation is performed on both BMUS and CEUS image sequences, followed by averaging over the 150 time frames to produce an image with improved signal-to-noise ratio (SNR). After that, we segment the lumen from these images using a novel method based on dynamic programming which uses the joint histogram of the CEUS and BMUS pair of images to distinguish between background, lumen, tissue and artifacts. Finally, the obtained lumen contour in the improved-SNR mean image is transformed back to each time frame of the original image sequence. Validation was done by comparing manual lumen segmentations of two independent observers with automated lumen segmentations in the improved-SNR images of 9 carotid arteries from 7 patients. The root mean square error between the two observers was 0.17±0.10mm and between automated and average of manual segmentation of two observers was 0.19±0.06mm. In conclusion, we present a robust and accurate carotid lumen segmentation method which overcomes the complexity of anatomical structures, noise in the lumen, artifacts and echolucent plaques by exploiting the information in this combined imaging modality.