The use of medical robotics has been increasing in recent years. This increase in popularity can be attributed to the improvement in dexterity robots provide over traditional laparoscopy, as well as the increasing number of applications of robotic surgery. The daVinci from Intuitive Surgical, one of the more commonly used robotic surgery systems, relies on stereo laparoscopic video for guidance, which restricts visualization to only surface anatomy. Oftentimes the localization of subsurface anatomic structures is critical to the success of surgical intervention. The implementation of image guidance in medical robotics adds the ability to see into the surface; however, current implementations are restrictive in terms of flexibility or scalability, especially in the ability to process real-time video data. We present a system architecture which allows for use of multiple computers through a centralized database; which can fuse additional information to the real-time video stream. This architecture is independent of hardware or software and is extensible to a large number of clinical applications.