The goal of the Vital Responder research project is to explore the synergies between innovative wearable technologies, scattered sensor networks, intelligent building technology and precise localization services to provide secure, reliable and effective first-response systems in critical emergency scenarios. Critical events, such as natural disaster or other large-scale emergency, induce fatigue and stress in first responders, such as fire fighters, policemen and paramedics. There are distinct fatigue and stress factors (and even pathologies) that were identified among these professionals. Nevertheless, previous work has uncovered a lack of real-time monitoring and decision technologies that can lead to in-depth understanding of the physiological stress processes and to the development of adequate response mechanisms. Our “silver bullet” to address these challenges is a suite of non-intrusive wearable technologies, as inconspicuous as a t-shirt, capable of gathering relevant information about the individual and disseminating this information through a wireless sensor network. In this paper we will describe the objectives, activities and results of the first two years of the Vital Responder project, depicting how it is possible to address wearable sensing challenges even in very uncontrolled environments.