Opportunity: Activity and Context Recognition with Opportunistic Sensor Configurations
Activity recognition systems must be robust against possible changes and failures in the sensor network which are typical in open-ended environments. We developed machine learning techniques and heuristics that provide these capabilities, including detection of sensor failure or degradation, dynamic fusion mechanisms and handling of missing data. These methods have been applied to activity recognition from wearable and ambient sensors, as well as BMI scenarios.
Activity recognition in real applications is a challenging task since conditions of the environment cannot be easily controlled. Although multiple sensory modalities can help to make more robust decisions, it increases complexity and multiply the possiblities of failure. We focused on the development of recognition systems that leverage data and robustness in the system to provide graceful degradation and self-correcting characteristics.
A great difficulty in this type of research is the lack of normative datasets and evaluation tools to benchmark novel methods. We developed significant efforts to create a large database to provide a relevant tool to the community. We collected, curated and annotated a dataset for benchmarking multimodal systems for activity recognition. It contains data from 23 body-worn sensors, 12 object-placed sensors, and 21 ambient sensors to measure daily living activities of four subjects. Multiple scientific papers have resulted from this work. The paper describing the data set has been cited 192 times, and the dataset page at the UCI Machine learning repository has been visited more than 80’000 times since 2012.
Resources
- Project web page
- Opportunity dataset at the Machine Learning repository
- Opportunity dataset video. This video describes a large dataset for activity and gesture recognition. Activities are performed in a highly rich sensor environment, and emphasis was placed on collecting a large number of activity instances.
Partners
- Wearable Computing Laboratory, ETHZ
- CNBI, EPFL
- Embedded Systems Laboratory, University of Passau
- Institut für Pervasive Computing, Johannes Kepler Universität Linz
Project funded by the 7th Framework Program of the European Commission from Feb. 2009 to Jan. 2012.
Selected publications
- Sagha, H. et al. Stream fusion for multi-stream automatic speech recognition. International Journal of Speech Technology 19, 669–675 (2016)
- Sagha, H., et al. Quantifying Electrode Reliability During Brain-Computer Interface Operation. IEEE Transactions on Biomedical Engineering 62, 858–864 (2015)
- Chavarriaga, R., et al. Unsupervised adaptation for acceleration-based activity recognition: robustness to sensor displacement and rotation. Personal and Ubiquitous Computing 17, 479–490 (2013)
- Chavarriaga, R. et al. The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition. Pattern Recognition Letters 34, 2033–2042 (2013)
- Roggen, D, et al.Opportunistic human activity and context recognition. IEEE Computer 46 (2013)
- Sagha, H., et al. On-line anomaly detection and resilience in classifier ensembles. Pattern Recognition Letters 34, 1916–1927 (2013)
- Leeb, R., et al. A hybrid brain-computer interface based on the fusion of electroencephalographic and electromyographic activities. Journal of Neural Engineering 8 (2011)