Human-Computer Interaction

Implementation of Augmented Reality Interface for Elderly

We consider and implement interfaces (in augmented reality) to reduce various physical and mental problems that elderly people have. We are collaborating with National Cheng Kung University (Taiwan) in implementing a rehabilitation device for fingers using gamification with music.

Implementation of Augmented Reality Interface for Patients Undergoing Rehabilitation

We are also exploring how AR technologies can become effective medical visual aids, to help medical professsionals with faster physical diagnosis of patients, or to assist people with color vision deficiency with color corrections to what they see. We are currently collaborating with University of Trento (Italy) with a project to investigate the effect of visualizing physical information of a patient during sit-to-stand rehabilitation.

  • Michele Stocco, Alessandro Luchetti, Paolo Tomasin, Alberto Fornaser, Patrizia Ianes, Giovanni Guandalini, J. Flores Ty, Sayaka Okahashi, Alexander Plopski, Hirokazu Kato, Mariolino De Cecco, “Augmented Reality to Enhance the Clinical Eye: The Improvement of ADL Evaluation by Mean of a Sensors Based Observation,” 16th EuroVR International Conference (EuroVR 2019), Springer, Vol.LNCS 11883, pp.291-296, Tallinn, Estonia, 23 Oct. 2019.
  • Alexander Plopski, Ada Virginia Taylor, Elizabeth Jeanne Carter, Henny Admoni, “InvisibleRobot: Facilitating Robot Manipulation Through Diminished Reality,” Poster in Proceedings of the International Symposium on Mixed and Augmented Reality, Beijing, China, 14 Oct. 2019
  • Silviya Hasana, Yuichiro Fujimoto, Alexander Plopski, Masayuki Kanbara, Hirokazu Kato, “Improving Color Discrimination for Color Vision Deficiency (CVD) with Temporal-domain Modulation,” International Symposium on Mixed and Augmented Reality, Beijing, China, 14 Oct. 2019

Augmented Reality for Sports Training

Our research aims to facilitate sports training activities by using augmented reality technology.

  • Kaplan, O., Yamamoto, G., Taketomi, T., Yoshitake, Y., Plopski, A., Sandor, C., and Kato, H., Promoting Short-Term Gains in Physical Exercise Through Digital Media Creation, Advances in Computer Entertainment Conference (ACE2017), Dec. 2017.
  • Taketomi, T., Yoshitake, Y., Yamamoto, G., Sandor, C., and Kato, H., 3D Ground Reaction Force Visualization onto Training Video for Sprint Training Support System, International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments (ICAT-EGVE2017), Nov. 2017.
  • Matsumura, H., Watanabe, H., Chen, T. C., Taketomi, T., Yoshitake, Plopski, A., Sandor, C., and Kato, H., Can Face Swapping Technology Facilitate Mental Imagery Training?, International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments (ICAT-EGVE2017), Nov. 2017.

Guideline on Designing and Implementing Augmented Reality Systems for Supporting Assembly Tasks

We proposed a guideline to support developers who aim to design an assisting system using augmented reality. We also published this guideline online (here). Even people who are not familiar with augmented reality can also design applications with high quality specifications.

3D Positioning Method for Handheld Augmented Reality

We propose a 3D positioning method for SLAM-based handheld augmented reality applications. With our interface, user scan efficiently control 3D positioning, such as augmented reality annotation, using a tablet device.

  • Polvi, J., Taketomi, T., Yamamoto, G., Dey, A., Sandor, C., and Kato, H., SlidAR: A 3D Positioning Method for SLAM-based Handheld Augmented Reality, International Journal of Computers and Graphics, Vol.55, pages 33-43, December, 2015.
  • Polvi, J., Taketomi, T., Yamamoto, G., Billinghurst, M., Sandor, C., and Kato, H., Evaluating a SLAM-based Handheld Augmented Reality Guidance System, Poster in Proceedings of the 2nd ACM Symposium on Spatial User Interaction, pages 147-147, New York, NY, USA, ACM, October, 2014.

Human-Robot Interactions (HRI) for Boosting Daily-Use Motivation

Our research focuses on implementing multimodal conversation systems using visual, haptic, and gesture information to maintain daily life as safe, secure, and comfortable as possible. We are focusing on various human robot interfaces, not only physical robots, but also robots in the virtuality-reality spectrum (e.g. augmented reality [AR] robots, virtual robots).

  • Shogo Nishimura, Masayuki Kanbara, Norihiro Hagita, “Atmosphere Sharing with TV Chat Agents for Increase of User’s Motivation for Conversation,” International Conference on Human-Computer Interaction 2019 (HCI International), Springer , pp.482-492, 29 Jul. 2019.

Computer Vision

Body Pose Estimation for Athlete

Movement analysis and visualization of sprinters’ movements can be improved by estimating poses of each body part of the sprinter from a series of RGB images.

Projector-based Augmented Reality

Projection-based augmented reality is a technique that visualizes digital information onto real scene by light projection. Specifically, we are focusing on controlling appearance of real objects based on human perception.

  • Ryo Akiyama, Goshiro Yamamoto, Toshiyuki Amano, Takafumi Taketomi, Alexander Plopski, Christian Sandor, Hirokazu Kato, “Robust Reflectance Estimation for Projection-Based Appearance Control in a Dynamic Light Environment,” Transactions on Visualization And Computer Graphics, IEEE, Sep. 2019
  • Ryo Akiyama, Goshiro Yamamoto, Toshiyuki Amano, Takafumi Taketomi, Alexander Plopski, Christian Sandor, Hirokazu Kato, “Perceptual Appearance Control by Projection-Induced Illusion,” Demo at the IEEE Conference on Virtual Reality and 3D User Interfaces, pp.1-2, Osaka, Japan, Mar. 2019
  • Ryo Akiyama., Goshiro Yamamoto, Toshiyuki Amano, Takafumi Taketomi, Christian Sandor, Hirokazu Kato, Appearance Control in Dynamic Light Environments with a Projector-Camera System, In USB Proceedings of IEEE International Conference on Virtual Reality 2016: Workshop on Perceptual and Cognitive Issues in AR, Greenville, South Carolina, March, 2016.

Computer Graphics

Developing HMDs with Light Field Display Capabilities with Corresponding Research on Rendering Methods

We are working on the development of a HMD model that features a micro-lens array and an eye tracking camera. On top of that, we are also studying a rendering method that provides a clear image based on pupil conditions.

  • Kohei Oshima, Kenneth R Moser, Damien Constantine Rompapas, J Edward Swan, Sei Ikeda, Goshiro Yamamoto, Takafumi Taketomi, Christian Sandor, Hirokazu Kato, Improved Clarity of Defocussed Content on Optical See-Through Head-Mounted Displays, IEEE Symposium on 3D User Interfaces 2016, Greenville, South Carolina, March, 2016.
  • Damien Constantine Rompapas, Kohei Oshima, Sei Ikeda, Takafumi Taketomi, Goshiro Yamamoto, Christian Sandor, Hirokazu Kato, [DEMO] EyeAR: Physically-Based Depth of Field through Eye Measurements, Demo at ISMAR’15: the 14th IEEE International Symposium on Mixed and Augmented Reality, Fukuoka, Japan, October, 2015.