News and status updates (August, 2014)

  • Course page for CSC554: Human-Computer Interaction (001), Fall, 2014.

  • Kyung Wha Hong has just successfully defended her Ph.D. dissertation. Congratulations to Kyung Wha!

  • Chris Healey and I are starting up an NSF-funded project on CAPTIVE, a Cube with Augmented Physical Tools, for exploration of three-dimensional information, with graduate research assistant William Hill.

  • Emerson Murphy-Hill and I are starting up an NSF-funded project on Cognitive modeling of strategies for dealing with errors in mobile touch interfaces, with graduate research assistant Prairie Rose Goodwin.

  • Dave Roberts and I continue to work on Low-level Analytics Models of Cognition for Novel Security Proofs.

  • Thomas Horton and I are working on cognitive modeling and sensemaking, with graduate research assistant Liang Dong.

  • My fifteen minutes.

Research

This is a brief summary of current research in my lab. Images for the different research areas are linked to representative videos.

Embodied cognitive models

CogTool experiment results

In the intersection between computer science and psychology we find embodied cognitive models: computational simulations of human performance on specific tasks. Projects in my lab include modeling of vision, gesture, and interaction with mobile devices, based on existing and novel cognitive architectures; we are also exploring brain computer interfaces. Current students: Sina Bahram, Kyung Wha Hong, Shea McIntee, Huseyin Sencan.

Accessibility and intelligent user interfaces

CAVIAR device

Techniques from artificial intelligence and related areas have great promise for improving interactive systems. One project in my lab, TIKISI (Touch It, Key It, Speak It), helps blind users interact with graphical information such as maps; a past project, CAVIAR, uses a specialized wristband and computer vision algorithms running on a mobile phone to guide a blind person's hand toward specific objects. Other work focuses on magnifying interfaces for people with vision impairment and improving mobile recommender systems. Current students: Sina Bahram, Pat Cash, Jeff Wilson.

Tool-based user interfaces

Augmented reality in a cube

Tool use is a hallmark of intelligent behavior, but current interactive systems do not fully exploit our abilities. A project called CAPTIVE began in the summer of 2013, in collaboration with Jae Yeol Lee at Chonnam National University, Korea. CAPTIVE is an augmented reality/tangible user interface system for dealing with 3D information: the user holds a physical cube, watching it through a display with a camera mounted on the back (a stereo configuration is in progress), and sees virtual objects that track the cube's movement. We have also built a tool-based user interface for managing documents on a large touch surface. Current students: William Hill, Sina Bahram, Prairie Rose Goodwin, Kyung Wha Hong, Shea McIntee.

Results

  • Technical papers.

  • Software: CAVIAR (Computer-vision Assisted Vibrotactile Interface for Accessible Reaching) is available for download. I maintain a set of AI planning systems written in Common Lisp intended for classroom use. (People may find the AI Planning Resources page useful too.) Two older systems may be of historical interest: G2A translates high-level procedural GOMSL models into detailed cognitive ACT-R 4 models. SegMan is a perceptual substrate that uses simple image processing techniques to "see" the Microsoft Windows graphical user interface.

Students

  • Sina Bahram, Ph.D. candidate. Accessibility, intelligent user interfaces.
  • Pat Cash, Ph.D. candidate. Intelligent user interfaces, mobile search.
  • Prairie Rose Goodwin, Ph.D. candidate. Human-computer interaction.
  • Shea McIntee, Ph.D. candidate. Gesture-based interaction and modeling.
  • Huseyin Sencan, Ph.D. candidate. Brain-computer interfaces.
  • Jeff Wilson, Ph.D. candidate. Accessibility.

Past students

  • Kyung Wha Hong, Ph.D., 2014. Improving interface usability through model transformation using interaction design models.
  • Arpan Chakraborty, Ph.D., 2014. A biologically inspired active vision framework for cognitive agents. (Now at Udacity.)
  • Shishir Kakaraddi, M.S., 2012. A comparison of summarization techniques for small sets of micro blogs. (Now at VMware.)
  • Yanglei Zhao, M.S., 2011. Gibbon: A wearable device for pointing gesture recognition. (Now at TransLoc.)
  • Thomas Horton, Ph.D., 2011. A partial contour similarity-based approach to visual affordances in habile agents.
  • Marivic Bonto-Kane, Ph.D., 2010. Statistical modeling of human response times for task modeling in HCI. (Now at the Naval Medical Information Management Center.)
  • Reuben Cornel, M.S., 2009. Coglaborate -- An environment for collaborative cognitive modeling. (Now at Salesforce.)
  • Lloyd Williams, Ph.D., 2009. Dynamic ontology driven learning and control of robotic tool using behavior. (Now a professor at Shaw University.)
  • Wei Mu, Ph.D., 2009. A schematic representation for cognitive tool-using agents. (Now at Microsoft.)
  • Lucas Layman. Ph.D., 2008 (co-chair with Laurie Williams). Information needs of developers for program comprehension during software maintenance tasks. (Now at the Fraunhofer Center for Experimental Software Engineering, University of Maryland.)
  • James Ward, M.S., 2008. A comparison of fuzzy logic spatial relationship methods for human robot interaction. (Now at U.S. Army Research Office.)
  • Chaya Narayanan Kutty, M.S., 2008. Toward video games on video. (Now at Cisco Systems.)
  • Kevin Damm, M.S., 2008. Incorporating student note-taking into online intelligent computer-assisted instruction. (Now at Google.)
  • Andrea Dawkins, M.S., 2007. Personalized hierarchical menu organization for mobile device users. (Now at Entrinsik.)
  • Kenya Freeman, Ph.D., 2006 (Psychology, co-chair with Eric Wiebe). The effects of automated decision aid reliability and algorithm modality on reported trust and task performance. (Now at LexisNexis Group.)
  • Curtis Boyce, M.S., 2006. Video-based augmented reality for robot navigation. (Now at GlaxoSmithKline.)
  • Sean P. McBride, M.S., 2005. Data organization and abstraction for distributed intrusion detection. (Now at the Washington Post Company.)
  • Alexander Wood, M.S., 2005. Effective tool use in a habile agent. (Now at Grayhawk Systems.)
  • Lloyd Williams, M.S., 2005. Opening the Black Box on Statistical Modeling, The Theory Behind VisualBayes.
  • Thomas Horton, M.S., 2004. HabilisDraw: a tool-based direct manipulation software environment.
  • Bradley Siegler, M.S., 2004. Supporting electronic CRC card sessions with natural interaction.
  • Colin G. Butler, M.S., 2004. Exploring bimanual tool-based interaction in a drawing environment.
  • Nihar Namjoshi, M.S., 2004. Web information retrieval using Web document structures. (Now at Microsoft.)
  • Martin Dulberg, Ph.D., 2003. A task-based evaluation framework for comparing input devices. (Now at DELTA, North Carolina State University.)
  • Ajay Dudani, M.S., 2003. User interface softbots. (Now at Qualcomm Innovation Center.)
  • Kunal Shah, M.S., 2003. Image processing for cognitive models in dynamic gaming environments. (Now at Adobe Systems.)
  • Sameer Rajyaguru, M.S., 2003. Image processing substrate to assist cognitive models interact with dynamic environments. (Now at Amazon.)
  • Mark O. Riedl, M.S., 2001. A computational model of navigation in social environments. (Now a professor at Georgia Tech.)
  • Troy Tolle, M.S., 2000. IDIOM: An intelligent, dynamically manipulable simulation for high school physics Education. (Now at Digital Chalk.)
  • T. Edward Long, M.S., 1999. A navigation testbed.

Awards

  • Outstanding Teacher Award, North Carolina State University, 2013.
  • Best Paper (with Reuben Cornel and Jeff Shrager), 19th Behavior Representation in Modeling & Simulation (BRIMS) Conference, Charleston, SC, 2010.
  • Best Paper (with Lucas Layman and Laurie Williams), First International Symposium on Empirical Software Engineering and Measurement (ESEM), Madrid, Spain, 2007.
  • Best Applied Paper (with Frank Ritter, Penn State University), Sixth International Conference on Cognitive Modeling (ICCM), Pittsburgh, August, 2004.
  • Outstanding new teacher, Department of Computer Science, North Carolina State University, 1999.
  • Recognition of special service, Office of the Army (Heeresamt), Cologne, Germany, 1991.