Projects

My current projects focus on improving computing education through intelligent, data-driven programming support.

iSnap: Intelligent Programming Support
[ Website | Code | Datasets]

iSnap is a programming environment designed to lower the barriers that novices face when first learning to program. It combines two effective support features: block-based programming and adaptive hints and feedback. iSnap is based on Snap!, an online programming environment where students construct programs from drag-and-drop blocks, which reduce the initial challenges of programming syntax. Snap! makes programming more interactive, visual and creative by letting students build games, stories and simulations.

iSnap augments this environment with detailed logging and intelligent support features including on-demand programming hints and feedback. These hints are generated from student data, allowing them to easily scale to new classrooms and problems (see below for more). Even in a block-based programming, students can still get stuck, and instructors are not always available to help. When this happens, iSnap can check students’ code for errors and offer suggestions for how to move forward. For more information and a demo of iSnap, visit: go.ncsu.edu/isnap.

Key Findings:

  • A pilot study showed qualitative evidence that students are capable of using iSnap to overcome difficulties and complete assignments, but it also revealed important challenges, such as the risk of students abusing the help features [SIGCSE’17].
  • A later study showed that many students perform poorly on homework assignments, but almost all students who followed iSnap’s hints performed adequately (despite the fact that weaker students may be more likely to request hints) [AIED’17].
  • Despite these successes, other work shows that many students do not ask for help when they need it, suggesting that interface and hint-quality issues may be preventing iSnap from having its desired impact on student learning [ICER’17].

Future Directions:

  • Measure the impact of iSnap’s hints and feedback on student learning.
  • Explore new types of intelligent programming support, such as worked examples, principle-based hints, and self-explanation prompts.
  • Support more creative, open-ended programming projects, such as game making and data-analysis, where the learner helps to define the goal of the program.

SourceCheck: Data-driven Hint Generation

SourceCheck is the algorithm that powers iSnap’s data-driven hints and feedback. These hints are generated automatically, with no need for an instructor or expert model. SourceCheck uses the solutions of previous students to build a model of correct programming behavior for a given problem. It can use this model to provide new students with adaptive hints, pointing towards a similar, correct solution. While SourceCheck’s primary application is the iSnap environment, the algorithm is programming language agnostic, and it has also been applied to Python code.

A major component of the SourceCheck research project is to establish methods for evaluating the quality of data-driven hints by using experts to rate and generate hints. This allows researchers to benchmark and compare the growing collection of data-driven programming hint generation algorithms (over 25 since 2015!). Additionally, an objective measure of hint quality allows researchers to test hypotheses about what changes will improve that quality of data-driven hints from a given algorithm.

Key Findings:

  • SourceCheck reproduced the feedback of human tutors 76-88% as well as another tutor for real student hint requests on two programming assignments. However, it also generated up to 50% more hints, suggesting that hint quality hinges on filtering out unhelpful hints [EDM’17].
  • SourceCheck requires only 10-20 student solutions to generate its highest-quality hints. However, data-driven hint quality does not always increase with more data - it can even decrease as more correct student solutions are added to the training dataset [AIED’18].
  • Generating hints with expert solutions, rather than student data, generally yields higher-quality hints. With some hint generation algorithms, a single expert solution outperforms a whole dataset of student solutions [AIED’18].

Future Directions:

  • Create hints that tell a student not just what to do, but also why by combining data-driven approaches with light-weight expert modeling.
  • Make hints more adaptive by leveraging models of student knowledge and detecting student progress through assignments.
  • Extend SourceCheck to additional programming languages and environments.

Understanding Student's Help-seeking Behavior

Many programming help features, such as iSnap’s hints, require the student to initiate a request for help. However, knowing when and how to seek help is a metacognitive skill that many students are still developing. Students may not always ask for help when they need it (help avoidance), or they may ask for too much help, using it to avoid engaging with the assignment (help abuse). Computer-based help may provide unique opportunities to overcome barriers to productive help-seeking, for example providing students help confidentially, without any social cost. This project studies how students seek and use help while programming, with the goal of using better user interface design to encourage productive help-seeking.



Key Findings:

  • In one study, the quality of the very first data-driven hints that a student received in iSnap significantly correlated with how many future hint requests that student made, suggesting that low-quality hints may deter students from seeking help when they need it [AIED’17].
  • However, many students who needed help never asked for it in the first place, despite performing poorly on the assignment. Even if hints were perfect, many students would never see them [AIED’17].
  • Many factors impact students’ decisions to seek help from both instructors and automated help like iSnap, including students’ desire for independence, their trust in the help system, their previous experiences with computer-based help and the accessibility and salience of the help [ICER’17].

Future Directions:

  • Explore how to reframe help features as tools that savvy programmers use, rather than sources of help that may threaten a students’ sense of independence.
  • Develop a model of how students seek and use help when programming and test the model through empirical studies.


Evaluating Programming Interfaces

Block-based, visual programming languages, like Scratch, Alice, Blockly and Snap! are becoming more popular for teaching novices how to code for the first time. These languages replace the textual syntax of programming languages with drag-and-drop blocks or menus that eliminate syntax errors. Intuitively, this should make programming easier for novices, but more empirical research is needed to explore the advantages and limitations of these languages. This project has compared students’ performance when using block-based, textual and frame-based programming environments. The goal of this work is to better understand how to best use these alternative programming interfaces to improve student learning.




Key Findings:

  • Middle school students using a block-based programming interface completed a programming assignment more quickly, more completely and spending more time on task, than those using an otherwise identical textual programming environment [ICER’15].
  • Middle school students using Greenfoot’s Stride frame-based programming interface also progressed through an assignment quicker than their textual counterparts, but the effects were less dramatic. Students using Stride also spent less time correcting syntax errors [ICER’16].

Future Directions:

  • Continue to evaluate cutting-edge programming interfaces as they are developed.
  • Evaluate other elements of novice programming environments, such as their focus on visual, interactive output and creative, open-ended projects, to determine their effect on student learning and engagement.
  • Explore how block-based interfaces interact with programming help features (perhaps differently than in textual environments).

Émigré [ Website | Video ]

Émigré is a prototype game to teach the history of immigration to the United States. The game currently follows Hind, and Lebanese immigrant at the turn of the 20th century, making here way from Mount Lebanon, through Beirut, Marseilles, and eventually Ellis Island to reach the United States. Along the way, she encounters challenges and friends, reflecting the real-life stories of similar immigrants who have made the journey. The game focuses on narrative to tell Hind's story, but weaves in resource management, exploration and random events to keep the game interesting. The game is currently in a prototype phase, and follows Hind only though the end of Beirut.


TuxBlocks [ Web | App | Code ]

TuxBlocks aims to be both a fun math game and a virtual manipulative which can help students visualize algebra problems in a new and useful way. The game plays like a traditional tower defence style game, with players building mazes out of a variety of towers, each with their own special ability, to prevent enemies from crossing from one side of a grid to the other. However, in TuxBlocks, the way the player acquires new towers is by solving algebra problems. Instead of simply asking the player for the solution the equation, forcing them to do a little math as the price for the game content, TuxBlocks integrates the solving process into the gameplay. The equation is represented by movable blocks, which the player can pick up, move around, simplify and modify to help solve the equation. The game rewards successful solving, not only with towers for the tower defence, but also with levels up, which allow players to skip easier arithmetic. TuxBlocks has a variety of difficulty levels, both for the gameplay and the mathematics, allowing students of many achievement levels to enjoy the game. TuxBlocks also features a "Build" mode which allows players to construct their own problems, perhaps even homework problems, and use the solve mechanics in the game to help find a solution.


PlatForge [ App | Code | Art ]

PlatForge is a an Android app that allows users to make, play and share games on their mobile phones and tablets. The app was the product of my undergraduate honors thesis, in which I investigated the applications of game making for the purpose of learning STEM subjects. The goal of PlatForge is not to teach STEM directly, but rather to expose students to concepts like vectors, 2D geometry and programming in a positive, informal setting. This is done in the hope that some of those students will be more receptive to those same concepts when they encounter them in formal academic settings.

The app itself focuses on the 2D platformer genre and includes a built-in physics engine, menu-based scripting, character classes and a customizable UI. It also comes with a set of art assets, from animated character sprites to tilling backgrounds, to allow users to dive right into the game making process. PlatForge has tutorials to get users started, and its menu-based scripting is easy to pick up. The art and code are both licensed under the CC-BY-SA 3.0 license, which means they're available for anyone to use in a non-commercial product.

NC Sunshine Center App [ Android | Code | iOS | Code ]

The Sunshine Center of the North Carolina Open Government Coalition helps inform citizens and the media about the NC "Sunshine" laws, which allow access to public information. It also informs government officials of their obligations under these laws and helps them comply. The Sunshine Center app puts this information into an easy-to-access mobile app for Android and iOS.


PlayN Platforms [ XNA | iOS ]

PlayN is an open-source, cross-platform game engine for PC, Web, Android and iOS. I used PlayN to build TuxBlocks and was impressed with their straightforward API and inventive approach to cross-platform functionality. I've developed a (mostly complete) implementation of the engine for the XNA platform, allowing developers to use Monogame to target even more platforms. I've also partially developed an iOS backend for PlayN which does not require an expensive MonoDevelop license, but instead uses j2objc to translate the app directly to objective-c. Game engines are one of my favorite projects to work on, and these implementations have presented some unique and enjoyable challenges.


PaperEngine [Code]

PaperEngine is a prototype 2D game engine I've been working on in my free time. The objective is to create a component-style game engine in the style Unity3D, but to target it solely at 2D games. There are a number of great game engines already out there (this one is built on PlayN), but PaperEngine is also a development environment. The object of the engine is to allow developers to manage assets and build levels visually, while coding reusable components. The project is only an experiment at the moment, but there's something particularly appealing to me about plug-and-play components that can be shared by developers for an open-source game engine. So when I have some free time, I work on PaperEngine.


Team Fractal [ Web | Code ]

I have a long-standing fascination with fractals. I like to say the first program I ever wrote was a TI84-BASIC program to render the Mandelbrot set in grainy black and white. (It took the better part of 40 minutes too!) TeamFractal was a weekend project that came out of a desire to make a super-high-resolution image of the Mandelbrot set. The website uses javascript to render the set, and then saves any rendered images to its database. The more people who use the app, the less rendering there needs to be.


Gaggle [ Game Page | Code ]

I built Gaggle with Nathan Shirley for the 2014 Global Game Jam using the Slick2D game engine. The game is based on the idea of natural selection. The player oversees a "gaggle" of odd-looking creatures as they try to make their way to the objective. Sadly for the little creatures, there's always an obstacle in the way. The only way the player can help is by deciding which genetic material gets passes on to the next generation of creatures. (They have a rather quick lifecycle.) Starting with a random population (in both their physical attributes and behaviors), players must shape their "gaggle" to overcome the challenge of the level.


2D Lighting [Code]

I was much less satisfied with my attempt at Global Game Jam 2013. (The concept was cool: a platformer where you play as the level and try to prevent the hero from making it through... but the implementation was lacking.) When I got home, I decided I wanted to make something interesting before the day was over. Around 3am, I had this demo, a simple 2D light engine in the style of Monoco. I'm sure there's a better implementation somewhere around the internet, but I don't want to look for it. I'm happy with my own small creation.


Awesomenauts Utils [ Replay Parser | AwesomeScript ]

Awesomenauts is a delightful and silly game. Think League of Legends or DotA as a platformer that doesn't take itself seriously. My friends and I wanted to keep track of our stats, so I wasted the better part of two weeks' evenings trying to parse our replays. The files were bit-packed and had no documentation. The only clue I had to go on was that coordinates were 13-bits each. Well, I'm far from discovering all of their secrets, but darned if I didn't get the positioning information out of the replays. It's a silly parser for a silly game, but I'm proud of it.