Augmented Reality Cookies (and Sushi)

Chris (cnelson) showed me a video about augmented reality cookies, and I knew that had to be the next baking project. The premise behind this is to hold a cookie with a special mark on it, and a webcam would render a shape over the cookie, even if you rotated it. We were going to make it so that when you bit into a cookie, you’d get a silly mustache on your face.

Of course, I waited til last minute to make the dough, and was trying to measure out 1cm tall chunks at 2am. But the bigger problem was that the dough was defrosting fast as I was trying to build the cookies, and I was worried that it wouldn’t be smooth enough to scan.

But the cookie block did look fine the next day, and sliced easily. We laid them onto cookie sheets and baked them. In Chris’s sunny computer room, they did scan. However, the program was having problems with the contrast. Next time, add more blue and red food coloring, as well as more melted chocolate, to give your cookies more contrast.


Chris was in charge of making the code work with the cookies. He used Processing and NyARToolkit to make that happen.

I also made some onigiri rice balls with the trigger shape cut out, and some quick California rolls. The rolls didn’t trigger the software, but the riceballs did. We showcased these at Hacker Drinkup 100, at Rick’s Tavern.



I drew a bunch of mustaches that would display randomly. Here is what your options were:


Screenshots from Chris’s App:

Eliot’s Blog Post:

My Flickr Set of Hacker Drinkup 100:

I think we’re a bit obsessed with augmented reality now, so expect more cool stuff from us soon!