National Science Foundation

Student researchers aim to equip homes and offices with intelligence.

Seamus Cullinane ’17 designed and 3-D printed Jamie, the quadraped. Here Cullinane consults with mentor Michael Spear about Jamie.

Imagine living in a home that can anticipate your movements—and your moods.

Bad day at the office? Your “smart” home would know that. When you walk through your front door, it would have the lights turned low, soft music playing and a fan creating a gentle breeze to soothe you. Running late at work? No worries. Your home would know that too, waiting until you are 15 minutes from the house to turn up the air conditioning in summer and the heat in winter.

For a cluster of Lehigh students spending the summer at Mountaintop’s Building C, it’s all about “Smart Spaces” – how to track people’s movements, anticipate people’s wants and needs, save energy and build and operate robots that can help people in need.

The multi-pronged Smart Spaces project is part of the Mountaintop initiative and is also a site for the National Science Foundation’s Research Experiences for Undergraduates (REU). Among the projects under way are those that would better integrate technology in the home or office.

“We want a space that knows who’s in it, that knows what’s in it, and that also knows how it can adapt to who and what’s in it,” said Michael Spear, assistant professor of computer science and engineering, about the students and projects he is mentoring under the Smart Spaces umbrella.

John Spletzer, who is heading the REU site at Mountaintop, talks with students about LILI the robot.

John Spletzer, associate professor of computer science and engineering, is heading the REU site at Mountaintop. Spletzer, director of Lehigh’s Vision, Assistive Devices, and Experimental Robotics Laboratory (VADER lab), says the $340,000 grant from the National Science Foundation (NSF) will fund 30 internships over the next three years, including eight this summer from Lehigh and other universities.

“The seeds for the REU Site were actually sown with last year’s Mountaintop experience in the summer of 2013. This provided the vision for the current “Smart Spaces” initiative. The fact that we received a prestigious grant from NSF provides validation of the significance of the research experience. Nearly every faculty member in our department (Computer Science and Engineering) is supporting this project, and we are grateful for NSF’s support.”

Projects taking place at the Mountaintop’s Building C under the Smart Spaces umbrella include:

From left, Cassandra Guerrero, Dan Coombe and Chris Garman pose with LILI, the robot. Photo by Mary Ellen Alu

LILI, a robot, whose acronym stands for Lehigh Instrument for Learning Interaction. Students are developing a low-cost interactive robot that can be used for research with autistic children. LILI was built from scratch and has an avatar head. When fully operational, the robot will be able to recognize faces, gestures and voices, and follow simple commands.

Dan Coombe ’14, who is working on the project with Chris Garman ’16 and Cassandra Guerrero ’16 (Kutztown University), says autistic children who interact with the robot would hopefully learn to better interact with people, including how to express themselves better and read other people’s expressions.

Spletzer and Mooi Choo Chuah, professor of computer science and engineering, are mentors.

Mooi Choo Chuah is helping to mentor students.

Rapiro, a small humanoid robot. Stephen Cuzzi ’16 is working to establish a useful application for Rapiro, which was able to do simple things, such as move right or left, or forward or backward. Cuzzi hopes to give the robot “eyes” and teach it to recognize objects, such as a coffee maker, Chuah says. He also wants to control it remotely with a smart phone.

Cloud Security. Robert Brotzman-Smith (Kutztown University) and Matthew Hartman (East Stroudsburg University) are investigating a security issue related to cloud computing. Since software applications from different companies may be hosted on the same machine, there is a risk for sensitive data to be compromised without detection, says Chuah. The students want to show how a memory bus-based covert channel can work and then design a simple memory monitoring scheme that can differentiate between the memory access delays observed when legitimate applications or covert channels are running.

In addition to these projects, other students at Mountaintop are working individually on technologies that can be integrated to create a Smart Space. To demonstrate how the projects fit together, students are building a small room inside Building C to install their technologies.

With everyone doing their own thing, the process can be “messy,” admits Patricia Sittikul ’16. But Spear, who is mentoring them, is helping each of them to see the bigger picture, she says.

“These are interesting problems,” says Spear about the many initiatives, “and they’re hard problems.”

Students are developing:

Jamie, the quadraped, is being designed to climb pipes.

JAMIE, the quadruped. Seamus Cullinane ’17 designed and 3-D printed a four-legged robot with magnetized feet that allows it to climb pipes. The robot is a test platform that could integrate other technologies, such as video streaming and Global Positioning Systems, and possibly be controlled from a computer screen.

A Poor Man’s Bar Code. How can you better keep track of the stuff in your house that’s boxed up or stored away? Erik Outhwaite ’16 created an app for an android that would help people keep track of their belongings by drawing shapes on color Post-it notes. He also is working on object tracking to determine who or what is in a room at a particular time.

A “Marauder’s Map.” Like the magical document of Harry Potter fame, this GPS system would pinpoint where people are, and where they are not. Shawn Mehltretter (East Stroudsburg University) created a web GUI (graphical user interface) that shows where people are on a Google map. Jon Wu ’17 made an app that sends data to a server with the coordinates of people’s locations.

Human Activity Detection. Using cameras, Connor Tench ’15 counts the number of people in a space, tracks their movement and calculates how fast they are walking around in that space.
Some 20 students are part of the Smart Spaces project at Mountaintop.

Ambient Intelligence. Can we learn people’s intent? Bruke Mammo ’17 and Laurel Warrell ’15 are using machine-learnings technique to detect whether people living in a smart space are wasting energy, says mentor Hector Muñoz-Avila, associate professor of computer science and engineering. If people act outside their patterns of behavior, that would raise a flag. For example, the system would notice if you fail to shut off your appliances at the time you usually do and then remind you to do so.

Interface: Patricia Sittikul is working on an interface that would tie all the Smart Space projects together and enable users to control the technology, whether a phone or computer app.

Remote control interface. James Currie ’16 has been building an interface for remote control of lights, fans, and other devices.

Speech interface. Geordan Johnson ’15 has been working on speech interfaces to the system.

“I like an environment where there’s a lot of intellectual diversity,” says Spletzer, about the Mountaintop space. If students run into problems they do not know how to solve, he says, they may find fellow students who do. Or they may end up collaborating on projects.

“It’s cool to walk around and see all the different things students are doing,” he says.

Mentor Michael Spear talks with Patricia Sittikul, who is working on an interface to tie all the Smart Space projects together.

In Spear’s group, many of the students are rising sophomores.

“This is the first time most of them had a project that didn’t have a correct answer,” he says. And for many of them, it’s the first time they have to rely on someone else for parts of their project.

“In so many ways,” Spear says, “this is a very unique learning experience.”

Photos by Christa Neu

Story by Mary Ellen Alu