Technology for Smarter Living Spaces

Student researchers aim to equip homes and offices with intelligence.

smart spaces

Seamus Cullinane ’17 designed and 3-D printed Jamie, the quadraped. Here Cullinane consults with mentor Michael Spear about Jamie.

Imagine living in a home that can anticipate your movements—and your moods.

Bad day at the office? Your “smart” home would know that. When you walk through your front door, it would have the lights turned low, soft music playing and a fan creating a gentle breeze to soothe you. Running late at work? No worries. Your home would know that too, waiting until you are 15 minutes from the house to turn up the air conditioning in summer and the heat in winter.

For a cluster of Lehigh students spending the summer at Mountaintop’s Building C, it’s all about “Smart Spaces” – how to track people’s movements, anticipate people’s wants and needs, save energy and build and operate robots that can help people in need.

The multi-pronged Smart Spaces project is part of the Mountaintop initiative and is also a site for the National Science Foundation’s Research Experiences for Undergraduates (REU). Among the projects under way are those that would better integrate technology in the home or office.

“We want a space that knows who’s in it, that knows what’s in it, and that also knows how it can adapt to who and what’s in it,” said Michael Spear, assistant professor of computer science and engineering, about the students and projects he is mentoring under the Smart Spaces umbrella.


John Spletzer, who is heading the REU site at Mountaintop, talks with students about LILI the robot.

John Spletzer, associate professor of computer science and engineering, is heading the REU site at Mountaintop.  Spletzer, director of Lehigh’s Vision, Assistive Devices, and Experimental Robotics Laboratory (VADER lab), says the $340,000 grant from the National Science Foundation (NSF) will fund 30  internships over the next three years, including eight this summer from Lehigh and other universities.

“The seeds for the REU Site were actually sown with last year’s Mountaintop experience in the summer of 2013.  This provided the vision for the current “Smart Spaces” initiative.  The fact that we received a prestigious grant from NSF provides validation of the significance of the research experience.   Nearly every faculty member in our department (Computer Science and Engineering) is supporting this project, and we are grateful for NSF’s support.”

Projects taking place at the Mountaintop’s Building C under the Smart Spaces umbrella include:

  • lili1

    From left, Cassandra Guerrero, Dan Coombe and Chris Garman pose with LILI, the robot. Photo by Mary Ellen Alu

    LILI, a robot, whose acronym stands for Lehigh Instrument for Learning Interaction. Students are developing a low-cost interactive robot that can be used for research with autistic children. LILI was built from scratch and has an avatar head. When fully operational, the robot will be able to recognize faces, gestures and voices, and follow simple commands.

Dan Coombe ’14, who is working on the project with Chris Garman ’16 and Cassandra Guerrero ’16 (Kutztown  University), says autistic  children who interact with the robot would hopefully learn to better interact with people, including how to express themselves better and read other people’s expressions.

Spletzer and Mooi Choo Chuah, professor of computer science and engineering, are mentors.

  • summer 2014 Mountaintop

    Mooi Choo Chuah is helping to mentor students.

    Rapiro, a small humanoid robot.  Stephen Cuzzi ’16 is working to establish a useful application for Rapiro, which was able to do simple things, such as move right or left, or forward or backward. Cuzzi hopes to give the robot “eyes” and teach it to recognize objects, such as a coffee maker, Chuah says. He also wants to control it remotely with a smart phone.

  • Cloud Security. Robert Brotzman-Smith (Kutztown University) and Matthew Hartman (East Stroudsburg University) are investigating a security issue related to cloud computing. Since software applications from different companies may be hosted on the same machine, there is a risk for sensitive data to be compromised without detection, says Chuah. The students want to show how a memory bus-based covert channel can work and then design a simple memory monitoring scheme that can differentiate between the memory access delays observed when legitimate applications or covert channels are running.

In addition to these projects, other students at Mountaintop are working individually on technologies that can be integrated to create a Smart Space. To demonstrate how the projects fit together, students are building a small room inside Building C to install their technologies.

With everyone doing their own thing, the process can be “messy,” admits Patricia Sittikul ’16. But Spear, who is mentoring them, is helping each of them to see the bigger picture, she says.

“These are interesting problems,” says Spear about the many initiatives, “and they’re hard problems.”

Students are developing:

  • smart spaces

    Jamie, the quadraped, is being designed to climb pipes.

    JAMIE, the quadruped. Seamus Cullinane ’17 designed and 3-D printed a four-legged robot with magnetized feet that allows it to climb pipes. The robot is a test platform that could integrate other technologies, such as video streaming and Global Positioning Systems, and possibly be controlled from a computer screen.

  • A Poor Man’s Bar Code. How can you better keep track of the stuff in your house that’s boxed up or stored away? Erik Outhwaite ’16 created an app for an android that would help people keep track of their belongings by drawing shapes on color Post-it notes. He also is working on object tracking to determine who or what is in a room at a particular time.
  • A “Marauder’s Map.” Like the magical document of Harry Potter fame, this GPS system would pinpoint where people are, and where they are not. Shawn Mehltretter (East Stroudsburg University) created a web GUI (graphical user interface) that shows where people are on a Google map. Jon Wu ’17 made an app that sends data to a server with the coordinates of people’s locations.
  • Human Activity Detection. Using cameras, Connor Tench ’15 counts the number of people in a space, tracks their movement and calculates how fast they are walking around in that space.

    summer 2014 Mountaintop

    Some 20 students are part of the Smart Spaces project at Mountaintop.

  • Ambient Intelligence. Can we learn people’s intent? Bruke Mammo ’17 and Laurel Warrell ’15 are using machine-learnings technique to detect whether people living in a smart space are wasting energy, says mentor Hector Muñoz-Avila, associate professor of computer science and engineering. If people act outside their patterns of behavior, that would raise a flag. For example, the system would notice if you fail to shut off your appliances at the time you usually do and then remind you to do so.
  • Interface: Patricia Sittikul is working on an interface that would tie all the Smart Space projects together and enable users to control the technology, whether a phone or computer app.
  • Remote control interface. James Currie ’16 has been building an interface for remote control of lights, fans, and other devices.
  • Speech interface. Geordan Johnson ’15 has been working on speech interfaces to the system.

“I like an environment where there’s a lot of intellectual diversity,” says Spletzer, about the Mountaintop space. If students run into problems they do not know how to solve, he says, they may find fellow students who do. Or they may end up collaborating on projects.

“It’s cool to walk around and see all the different things students are doing,” he says.

smart spaces

Mentor Michael Spear talks with Patricia Sittikul, who is working on an interface to tie all the Smart Space projects together.

In Spear’s group, many of the students are rising sophomores.

“This is the first time most of them had a project that didn’t have a correct answer,” he says. And for many of them, it’s the first time they have to rely on someone else for parts of their project.

“In so many ways,” Spear says, “this is a very unique learning experience.”

Photos by Christa Neu

Story by Mary Ellen Alu




Mountaintop in Motion – Meet Jamie

smart spaces Lehigh University Mountaintop robotThis is Jamie.  Jamie is the hard-working robot dedicated to helping the Smart Spaces team research intelligent living spaces.  Next week we’ll have an overview on this student-led project and what role Jamie plays in IAI.

“Xiphias” November Update

Our team is about a month away from wrapping up work on the official project before I finish at Lehigh and the others branch off with some independent projects next semester. It’s been a wild ride since gaining traction at Mountaintop over the summer, and the finish line is now in sight.  We look forward to keeping you posted through December, but until then, I’ve attached our most recent info-graphic detailing our progress.  Hope you all enjoy it.

– Penn Scott

“Who Knows Where It Will Take Me?”

Jake Puzycki reflected on his experience with this post:

poster image

“My group members and I had quite an interesting experience this summer. We were given a very undefined goal: utilize new technologies and new innovations to help eliminate poverty in small Kenyan communities.

To begin we wanted to see how feasible it was for inexperienced individuals to use new technologies, so we built our own 3D printer: a technology that was new to me, and had blown my mind just months before.

Through that process, we discovered that we were capable of more than we had thought we were. None of us had real experience in engineering, yet we were able to build this complicated device, as long as we had proper guidance, support, and motivation.

That mental framework became the foundation of our project: that people are capable of much more than they may think they are. We decided that the best thing we could do for a group of people, struggling with poverty, was to empower them to help themselves.

So, after undergoing 4 weeks of brainstorming and idea analysis, we decided that we would develop a curriculum that Kenyan community leaders could use to help struggling entrepreneurs in their communities. The curriculum focuses on developing business skills and discovering creative new ways of thinking, that they may solve problems in their communities- benefiting their own lives and giving sustainable growth to their local economies.

I can say for sure that this project has changed the way I think about “work”. It was the first time I’ve set the criteria for what “success” looks like and it built an important bridge between my college experience and “the real world”. I’m really excited for where this project will go, and hope to continue this work through the rest of the school year or even beyond. Who knows where it will take me?”