Making Machine Learning Smarter

Making Machine Learning Smarter

Technology today offers new products with “smart” and “intelligent” features to help simplify things in our homes, such as a thermostat that adapts to a user’s preference and adjusts automatically to fit those preferences or lights and window shades that track movement, time, and weather to set the ideal environment throughout the day. These devices are marketed as sleek and simple. We envision the connected home of the future where, as a result of “smart” interfaces and algorithms, these products become aware of our patterns and preferences, and even anticipate our needs. So instead of manually programming your thermostat, smart technology learns your daily patterns and automatically adjusts itself around your schedule and preferences.

But are today’s devices actually learning?

The Internet of Things (IoT) describes a world of smart, connected devices including home automation devices, wellness and fitness wearables, etc, that collect, relay and exchange data. Vonage, a leading cloud communications services provider, recently partnered with NYC Media Lab on a research project to consider the potential of IoT and its impact on our everyday lives.

Vonage and NYC Media Lab convened a team from NYU’s Tandon School of Engineering to examine IoT products to develop a perspective on what’s possible with machine learning platforms and algorithms, and to determine which algorithms would be best suited for the various types of learning that would be required in the IoT field.

Forming the core team

Baruch Sterman, Vice President of Technology Research, led the Vonage team, which included two colleagues with backgrounds in networks, communications devices and data analytics, Deepak Ottur, Senior Director of Network Services and Ajay Sathyanath, Senior Director of Big Data.

Vonage outlined high level questions about the possibilities of IoT relative to the current, state-of-the-art machine learning platforms and algorithms. NYC Media Lab then posed those questions to the NYU Tandon School of Engineering faculty lead on the project, Yong Liu, Associate Professor of Electrical and Computer Engineering to gain insight from his expertise on the subject. Yong’s research background in systems, networks and machine learning, coupled with colleagues Chenguang Yu, PhD Candidate in Electrical and Computer Engineering and Guangyu Li, PhD Candidate in Electrical and Computer Engineering, would serve the team well.

The 10-week project began with extensive research about IoT and related products. The core team held weekly meetings to exchange updates on their findings, enabling real-time collaboration.

At the culmination of the research phase, the team succeeded in gathering a comprehensive list of the algorithms and methods used to support machine learning. This document is a valuable asset for Vonage, as it allows developers and architects to hone in on the best options for applying Artificial Intelligence, Statistical Modeling and Machine Learning to their individual projects and tasks. It will also reduce the learning curve and increase the chances of success if Vonage looks to enter new areas such as IoT, network analysis, and robotic sensing. More important still, this document also helps to frame the current competitive landscape in machine learning and home automation highlighting what the Company needs to do to stay ahead of the curve.

Testing Computer Vision Algorithms

With a comprehensive list of machine learning algorithms in hand, the team decided to test if the algorithms could be applied in real-world scenarios where IoT products and services could help consumers. Utilizing publicly-available test data as a baseline to observe the activity of two households over the course of 30+ days, the team was able to see if their algorithms could automatically detect patterns, which could help with automating heating and cooling, or any deviations from those patterns in either of the houses. These algorithms are used for things such as warming or cooling a home before people arrive, turning lights on and off to save energy, recommending goods to purchase according to projected needs, and more. The team then presented their results to a greater audience of Vonage executives as an example of how to successfully implement machine learning to provide input for automation or other processes of value to consumers and businesses. The exercise was a good first step toward understanding the challenges, finding the best methods and getting hands-on experience in this important area.


“This project continues Vonage’s tradition as a source of innovation, collaboration and support for research within academia. Prof Liu and his students, together with Deepak Ottur and other engineers at Vonage, combined fresh ideas and cutting edge methods, with industry-leading experience, real-world data and problem sets. We are sure that the lessons learned will benefit both parties in terms of enhanced skills in solving day-to-day problems, along with the wider community as the results are published in academic papers.”

Baruch Sterman
Vice President of Technology Research, Vonage

“Through this exploratory project, Vonage demonstrated its interest in and commitment to exploring new services and technology in the Internet-of-Things, and the smart home in particular. We found there is rich information embedded in smart-home user datasets.”

Yong Liu
Associate Professor, Electrical & Computer Engineering, NYU Tandon School of Engineering

“This collaboration with NYC Media Lab and Vonage gave me great experience with understanding current technology trends. I realized that nowadays companies like Vonage are embracing cutting edge concepts and technology to explore new business areas. Meanwhile, I gained valuable knowledge and hands-on experience within IoT through my participation in this project.”

Chenguang Yu
PhD Candidate, Electrical & Computer Engineering, NYU Tandon School of Engineering

“This project gave me hands-on experience on technologies in the smart home and allows me to know what the industry really needs in R&D for future products. The user behavior data and strong patterns collected from the smart home is very interesting. We believe we can build more powerful algorithm for predicting users actions.”

Guangyu Li
PhD Candidate, Electrical & Computer Engineering, NYU Tandon School of Engineering