Four percent of Manhattanites go to bed before 7:30 p.m. on weeknights. Only 6% turn off the lights after midnight.
For more fine-grained data on what makes New York City tick, ask researcher Steven Koonin. Hidden on a Brooklyn rooftop, his wide-angle infrared camera peers at windows of thousands of buildings across the East River. The camera detects 800 gradations of light, a sensitivity that lets his software determine what time households turn in, what kind of light bulbs they use, and even what pollutants their buildings emit.
He has also mounted sound sensors in Brooklyn on streetlight poles and building facades to gauge the volume of house parties and car horns.
Mr. Koonin, a former undersecretary of science in the Obama administration who directs New York University’s Center for Urban Science and Progress, is at the forefront of an academic movement to quantify urban life.
Tech companies have used the technologies and techniques collectively known as big data to make business decisions and shape their customers’ experience. Now researchers are bringing big data into the public sphere, aiming to improve quality of life, save money, and understand cities in ways that weren’t possible only a few years ago.
“It’s like when Galileo first turned the telescope on the heavens,” said Mr. Koonin. “It’s just a whole new way of looking at society.”
In doing so, researchers are raising questions about the proper balance between privacy and efficiency. Municipal sensor networks offer big opportunities, but they also carry risks. In turning personal habits into digital contrails, the technology may tempt authorities to misuse it. While academics aim to promote privacy and transparency, some worry that the benefits of big data could be lost if the public grows wary of being monitored.
“For a long time, people living in cities pretended that they could be anonymous,” said Anthony Townsend, author of the book “Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia.” “It’s not about anonymity but about surveillance. There is a real reckoning that’s going to happen.”
The NYU center, which receives millions of dollars in financing from several corporations, including Microsoft Corp. and International Business Machines Corp. , as well as New York City, is one of a handful of new academic institutions undertaking municipal big data projects.
In the coming weeks, the University of Chicago will install dozens of sensor packs on street lamps in the city’s central business district and elsewhere. Each pack, roughly the size of a thick laptop, contains 65 sensors intended to capture data on environmental conditions including sound volume, wind and carbon-dioxide levels, as well as behavioral data such as pedestrian traffic flow as revealed by Wi-Fi-enabled smartphones.
The Chicago installation is funded by a $200,000 federal grant plus donations from Qualcomm Inc., Cisco Systems Inc. and other companies.
“It’s like a Fitbit for the city,” said Charlie Catlett, director of the University of Chicago’s Urban Center for Computation and Data, the institute leading the city’s “Array of Things” project.
These projects build on recent government efforts to use data to make cities more efficient. In Houston, for example, officials track smartphones to understand road congestion and synchronize traffic lights. Sensors in Barcelona trash cans help sanitation workers optimize their collection routes.
Such projects are in their early stages, but they’re already fueling controversy. “This type of invasion is a very slippery slope,” said Bob Fioretti, a Chicago alderman and mayoral candidate, about the current project under way in the city. He said the country has “reached a point where technology has exceeded the parameters of the law.”
U.S. Attorney General Eric Holder recently pointed out the dangers of so-called “predictive policing.” Police in Los Angeles and Chicago are using historical crime data, another example of data mining, to predict where offenses will erupt, a strategy that sends officers to knock on doors and warn residents to stay within the law before they’ve committed a crime.
A handful of cities such as Chicago, Boston and Los Angeles have adopted open-data policies, which make municipal data publicly available. Nonetheless, commercial and governmental sensor networks tend to operate under few rules and little scrutiny. Earlier this month, New York City’s Department of Information Technology and Telecommunications shutdown a commercial project in which a company had installed hundreds of sensors in public phone booths. The sensors surreptitiously transmitted marketing messages.
The goal isn’t to sell products or spy on people, the academics say, but to bolster quality of life and knowledge of how cities function. In an age when citizens have grown accustomed—or at least resigned—to high-tech surveillance, they argue, their efforts are relatively benign.
Moreover, they say they’ve taken pains to protect privacy and foster transparency. Chicago, for instance, won’t store the code that identifies specific devices and will publish data it collects on the Web. The NYU researchers scramble images taken by Mr. Koonin’s camera, so they can’t see inside homes.
Knowing block-by-block pollution levels can help families choose places to live, and noise data puts teeth in long-neglected noise ordinances, Mr. Koonin said. Cities with pollution laws on the books could monitor emissions rather than depend on building owners to report them.
Beyond its potential impact on urban life, municipal data may have huge economic value. Property values may rise on blocks with low pollution levels, cities may collect greater revenue for violations of noise and emission ordinances, and retailers may use pedestrian traffic data to choose more profitable store locations, according to academics and city officials.
Whether precise knowledge of when New Yorkers go to bed can make money, save resources, or help the city run more efficiently remains an open question. But researchers hope the benefits of the technology will win over a surveillance-wary public.
“This is the inverse of Big Brother,” University of Chicago’s Mr. Catlett said. “If you think about Big Brother as the city watching the people, this is the city putting data out there so that the people can watch the city.”