Hitachi Developed Technology which Accurately Measures Human Behavior and Estimates Attributes in Real Time

Realized by analyzing 3D data obtained from a stereo camera

Estimation of attributes by analyzing 3D data obtained from a stereo camera and examples of services

Tokyo, October 27, 2015 --- Hitachi, Ltd. (TSE: 6501 "Hitachi") today announced that it has developed a technology which can accurately measure human behavior and estimate attributes(characteristics) in real time. By analyzing 3D data obtained from a stereo camera*1 , human behavior can be measured accurately, in real time, even in a crowded space, which used to be difficult with 2D image data from conventional surveillance cameras. Additionally, by analyzing 3D data of the shapes of people and the objects around them, accessory objects such as baby strollers and wheelchairs for example, can be identified. By using this data, the each person's attribute of their owners - for example, a guest with his/her child or an elderly person who needs assistance - can also be estimated. The new technology can be applied to a variety of fields such as marketing to provide services depending on the behavior and attributes of people in commercial facilities, offices, event halls, and other spaces.

^{*1Stereo camera: A camera equipped with two lenses on both sides and sensor devices to capture two images. It takes a right and a left image simultaneously, and from the visual differences between the two images, depth direction information can be obtained and space can be understood stereoscopically.}

In recent years, advances in sensor technologies and improvements in information processing capability such as big data analysis has been raising expectation for efforts to increase the value of spaces. For example, highly convenient layout can be realized by statistically analyzing vast amounts of data, such as data of human behavior, generated in spaces where people gather. Also, by combining and analyzing data of attributes, such as age group and the presence of a accompanied child, found in individual behavioral data, services tailored to diverse individual needs is expected to offer. Support services to improve QoL (Quality of Life) is especially needed to address issues facing society today, such as an aging and falling birth-rate population and reduction of the burden that physically challenged persons experience.

Conventional technologies to measure human behavior include the use of surveillance cameras, laser radars, infrared tags, and GPS-equipped information terminals. Measurement using surveillance camera image data generally determines the positions of people by detecting their silhouettes. However, silhouettes cannot be recognized by this method in a crowded space, and manual manipulation is necessary. Conventional methods thus have difficulty in measuring human behavior in real time and with great accuracy. Hitachi has now developed a technology that solves this problem. By leveraging Hitachi's image recognition technologies and stereo camera technologies accumulated over many years, the new technology measures the behavior of people and estimates their attributes with high accuracy and in real time. A stereo camera is a sensor featuring two separate lenses which recognize the external world as human eyes, and the camera can measure the distance between itself and objects and acquire its 3D data. It is thus possible to stably measure human behavior even in a crowded space where people overlap, and process the obtained data accurately and in real time. Furthermore, by performing stereoscopic analysis of people and their surrounding objects, for example, large luggage and baby strollers can be identified, and the attributes of their owners can be estimated.

Hitachi is engaged in research and development to commercialize this technology and contribute to realizing more comfortable urban spaces and providing services tailored to individual needs.

Features of the Developed Technology

Technology that Accurately Estimates Human Behavior in Real Time by Analyzing 3D Data Obtained from a Stereo Camera

Unlike conventional monocular surveillance cameras, a stereo camera features the ability to capture stereoscopic 3D data with right and left lens. In the case of measurement using image data from conventional surveillance cameras, people and objects are identified by matching their 2D silhouettes with data registered in a database in advance. In a space crowded with occluded people, manual adjustments are required to recognize silhouettes as people. With a stereo camera, the distance of an object from the camera can be determined in the same manner as human eyes do by calculating the disparities in images captured by the left and right lenses. Based on this range information, 3D data is created. By extracting data near the head region of people and converting it to image data representing a downward view, the positions of people in a crowded space can be measured stably, and highly accurate processing can be performed in real time. Also, by determining temporal changes of people's positions in the images, their positions can be precisely tracked. In evaluations using data from actual environments, the tracking rate improved by approximately 30 percent^*2 .

^{*2 Compared with conventional technologies in Hitachi's evaluation data.}

2. Technology that Estimates People's Attributes with Shape Analysis of 3D Data

Hitachi's new technology estimates people's attribute by performing stereoscopic analysis of the shapes of peoples and the objects around them, such as baby strollers and wheelchairs. The technology makes it possible to identify the mass and shapes of objects from the analysis of 3D data of people's positions at any height and their surroundings. By comparing the 3D data with the data of objects registered in advance, accessory objects such as baby strollers, shopping carts, and wheelchairs can be identified. The traits of their owners - for example, a guest with his/her child or an elderly person who needs assistance - can be also estimated. In this way, services that meet individual needs can be provided. Examples include changing contents presented on information terminals and signage in response to different estimated attributes, and providing assistance to help elderly customers in moving by dispatching staff.