APPARATUS AND METHOD FOR SCENT VISUALIZATION

• US 20200132999A1
• Filed: 10/14/2019
• Published: 04/30/2020
• Est. Priority Date: 10/24/2018
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: CHOI, Jong Woo, PARK, Jun Seok, BANG, Joon Hak, LEE, Hae Ryong, CHANG, Sung June
• Granted Patent: US 11,221,484 B2
• Time in Patent Office: Days
• CPC Class Codes: A61L 2209/111 Sensor means, e.g. motion, brightness, scent, contaminant sensors
• CPC Class Codes: A61L 9/125 emanating multiple odours
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A scent visualization system may comprise a display apparatus for generating a target image including a target object; an olfactory sensor for detecting a scent; and a scent visualization apparatus for generating target associative visualization information that reminds the scent from the target image received from the display apparatus and sensing information received from the olfactory sensor, and generating an associative image by combining the target associative visualization information and the target image. Therefore, a low-cost, high-efficiency, high-utilization, and high-convenience scent visualization system can be provided.

Claims

1. A scent visualization method of a scent visualization apparatus interworking with a display apparatus and an olfactory sensor, the scent visualization method comprising:

receiving a target image including a target object from the display apparatus;

receiving sensing information from the olfactory sensor;

outputting first associative visualization information on the target image using a first learning model;

outputting second associative visualization information on the sensing information using a second learning model;

comparing the first associative visualization information and the second associative visualization information to select target associative visualization information having a higher similarity; and

combining the selected target associative visualization information and the target image to generate an associative image.

2. The scent visualization method according to claim 1, wherein the first learning model is generated through machine learning by extracting visualization information representing first basic data which is olfactory sensibility information of the target object with visual attribute information, and extracting the first associative visualization information having a high similarity among the visualization information.

3. The scent visualization method according to claim 2, wherein the second learning model is generated through machine learning by extracting visualization information representing second basic data which is olfactory sensitivity information of the sensing information with visual attribute information, and extracting the second associative visualization information having a high similarity among the visualization information.

4. The scent visualization method according to claim 3, wherein the visualization information is represented by at least one of an attribute representing a particle effect, an icon, an illustration, a photograph, and a constellation.

5. The scent visualization method according to claim 3, wherein the first associative visualization information and the second associative visualization information are extracted using a softmax algorithm.

6. The scent visualization method according to claim 1, wherein the display apparatus is a head mounted display (HMD) apparatus.

7. The scent visualization method according to claim 1, further comprising generating a least one scent corresponding to the sensing information by interworking with a scent diffusion apparatus.

8. A scent visualization apparatus comprising a processor and a memory storing at least one instruction executable by the processor, wherein when executed by the processor, the at least one instruction is configured to:

interwork with a display apparatus and an olfactory sensor;

receive a target image including a target object from the display apparatus;

receive sensing information from the olfactory sensor;

output first associative visualization information on the target image using a first learning model;

output second associative visualization information on the sensing information using a second learning model;

compare the first associative visualization information and the second associative visualization information to select target associative visualization information having a higher similarity; and

combine the selected target associative visualization information and the target image to generate an associative image.

9. The scent visualization apparatus according to claim 8, further comprising a communication unit providing a communication network for interworking with the display apparatus and the olfactory sensor.

10. The scent visualization apparatus according to claim 8, further comprising a storage unit for storing data, wherein the storage unit comprises:

a first storage unit storing first basis data which is olfactory sensibility information of the target image;

a second storage unit storing second basic data which is olfactory sensibility information of the sensing information; and

a third storage unit storing the first learning model and the second learning model.

11. The scent visualization apparatus according to claim 8, wherein the at least one instruction is further configured to:

learn the first basic data;

learn the second basic data;

generate the first learning model; and

generate the second learning model.

12. The scent visualization apparatus according to claim 8, wherein the first learning model is generated through machine learning by extracting visualization information representing the first basic data with visual attribute information, and extracting the first associative visualization information having a high similarity among the visualization information.

13. The scent visualization apparatus according to claim 12, wherein the second learning model is generated through machine learning by extracting visualization information representing the second basic data with visual attribute information, and extracting the second associative visualization information having a high similarity among the visualization information.

14. The scent visualization apparatus according to claim 13, wherein the visualization information is represented by at least one of an attribute representing a particle effect, an icon, an illustration, a photograph, and a constellation.

15. The scent visualization apparatus according to claim 13, wherein the first associative visualization information and the second associative visualization information are extracted using a softmax algorithm.

16. The scent visualization apparatus according to claim 8, wherein the display apparatus is a head mounted display (HMD) apparatus.

17. The scent visualization apparatus according to claim 8, wherein the at least one instruction is further configured to generate a least one scent corresponding to the sensing information by interworking with a scent diffusion apparatus.

18. A scent visualization system comprising:

a display apparatus for generating a target image including a target object;

an olfactory sensor for detecting a scent; and

a scent visualization apparatus for generating target associative visualization information that reminds the scent from the target image received from the display apparatus and sensing information received from the olfactory sensor, and generating an associative image by combining the target associative visualization information and the target image.

19. The scent visualization system according to claim 18, further comprising a scent diffusion apparatus generating at least one scent corresponding to the sensing information.

20. The scent visualization system according to claim 18, wherein first associative visualization information on the target image is output using a first learning model, second associative visualization information on the sensing information is output using a second learning model, and associative visualization information having a higher similarity among the first associative visualization information and the second associative visualization information is selected as the target associative visualization information.