Interactive electronic games based on chewing motion

• US 9,873,038 B2
• Filed: 06/13/2014
• Issued: 01/23/2018
• Est. Priority Date: 06/14/2013
• Status: Active Grant

• Current Assignee: Perfetti Van Melle Benelux BV (Perfetti Van Melle S.P.A.)
• Original Assignee: Intercontinental Great Brands LLC (Mondelez International, Inc.)
• Inventors: Hooke, William Roland, Harrison, Katherine Anne
• Primary Examiner(s): Yen, Jason
• Application Number:
  US14/898,047
• Publication Number:
  US 20160136516A1
• Time in Patent Office: 1,320 Days
• Field of Search: None
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/23 for interfacing with the game device, e.g. specific interfaces between game controller and console
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2009/2435 using a video camera
• CPC Class Codes: A63F 2009/2447 Motion detector
• CPC Class Codes: A63F 2009/2457 Display screens, e.g. monitors, video displays
• CPC Class Codes: A63F 2009/2488 Remotely playable
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/556 Player lists, e.g. online players, buddy list, black list
• CPC Class Codes: A63F 2300/695 Imported photos, e.g. of the player
• CPC Class Codes: A63F 9/24 Electric games; Games using electronic circuits not otherwise provided for video games A63F13/00; computerized gaming systems G07F17/32

Abstract

An interactive game in which one or more game characters are controlled by chewing, or other mouth-related movements of one or more players. The game may be used in conjunction with marketing of a chewable product, and the game may include a feature in which data associated with a particular product may be detected by a device associated with the game to affect play of the game. The data may be in the form of graphics such as a QR code, bar code or other code on the package interior or exterior that may be read by a camera, or may be in other forms, e.g., a code provided on a website, in an e-mail or text message, or in another way, to be entered manually by the player. The game may be played on a mobile electronic device. The game play may comprise one or more actions of the character(s) that are caused by one or more different mouth movements of the player(s). The actions of the characters caused by the mouth movements of the users may comprise actions that are different from and normally unrelated to mouth movements, e.g., walking, running, jumping, swinging, spinning, falling, flying, climbing, or other movements involving travel of the character.

Claims

1. A method of controlling an interactive game on a display screen of a mobile electronic device, the method comprising:

displaying on the display screen of the mobile electronic device the interactive game including a character controllable by a user of the mobile electronic device;

detecting, using a camera, chewing motion of the user; and

causing the character to perform a first action on the display screen in response to the chewing motion of the user;

wherein the displaying the interactive game includes displaying a window overlaying a portion of the interactive game on the display screen and displaying in the window a real time video of at least a mouth of the user captured by the camera.

2. The method of claim 1, further comprising displaying in the window a virtual grid overlaying at least the mouth of the user displayed in the window;

and modifying the virtual grid in real time in response to chewing motion of the user.

3. A method of controlling an interactive game on a display screen of a mobile electronic device, the method comprising:

displaying on the display screen of the mobile electronic device the interactive game including a character controllable by a user of the mobile electronic device;

detecting, using a camera, chewing motion of the user; and

causing the character to perform a first action on the display screen in response to the chewing motion of the user;

wherein the detecting chewing motion of the user includes detecting the chewing motion of the user by determining, via the camera, position of at least one of chin, jaw, lips, and mouth of the user relative to a position of a virtual grid overlaying at least the mouth of the user.

4. The method of claim 3, further comprising calibrating the chewing motion of the user relative to the virtual grid prior to initiating start of the interactive game.

5. A method of controlling an interactive game on a display screen of a mobile electronic device, the method comprising:

displaying on the display screen of the mobile electronic device the interactive game including a character controllable by a user of the mobile electronic device;

detecting, using a camera, chewing motion of the user; and

causing the character to perform a first action on the display screen in response to the chewing motion of the user;

wherein the detecting the chewing motion of the user includes detecting a speed of the chewing motion of the user and causing the character on the display screen to move faster in response to detecting an increase in the speed of the chewing motion and to move slower in response to detecting a decrease in the speed of the chewing motion.

6. A method of controlling an interactive game on a display screen of a mobile electronic device, the method comprising:

displaying on the display screen of the mobile electronic device the interactive game including a character controllable by a user of the mobile electronic device;

detecting, using a camera, chewing motion of the user; and

causing the character to perform a first action on the display screen in response to the chewing motion of the user;

further comprising detecting opening of a mouth of the user, and causing the virtual character to perform a second action in the interactive game different from the first action.

7. A method of controlling an interactive game on a display screen of a mobile electronic device, the method comprising:

displaying on the display screen of the mobile electronic device the interactive game including a character controllable by a user of the mobile electronic device;

detecting, using a camera, chewing motion of the user; and

causing the character to perform a first action on the display screen in response to the chewing motion of the user;

further comprising taking a photograph of at least a face of the user when a session of the video game ends, and incorporating at least a portion of the photograph into a graphic of the interactive game including a score accumulated by the user while playing the interactive game, and further comprising permitting the user to share the graphic with one or more people via at least one of an electronic mail, a text message, and a social media website.

8. A method of controlling an interactive game on a display screen of a mobile electronic device, the method comprising:

displaying on the display screen of the mobile electronic device the interactive game including a character controllable by a user of the mobile electronic device;

detecting, using a camera, chewing motion of the user; and

causing the character to perform a first action on the display screen in response to the chewing motion of the user;

further comprising scanning an image on a packaging of a consumer product using the mobile electronic device and incorporating data retrieved in response to the scanning of the image into the interactive game to affect game play of the interactive game.

9. The method of claim 8, wherein the mobile electronic device is at least one of a mobile phone, a tablet computer, and a hand-held video game console.

10. An apparatus configured to allow play of an interactive game, the apparatus comprising:

a display screen;

a camera;

a memory storing instructions relating to the interactive game; and

a processing device in operative communication with the display screen, the camera, and the memory, the processing device configured to access the instructions to effect;

displaying on the display screen the interactive game including a character controllable by a user of the electronic device;

displaying a window overlaying a portion of the interactive game on the display screen to show an image of at least a mouth of the user captured by the camera;

displaying in the window a virtual grid overlaying at least the mouth of the user and movable in real time in response to chewing motion of the mouth of the user;

detecting the chewing motion of the user by determining, via the camera, position of at least one of a chin, a jaw, a lip, and a mouth of the user relative to the virtual grid; and

causing the character to perform a first action on the display screen in response to the chewing motion by the user.

11. The apparatus of claim 10, wherein the processing device is configured to effect detecting opening of the mouth of the user and causing the character to perform a second action different from the first action on the display screen.

12. The apparatus of claim 10, wherein the processing device is configured to effect using the camera to take a photograph of at least a face of the user when a session of the interactive game ends and incorporate at least a portion of the photograph into a graphic of the interactive game including a score accumulated by the user while playing the interactive game, and wherein the processing device is configured to effect using the camera to scan an image on a packaging of a consumer product and incorporating data retrieved in response to the scanning of the image into the interactive game to affect game play of the interactive game.

13. A non-transitory computer readable storage medium for storing instructions that, in response to execution by a processor, cause the processor to perform operations for displaying and controlling an interactive game on an electronic device, the operations comprising:

displaying on the display screen of the electronic device the interactive game including a character controllable by a user of the electronic device;

detecting, using a camera, chewing motion of the user; and

causing the character to perform a first action on the display screen in response to the chewing motion of the user;

further comprising instructions that, in response to execution by the processor, cause the processor to perform further operations comprising detecting opening of a mouth of the user and causing the virtual character to perform a second action different from the first action on the display screen.

14. A non-transitory computer readable storage medium for storing instructions that, in response to execution by a processor, cause the processor to perform operations for displaying and controlling an interactive game on an electronic device, the operations comprising:

displaying on the display screen of the electronic device the interactive game including a character controllable by a user of the electronic device;

detecting, using a camera, chewing motion of the user; and

causing the character to perform a first action on the display screen in response to the chewing motion of the user;

further comprising instructions that, in response to execution by the processor, cause the processor to perform further operations comprising taking a photograph of at least a face of the user when a session of the interactive game ends and incorporating at least a portion of the photograph into a graphic of the interactive game including a score accumulated by the user while playing the interactive game, and

further comprising instructions that, in response to execution by the processor, cause the processor to perform further operations comprising scanning an image on a packaging of a consumer product and incorporating data retrieved in response to the scanning of the image into the interactive game to affect game play of the interactive game.

15. A method of interacting with customers of a chewable product through a game, comprising:

marketing a plurality of different chewable products to customers; and

providing a game that is playable on a hand-held device that includes a camera and a display screen;

displaying on the display screen an image of a game character that is capable of travel relative to other images on the display screen;

processing input from the camera to detect a chewing motion of a mouth of a player;

in response to detection of the chewing motion, affecting travel of the game character displayed on the screen such that game play is controlled at least partially by detection of the chewing motion.

16. The method of claim 15 further comprising detecting packaging of the plurality of different chewable products with the camera, and affecting the game play in response to the detection of such packaging in different ways depending on which of the plurality of different chewable products is detected.

17. The method of claim 16 wherein the affecting movement of the character comprises accelerating movement of the game character relative to images in a 2D vertically scrolling world in response to the player'"'"'s chewing.

18. The method of claim 15, wherein the processing input from the camera comprises detecting a nose and chin in a first frame using software running on a CPU;

detecting a region darker than adjacent regions between the mouth and chin in subsequent frames using software running on a graphics processing unit; and

comparing changes in and adjacent to the region from frame to frame by analysis of individual pixels using a graphics processing unit.