Entertainment system and method

• US 8,216,036 B2
• Filed: 05/27/2008
• Issued: 07/10/2012
• Est. Priority Date: 05/31/2007
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Europe Limited (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.)
• Inventors: Eyzaguirre, Adriana Maria, Hodges, Peter John
• Primary Examiner(s): Vo, Peter DungBa
• Assistant Examiner(s): WONG, JEFFREY KEITH
• Application Number:
  US12/595,075
• Publication Number:
  US 20100203933A1
• Time in Patent Office: 1,505 Days
• Field of Search: 463/2, 463/31, 463/34, 463/40, 463/46
• US Class Current: 463/2
• CPC Class Codes: A63H 13/10 shooting arrows or other missiles
• CPC Class Codes: A63H 17/26 Details; Accessories drive mechanisms A63H29/00
• CPC Class Codes: A63H 30/04 using wireless transmission

Abstract

An entertainment system comprises a remotely controlled toy having a video camera operable to capture video images of a real environment and an entertainment device operable to communicate using a data communications link with the remotely controlled toy, in which the device comprises transmitting means operable to transmit, via the data communications link, control data to the remotely controlled toy that relates to the control of the remotely controlled toy, receiving means operable to receive, from the data communications link, video images captured by the video camera—of the remotely controlled toy, detecting means operable to detect a real environment feature within the real environment, processing means operable to generate a virtual image feature in dependence upon the detected real environment feature, and displaying means operable to generate a combined display of the captured video images and the virtual image feature such that the virtual image feature is arranged with respect to the video images so as to correspond to the position of the real environment feature within the real environment.

Claims

1. An entertainment system comprising a remotely controlled toy and an entertainment device operable to communicate using a data communications link with the remotely controlled toy:

the remotely controlled toy having a video camera operable to capture video images of a real environment, the remotely controlled toy comprising;

a controller operable to control behaviour of the remotely controlled toy; and

a control data receiver operable to receive, via the data communications link, the control data generated by the entertainment device,

in which the controller is operable to modify the behaviour of the remotely controlled toy in dependence upon the control data received from the entertainment device; and

the entertainment device comprising;

a transmitter operable to transmit, via the data communications link, control data to the remotely controlled toy that relates to the control of the remotely controlled toy;

a receiver operable to receive, from the data communications link, video images captured by the video camera of the remotely controlled toy;

a detector operable to detect a real environment feature within the real environment;

a processor operable to generate a virtual image feature comprising a computer generated object in dependence upon the detected real environment feature; and

a display arrangement operable to generate a combined display of the captured video images and the virtual image feature such that the virtual image feature is arranged and displayed together with the received video images so that the display position of the virtual image feature corresponds to the position of the real environment feature within the received video images.

2. A system according to claim 1, the real environment feature being an image feature that relates to a predetermined feature within the real environment and the image feature being detected within the received video images.

3. A system according to claim 1, in which:

the real environment feature is a radio frequency communication device; and

the remotely controlled toy comprises a transponder communicator operable to detect radio frequency signals transmitted by the radio frequency communication device.

4. A system according to claim 3, in which:

the transponder communicator is operable to generate radio frequency signal strength data in dependence upon the detected signal strength of the radio frequency signal transmitted by the radio frequency communication device;

the remotely controlled toy is operable to transit the signal strength data to the entertainment device via the communication link; and

the processor is operable to detect the position of the real environment feature within the real environment in dependence upon the signal strength data generated by the transponder communicator.

5. A system according to claim 3, in which:

the radio frequency communication device comprises a radio frequency transponder; and

the transponder communicator is operable to transmit radiofrequency signals to the radio frequency transponder and to detect radio frequency signals transmitted by the transponder in response to an interrogation by the transponder communicator.

6. A system according to claim 5, in which the radio frequency transponder comprises a radio frequency identification tag.

7. A system according to claim 1, in which:

the processor is operable to alter the control data in dependence upon an attribute associated with the virtual image feature.

8. A system according to claim 7, in which:

the processor is operable to transmit attribute-defining data in dependence upon an attribute associated with the virtual image feature; and

the remotely controlled toy comprises storage operable to store program code that causes the controller to modify the behaviour of the remotely controlled toy in dependence upon the attribute-defining data received from the entertainment device.

9. A system according to claim 8, in which:

the processor is operable to generate update program code that relates to the control of the remotely controlled toy;

the transmitter is operable to transmit the update program code to the remotely controlled toy via the data communication link; and

the controller is operable to update the program code stored in the storage in dependence upon the update program code data received from the data communication link sent by the entertainment device.

10. A system according to claim 7, in which:

the processor is operable to detect whether the distance between the remotely controlled toy and the predetermined feature is less than a predetermined threshold distance; and

the processor is operable to alter the control data in respect of that predetermined feature if the detected distance is less than the predetermined threshold distance.

11. A system according to claim 1, in which:

the control data represents user input control data that relates to input by a user; and

the entertainment device comprises user a input device operable to generate the user input control data in dependence upon the input by the user.

12. A system according to claim 1, in which the virtual image feature comprises a game feature of a game associated with the entertainment device.

13. A system according to claim 12, in which the processor is operable to generate the control data in dependence upon game processes generated by the game associated with the entertainment device.

14. A system according to claim 1, in which:

the transmitter is operable to transmit audio data to the remotely controlled toy using the data communications link, in which the audio data relates to audio content associated with the entertainment device; and

the remotely controlled toy comprises an audio reproducer operable to reproduce the audio content in dependence upon the audio data received via the data communications link from the entertainment device.

15. A system according to claim 14, in which the audio content associated with the entertainment device comprises data according to Motion Pictures Expert Group 1 level 3 (MP3) standard.

16. A system according to claim 1, in which the data communication link comprises a wireless communication link.

17. A system according to claim 1, in which:

the entertainment device comprises a motion sensor operable to detect motion of the entertainment device; and

the processor is operable to generate the control data in dependence upon the detected motion of the entertainment device.

18. A system according to claim 1, in which, in response to a user input, the processor is operable to generate control data that, when transmitted to the toy via the communication link, causes the toy to carry out a sequence of predetermined actions.

19. A system according to claim 18, in which:

the toy comprises an instruction storage; and

data relating to the sequence of predetermined actions is stored in the instruction storage.

20. An entertainment system comprising:

a remotely controlled toy having a video camera operable to capture video images of a real environment, the position of the toy being associated with a real path within the real environment; and

an entertainment device operable to communicate using a data communications link with the remotely controlled toy, the device comprising;

a transmitter operable to transmit, via the data communications link, control data to the remotely controlled toy that relates to the control of the remotely controlled toy;

a receiver operable to receive, from the data communications link, video images captured by the video camera of the remotely controlled toy;

a detector operable to detect the real path of the toy within the real environment and generate real path data in dependence upon the detected real path;

a processor operable to generate a virtual path in dependence upon the real path data generated by the detector and to generate a virtual image feature comprising a computer generated object in dependence upon the virtual path; and

a display arrangement operable to generate a combined display of the captured video images and the virtual image feature such that the virtual image feature is arranged and displayed together with the received video images so that the display position of the virtual image feature corresponds to the position of the real environment feature within the received video images.

21. A system according to claim 20, in which the detector is operable to detect the position of the toy with respect to the real environment in dependence upon optical flow data generated by the processor using the video images captured by the video camera.

22. A system according to claim 20, in which the detector is operable to detect the real path of the toy in dependence upon a comparison between a current detected position of the toy with respect to a previous detected position of the toy.

23. A system according to claim 20, in which:

the detector is operable to;

detect the position of the toy with respect to the real environment in dependence upon optical flow data generated by the processor using the video images captured by the video camera so as to generate a first virtual path; and

detect the real path of the toy in dependence upon a comparison between a current detected position of the toy with respect to a previous detected position of the toy so as to generate a second virtual path; and

the processor is operable to detect difference between data relating to the first virtual path and data relating to the second virtual path and, if that detected difference is less than a predetermined threshold, generate the virtual image feature in dependence upon any one of;

the first virtual path;

the second virtual path; and

an average of the data defining the first virtual path and the second virtual path.

24. A system according to claim 23, in which, if the detected difference is greater than the predetermined threshold, the processor is operable to not generate the virtual image feature.

25. A system according to claim 20, in which:

the detector is operable to detect the path of the toy by detecting the position of the toy with respect to the real environment at predetermined time intervals; and

the processor is operable to generate the virtual path by interpolating the real path data generated by detecting the position of the toy at predetermined time intervals.

26. An entertainment device operable to communicate using a data communications link with a remotely controlled toy having a video camera operable to capture video images of a real environment, the device comprising:

a transmitter operable to transmit, via the data communications link, control data to the remotely controlled toy that relates to the control of the remotely controlled toy;

a receiver operable to receive, from the data communications link, the video images captured by the video camera of the remotely controlled toy;

a detector operable to detect a real environment feature within the real environment;

a processor operable to generate a virtual image feature comprising a computer generated object in dependence upon the detected real environment feature; and

a display arrangement operable to generate a combined display of the captured video images and the virtual image feature such that the virtual image feature is arranged and displayed together with the received video images so that the display position of the virtual image feature corresponds to the position of the real environment feature within the received video images; and

the remotely controlled toy comprising;

a controller operable to control behaviour of the remotely controlled toy; and

a control data receiver operable to receive, via the data communications link, the control data generated by the entertainment device,

in which the controller is operable to modify the behaviour of the remotely controlled toy in dependence upon the control data received from the entertainment device.

27. A device according to claim 26, in which:

the control data comprises attribute control data that relates to an attribute associated with the virtual image feature; and

the processor is operable to generate the attribute control data in dependence upon the attribute associated with the virtual image feature.

28. A device according to claim 27, in which the attribute control data causes the remotely controlled toy to modify behaviour of the remotely controlled toy.

32. A tangible, non-transitory data carrier recorded with computer readable instructions that when executed by a computer, cause the computer to operate as an entertainment device in accordance with claim 26.

29. An entertainment device operable to communicate using a data communications link with a remotely controlled toy having a video camera operable to capture video images of a real environment, the position of the toy being associated with a real path within the real environment;

the device comprising;

a transmitter operable to transmit, via the data communications link, control data to the remotely controlled toy that relates to the control of the remotely controlled toy;

a receiver operable to receive, from the data communications link, the video images captured by the video camera of the remotely controlled toy;

a detector operable to detect the real path of the toy within the real environment and generate real path data in dependence upon the detected teal path;

a processor operable to generate a virtual path in dependence upon the real path data to generated by the detector and to generate a virtual image feature comprising a computer generated object in dependence upon the virtual path; and

a display arrangement operable to generate a combined display of the captured video images and the virtual image feature such that the virtual image feature is arranged and displayed together with the received video images so that the display position of the virtual image feature corresponds to the position of the real environment feature within the received video images.

30. A device according to claim 29, in which:

the position of the toy is associated with a real path within the real environment;

the detector is operable to detect the real path of the toy within the real environment and generate real path data in dependence upon the detected real path; and

the processor is operable to generate a virtual path in dependence upon the real path data generated by the detector and to generate the virtual image feature in dependence upon the virtual path.

31. A device according to claim 30, in which:

the detector is operable to;

detect the position of the toy with respect to the real environment in dependence upon optical flow data generated by the processor using the video images captured by the video camera so as to generate a first virtual path; and

detect the real path of the toy in dependence upon a comparison between a current detected position of the toy with respect to a previous detected position of the toy so as to generate a second virtual path; and

the processor is operable to detect difference between data relating to the first virtual path and data relating to the second virtual path and, if that detected difference is less than a predetermined threshold, generate the virtual image feature in dependence upon any one of;

the first virtual path;

the second virtual path; and

an average of the data defining the first virtual path and the second virtual path.

33. A remotely controlled toy operable to communicate using a data communications link with an entertainment device, the remotely controlled toy comprising:

a video camera operable to capture video images of a real environment;

a transmitter operable to transmit, via the data communications link, the video images captured by the video camera to the entertainment device;

a receiver operable to receive, from the data communications link, control data generated by the entertainment device that relates to the control of the remotely controlled toy; and

a controller operable to control behaviour of the remotely controlled toy,

in which;

the control data comprises attribute control data that relates to an attribute associated with a virtual image feature comprising a computer generated object generated by the entertainment device, the virtual image feature being generated by the entertainment device in dependence upon a detection by the entertainment device of a real environment feature within the real environment; and

the controller is operable to modify the behaviour of the remotely controlled toy in dependence upon the attribute control data received from the entertainment device.

34. A toy according to claim 33, comprising one or more selected from the list consisting of:

a horn;

an accelerometer operable to detect an impact with the toy;

a bumper or buffer operably coupled to a switch to detect an impact between the toy and another object;

an audio reproducer;

a microphone;

a light source;

a projectile firing device operable to fire projectiles;

a battery charge detector operable to detect the amount of charge in a battery used to power the remotely controlled toy; and

a distance measurer operable to detect the distance between the toy and another object.

35. A toy according to claim 34, having a distance measurer, in which the controller is operable to:

detect, in dependence upon a signal generated by the distance measurer, whether the relative distance between the remotely controlled toy and the detected image feature is less than a predetermined threshold distance; and

modify the behaviour of the remotely controlled toy if the detected relative distance is less than the predetermined threshold distance.

36. A toy according to claim 33, comprising storage operable to store program code that causes the controller to modify the behaviour of the remotely controlled toy in dependence upon the attribute control data received from the entertainment device.

37. A toy according to claim 36, in which the controller is operable to update the program code stored in the storage in dependence upon update program code data generated by the entertainment device and received via the data communication link from the entertainment device.

38. A remotely controlled toy operable to communicate using a data communications link with an entertainment device, the remotely controlled toy comprising:

a video camera operable to capture video images of a real environment;

a transmitter operable to transmit, via the data communications link, the video images captured by the video camera to the entertainment device;

a receiver operable to receive, from the data communications link, control data generated by the entertainment device that relates to the control of the remotely controlled toy; and

a controller operable to control behaviour of the remotely controlled toy,

in which;

the control data comprises attribute control data that relates to an attribute associated with a virtual image feature comprising a computer generated object generated by the entertainment device, the virtual image feature being generated by the entertainment device in dependence upon a detection by the entertainment device of a position of the remotely controlled toy with respect to the real environment; and

the controller is operable to modify the behaviour of the remotely controlled toy in dependence upon the attribute control data received from the entertainment device.

39. An entertainment method for controlling a remotely controlled toy having a video camera operable to capture video images of a real environment and an entertainment device operable to communicate using a data communications link with the remotely controlled toy, the method comprising:

transmitting, via the data communications link, control data to the remotely controlled toy that relates to the control of and modifies the behaviour of the remotely controlled toy;

receiving, from the data communications link, video images captured by the video camera of the remotely controlled toy;

detecting a real environment feature within the real environment;

generating a virtual image feature comprising a computer generated object in dependence upon the detected real environment feature; and

generating a combined display of the captured video images and the virtual image feature such that the virtual image feature is arranged with respect to the video images so as to correspond to the position of the real environment feature within the real environment.

41. A tangible, non-transitory data carrier recorded with computer readable instructions that when executed by a computer, cause the computer to carry out the method of controlling a remotely controlled toy in accordance with claim 39 or 40.

40. An entertainment method for controlling a remotely controlled toy having a video camera operable to capture video images of a real environment and an entertainment device operable to communicate using a data communications link with the remotely controlled toy, the position of the toy being associated with a real path within the real environment;

the method comprising;

transmitting, via the data communications link, control data to the remotely controlled toy that relates to the control of the remotely controlled toy;

receiving, from the data communications link, video images captured by the video camera of the remotely controlled toy;

detecting the real path of the toy within the real environment and generating real path data in dependence upon the detected real path;

generating a virtual path in dependence upon the real path data and generating a virtual image feature comprising a computer generated object in dependence upon the virtual path; and

generating a combined display of the captured video images and the virtual image feature such that the virtual image feature is arranged with respect to the video images so as to present the virtual image feature at a predetermined position within the real environment.

42. A tangible, non-transitory data carrier recorded with computer readable instructions that when executed by a computer, cause the computer to carry out the method of controlling a remotely controlled toy in accordance with claim 40.