Arithmetic unit, transmission program, and transmission method

• US 10,757,439 B2
• Filed: 11/30/2017
• Issued: 08/25/2020
• Est. Priority Date: 12/06/2016
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Hiramatsu, Yoshitaka, Kabuto, Nobuaki
• Primary Examiner(s): Dang, Philip P.
• Application Number:
  US15/827,145
• Publication Number:
  US 20180160140A1
• Time in Patent Office: 999 Days
• CPC Class Codes: H03M 13/33 Synchronisation based on error coding or decoding
• CPC Class Codes: H03M 13/356 Unequal error protection [UEP]
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/521 for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors
• CPC Class Codes: H04N 19/66 involving data partitioning, i.e. separation of data into packets or partitions according to importance
• CPC Class Codes: H04N 19/67 involving unequal error protection [UEP], i.e. providing protection according to the importance of the data

Abstract

An arithmetic unit comprises a video input unit to which is input recorded video information obtained by taking a video of a human, a memory which stores human motion information as information related to movement of the human, and a transmission unit which determines importance of the human motion information, and sends the recorded video information and the human motion information to an external device so that availability of the human motion information in the external device will increase more for the human motion information determined as having high importance.

Claims

1. A mobile object, comprising:

an interface which communicates via a wireless link with an external device;

a memory which stores a redundancy condition in which a plurality of change amount conditions and a plurality of redundancy amounts are held in correspondence; and

a processor,

wherein the memory further stores instructions that, when executed by the processor, cause the processor to;

encode video information obtained by taking a video of a human;

extract, from the video information, human motion information;

detect a change in a position of the human based on the human motion information;

determine a redundancy amount based on the detected change in the position of the human and the redundancy condition;

generate a data unit which includes the encoded video information, the extracted human motion information and one or more pieces of redundant human motion information in which the extracted human motion information is made redundant, a number of the one or more pieces of redundant human motion information being specified by the determined redundancy amount;

divide the data unit into a plurality of packets, the extracted human motion information and each piece of the redundant human motion information being separated in different ones of the packets; and

transmit the plurality of packets from the interface to the external device,

wherein the human motion information is extracted from a latest image,

wherein the human motion information includes face information related to a face of the human in the video, the face information including at least one of an area of the face, a facial part position, and a facial part status, and

wherein each piece of the redundant human motion information is redundant with respect to the extracted human motion information in the data unit.

2. The mobile object according to claim 1, wherein the facial part status includes at least one of a status of eyes and a status of a mouth.

3. The mobile object according to claim 1, wherein the human motion information includes joint information related to a joint of the human in the video, the joint information including at least one of an position of the joint and an orientation of the joint.

4. The mobile object according to claim 1, wherein the memory stores the redundancy condition in which a plurality of frequency band conditions of the wireless link, the plurality of change amount conditions and the plurality of redundancy amounts are held in correspondence, and

wherein the memory further stores instructions that, when executed by the processor, cause the processor to:

determine, from among the plurality of redundancy amounts, the redundancy amount which corresponds to one of the plurality of change amount conditions satisfied by the detected change and which corresponds to one of the plurality of frequency band conditions satisfied by a state of the wireless link.

5. The mobile object according to claim 1, wherein the video information is encoded from the latest image of the video of the human, and

wherein the data unit includes the encoded video information from the latest image, the extracted human motion information from the latest image and the redundant human motion information in which the extracted human motion information from the latest image is made redundant according to the determined redundancy amount.

6. The mobile object according to claim 1, wherein the memory stores the redundancy condition in which higher redundancy amounts are held in correspondence with higher change amount conditions.

7. The mobile object according to claim 1, wherein the plurality of change amount conditions include at least one of an overall average change amount, an arm change amount, a leg change amount and a maximum change amount.

8. The mobile object according to claim 1, wherein the memory further stores instructions that, when executed by the processor, cause the processor to:

transmit the plurality of packets which are divided from the data unit from the interface to the external device with a higher priority than other packets transmitted over the wireless link.

9. The mobile object according to claim 1, wherein the memory further stores instructions that, when executed by the processor, cause the processor to:

transmit the plurality of packets which are divided from the data unit from the interface to the external device with a higher priority than other packets.

10. A method performed by a mobile object which communicates via a wireless link with an external device, comprising:

encoding video information obtained by taking a video of a human;

extracting, from the video information, human motion information;

detecting a change in a position of the human based on the human motion information;

determining a redundancy amount based on the detected change in the position of the human and a redundancy condition in which a plurality of change amount conditions and a plurality of redundancy amounts are held in correspondence;

generating a data unit which includes the encoded video information, the extracted human motion information and one or more pieces of redundant human motion information in which the extracted human motion information is made redundant, a number of the one or more pieces of redundant human motion information being specified by the determined redundancy amount;

dividing the data unit into a plurality of packets, the extracted human motion information and each piece of the redundant human motion information being separated in different ones of the packets; and

transmitting the plurality of packets over the wireless link to the external device,

wherein the human motion information is extracted from a latest image,

wherein the human motion information includes face information related to a face of the human in the video, the face information including at least one of an area of the face, a facial part position, and a facial part status, and

wherein each piece of the redundant human motion information is redundant with respect to the extracted human motion information in the data unit.

11. The method according to claim 10, wherein the facial part status includes at least one of a status of eyes and a status of a mouth.

12. The method according to claim 10, wherein the human motion information includes joint information related to a joint of the human in the video, the joint information including at least one of an position of the joint and an orientation of the joint.

13. The method according to claim 10, wherein the redundancy condition holds a plurality of frequency band conditions of the wireless link, the plurality of change amount conditions and the plurality of redundancy amounts in correspondence, and

wherein the redundancy amount is determined, from among the plurality of redundancy amounts, to correspond to one of the plurality of change amount conditions satisfied by the detected change and to correspond to one of the plurality of frequency band conditions satisfied by a state of the wireless link.

14. The method according to claim 10, wherein the video information is encoded from the latest image of the video of the human, and

wherein the data unit includes the encoded video information from the latest image, the extracted human motion information from the latest image and the redundant human motion information in which the extracted human motion information from the latest image is made redundant according to the determined redundancy amount.

15. The method according to claim 10, wherein the plurality of change amount conditions include at least one of an overall average change amount, an arm change amount, a leg change amount and a maximum change amount.

16. The method according to claim 10, wherein the plurality of packets which are divided from the data unit are transmitted over the wireless interface to the external device with a higher priority than other packets.

17. The method according to claim 10, wherein the plurality of packets which are divided from the data unit are transmitted over the wireless link to the external device with a higher priority than other packets transmitted over the wireless link.

18. A system comprising:

an external device; and

a mobile object which communicates via a wireless link with the external device,

wherein the mobile object includes a first memory which stores a redundancy condition in which a plurality of change amount conditions and a plurality of redundancy amounts are held in correspondence, and a first processor,

wherein the first memory further stores instructions that, when executed by the first processor, cause the first processor to;

encode video information obtained by taking a video of a human;

extract, from the video information, human motion information;

detect a change in a position of the human based on the human motion information;

determine a redundancy amount based on the detected change in the position of the human and the redundancy condition;

generate a data unit which includes the encoded video information, the extracted human motion information and one or more pieces of redundant human motion information in which the extracted human motion information is made redundant, a number of the one or more pieces of redundant human motion information being specified by the determined redundancy amount;

divide the data unit into a plurality of packets, the extracted human motion information and each piece of the redundant human motion information being separated in different ones of the packets; and

transmit the plurality of packets over the wireless link to the external device,

wherein the external device includes a second memory and a second processor, and the second memory stores instructions that, when executed by the second processor, cause the second processor to;

receive at least a portion of the plurality of packets from the mobile object;

analyze the extracted human motion information included in the received portion of the plurality of packets and determine an action of the mobile object based thereon; and

transmit the determined action to the mobile object,

wherein the human motion information is extracted from a latest image,

wherein the human motion information includes face information related to a face of the human in the video, the face information including at least one of an area of the face, a facial part position, and a facial part status, and

wherein each piece of the redundant human motion information is redundant with respect to the extracted human motion information in the data unit.