IMAGE PROCESSING APPARATUS, IMAGE PROCESSING SYSTEM, IMAGE PROCESSING METHOD AND STORAGE MEDIUM

• US 20160127508A1
• Filed: 06/10/2014
• Published: 05/05/2016
• Est. Priority Date: 06/17/2013
• Status: Abandoned Application

• Current Assignee: Square Enix Holdings Co., Ltd.
• Original Assignee: Square Enix Holdings Co., Ltd.
• Inventors: PERRIN, Cyril
• Application Number:
  US14/898,834
• Publication Number:
  US 20160127508A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/537 using indicators, e.g. showing the condition of a game character on screen
• CPC Class Codes: A63F 13/79 involving player-related data, e.g. identities, accounts, preferences or play histories
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: H04L 65/75 Media network packet handling
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

An image processing apparatus acquires a first frame and acquiring additional image data for at least one of a first user or a second user, generates a first composite frame for the first user by combining the first frame and the additional image data for the first user and/or a second composite frame for the second user by combining the first frame and the additional image data for the second user, outputs, for the first user, the first composite frame if the first composite frame is generated or the first frame otherwise, and outputs, for the second user, the second composite frame if the second composite frame is generated or the first frame otherwise.

Claims

1-24. -24. (canceled)

25. An image processing apparatus, comprising:

an acquisition unit configured to acquire a first frame, which is rendered by a rendering unit and is included in a video data stream, and acquire additional image data for at least one of a first user or a second user;

a combining unit configured to generate a first composite frame for the first user by combining the first frame and the additional image data for the first user and/or a second composite frame for the second user by combining the first frame and the additional image data for the second user; and

an outputting unit configured to output, for the first user, the first composite frame if the first composite frame is generated or the first frame otherwise, and output, for the second user, the second composite frame if the second composite frame is generated or the first frame otherwise.

26. The apparatus according to claim 25, wherein the acquisition unit is configured to acquire the additional image data from a storage unit.

27. The apparatus according to claim 25, wherein the additional image data is a second frame containing one or more pre-positioned graphical elements.

28. The apparatus according to claim 25, further comprising an accepting unit configured to accept customization information for each user,

wherein the acquisition unit is configured to acquire the additional image data according to the accepted customization information.

29. The apparatus according to claim 28, the customization information for a user comprises at least one of an identifier of the additional image data for the user or information regarding an insertion time of the additional image data.

30. The apparatus according to claim 29, wherein the video data stream relates to a video game, and wherein the identifier of the additional image data is determined based on a situation of the video game.

31. The apparatus according to claim 29, wherein the information regarding the insertion time designates a time at which the additional image data is combined with the first frame or at which the additional image data is acquired by the acquisition unit.

32. The apparatus according to claim 29, wherein the video data stream relates to a video game, and wherein the insertion time is determined based on a situation of the video game.

33. The apparatus according to claim 25, wherein the acquisition unit is configured to acquire one or more graphical elements as the additional image data, and wherein the combining unit is configured to generate a composite frame by placing each of the one or more graphical elements within the first frame.

34. The apparatus according to claim 33, wherein the combining unit is configured to control a position and/or a size at which each of the one or more graphical elements is placed within the first frame.

35. The apparatus according to claim 34, wherein the combining unit is configured to receive a parameter set designating a desired position and/or size at which at least one of the one or more graphical elements is placed within the first frame and to control the position and/or the size at which each of the one or more graphical elements is placed within the first frame in accordance with the desired position and/or size.

36. The apparatus according to claim 34, wherein the position and/or the size at which each of the one or more graphical elements is placed within the first frame changes over time.

37. The apparatus according to claim 35, wherein the parameter set further designates a characteristic quality of at least one of the one or more graphical elements, which indicates how the at least one of the one or more graphical elements is presented in the composite frame.

38. The apparatus according to claim 37, wherein the characteristic quality includes at least one of font, layout, shape and color.

39. The apparatus according to claim 33, further comprising an accepting unit configured to accept customization information for each user,

wherein the acquisition unit is configured to acquire the one or more graphical elements according to the accepted customization information.

40. The apparatus according to claim 35, wherein the combining unit is configured to receive the parameter set as a part of a rendering command set, based on which the rendering unit renders the first frame.

41. The apparatus according to claim 35, further comprising parameter generation unit configured to generate the parameter set by analyzing the first frame.

42. An image processing system, comprising an image processing apparatus and a rendering unit configured to render a first frame, wherein the image processing apparatus comprises:

an acquisition unit configured to acquire a first frame, which is rendered by the rendering unit and is included in a video data stream, and acquire additional image data for at least one of a first user or a second user;

a combining unit configured to generate a first composite frame for the first user by combining the first frame and the additional image data for the first user and/or a second composite frame for the second user by combining the first frame and the additional image data for the second user; and

an outputting unit configured to output, for the first user, the first composite frame if the first composite frame is generated or the first frame otherwise, and output, for the second user, the second composite frame if the second composite frame is generated or the first frame otherwise.

43. A method of processing an image, comprising:

acquiring a first frame, which is rendered by a rendering unit and is included in a video data stream;

acquiring additional image data for at least one of a first user or a second user;

generating a first composite frame for the first user by combining the first frame and the additional image data for the first user and/or a second composite frame for the second user by combining the first frame and the additional image data for the second user;

outputting, for the first user, the first composite frame if the first composite frame is generated or the first frame otherwise; and

outputting, for the second user, the second composite frame if the second composite frame is generated or the first frame otherwise.

44. A non-transitory computer-readable storage medium comprising computer-readable instructions which, when executed by a computing entity, cause the computing entity to implement a method that comprises:

acquiring a first frame, which is rendered by a rendering unit and is included in a video data stream;

acquiring additional image data for at least one of a first user or a second user;

generating a first composite frame for the first user by combining the first frame and the additional image data for the first user and/or a second composite frame for the second user by combining the first frame and the additional image data for the second user;

outputting, for the first user, the first composite frame if the first composite frame is generated or the first frame otherwise; and

outputting, for the second user, the second composite frame if the second composite frame is generated or the first frame otherwise.

45. An image processing apparatus, comprising:

a rendering unit configured to render a set of initial frames representing a sequence of images; and

a customization unit configured to receive customization information for each of a plurality of users and modify the set of initial frames to produce a plurality of sets of output frames, wherein each set of output frames representing a sequence of images that have been customized for a respective one of the users based on the customization information for that user.

46. An image processing systems, comprising an image processing apparatus and a rendering command generation unit configured to generate rendering commands, wherein the image processing apparatus comprises:

a rendering unit configured to render a set of initial frames representing a sequence of images; and

a customization unit configured to receive customization information for each of a plurality of users and modify the set of initial frames to produce a plurality of sets of output frames, wherein each set of output frames representing a sequence of images that have been customized for a respective one of the users based on the customization information for that user, and

wherein the rendering unit is configured to render the set of initial frames in accordance with the rendering commands.

47. A method of processing an image, comprising:

rendering a set of initial frames representing a sequence of images;

receiving customization information for each of a plurality of users; and

modifying the set of initial frames to produce a plurality of sets of output frames, wherein each set of output frames representing a sequence of images that have been customized for a respective one of the users based on the customization information for that user.

48. A non-transitory computer-readable storage medium comprising computer-readable instructions which, when executed by a computing entity, cause the computing entity to implement a method that comprises:

rendering a set of initial frames representing a sequence of images;

receiving customization information for each of a plurality of users; and

modifying the set of initial frames to produce a plurality of sets of output frames, wherein each set of output frames representing a sequence of images that have been customized for a respective one of the users based on the customization information for that user.