SYSTEMS, METHODS, AND COMPUTER-READABLE MEDIA FOR MANAGING COLLABORATION ON A VIRTUAL WORK OF ART

• US 20130027404A1
• Filed: 07/29/2011
• Published: 01/31/2013
• Est. Priority Date: 07/29/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Sarnoff, Matthew Jacob
• Granted Patent: US 9,075,561 B2
• Time in Patent Office: Days
• US Class Current: 345/441
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 10/101 Collaborative creation, e.g. joint development of products or services
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 11/203 Drawing of straight lines or curves
• CPC Class Codes: G09G 2340/045 Zooming at least part of an image, i.e. enlarging it or shrinking it
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2360/02 Graphics controller able to handle multiple formats, e.g. input or output formats
• CPC Class Codes: G09G 2360/18 Use of a frame buffer in a display terminal, inclusive of the display panel
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 2370/042 for monitor identification
• CPC Class Codes: G09G 2370/06 Consumer Electronics Control, i.e. control of another device by a display or vice versa
• CPC Class Codes: G09G 2370/10 Use of a protocol of communication by packets in interfaces along the display data pipeline
• CPC Class Codes: G09G 5/006 Details of the interface to the display terminal specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

Systems, methods, and computer-readable media for managing collaboration on a virtual work of art between multiple electronic devices are provided. A first graphical display system of a first device may generate an input command in response to receiving user information through a user interface of the first device, and may then share this input command with a second graphical display system of a second device. The first graphical display system may process the shared input command to generate pixel array data in a canvas of the first device while the second graphical display system may process the shared input command to generate pixel array data in a canvas of the second device. By sharing input commands rather than pixel array data, system latency may be reduced. Despite operating on the same artwork, the user interfaces and graphical processing capabilities of each device may vary, thereby providing the user greater expressiveness.

Claims

1. A method for sharing graphical data comprising:

receiving first user instructions with a first user interface of a first electronic device;

generating a first input command based on the received first user instructions with a first graphics application on the first electronic device;

transmitting the first input command from the first electronic device to a second electronic device; and

processing the first input command with the first graphics application on the first electronic device to generate first pixel array data in a first canvas of the first electronic device.

2. The method of claim 1, wherein at least a portion of the transmitting occurs at the same time as at least a portion of the processing the first input command with the first graphics application.

3. The method of claim 1 further comprising:

processing the first input command with a second graphics application on the second electronic device to generate second pixel array data in a second canvas of the second electronic device.

4. The method of claim 3, wherein at least a portion of the processing the first input command with the first graphics application occurs at the same time as at least a portion of the processing the first input command with the second graphics application.

5. The method of claim 3, wherein the first graphics application and the second graphics application share a semantic command set.

6. The method of claim 3, wherein the first pixel array data in the first canvas is the same as the second pixel array data in the second canvas.

7. The method of claim 3 further comprising:

presenting at least a portion of the first canvas on a first display of the first device; and

presenting at least a portion of the second canvas on a second display of the second device.

8. The method of claim 7, wherein the first display is larger than the second display.

9. The method of claim 7, wherein:

the first pixel array data in the first canvas is the same as the second pixel array data in the second canvas; and

the pixel array data in the at least a portion of the first canvas presented on the first display is different than the pixel array data in the at least a portion of the second canvas presented on the second display.

10. The method of claim 3 further comprising:

receiving second user instructions with a second user interface of the second device;

generating a second input command based on the received second user instructions with the second graphics application on the second device;

transmitting the second input command from the second device to the first device; and

processing the second input command with the second graphics application on the second device to generate third pixel array data in the second canvas.

11. The method of claim 10, wherein at least a portion of the transmitting the first input command occurs at the same time as at least a portion of the transmitting the second input command.

12. The method of claim 10, wherein at least a portion of the processing the first input command on the first device occurs at the same time as at least a portion of the processing the second input command on the second device.

13. The method of claim 10, wherein at least a portion of the processing the first input command on the second device occurs at the same time as at least a portion of the processing the second input command on the second device.

14. The method of claim 10, wherein:

the first user interface comprises touch input capabilities; and

the second user interface does not comprise touch input capabilities.

15. The method of claim 10 further comprising:

processing the second input command with the first graphics application on the first device to generate fourth pixel array data in the first canvas.

16. The method of claim 1, wherein the first input command does not comprise pixel array data.

17. The method of claim 1, wherein the size of the first pixel array data is larger than the size of the first input command.

18. The method of claim 1 further comprising:

processing the first input command with a second graphics application on the second electronic device;

generating a second input command with the second graphics application on the second electronic device based on the processing the first input command with the second graphics application; and

transmitting the second input command from the second electronic device to the first electronic device, wherein the second input command is configured to instruct the first electronic device to transmit at least a portion of the first pixel array data to the second electronic device.

19. The method of claim 18, wherein the processing the first input command with the second graphics application comprises determining that the second electronic device is unable to generate the at least a portion of the first pixel array data.

20. The method of claim 18, wherein the processing the first input command with the second graphics application comprises evaluating the first input command with respect to at least one of the processing capabilities of the second electronic device, the power supply of the second electronic device, and the processing capabilities of the first electronic device.

21. A method for sharing graphical data comprising:

loading a first graphics application on a first electronic device;

loading an artwork into the first graphics application on the first electronic device; and

sending first information from the first electronic device to a second electronic device, wherein the first information is configured to instruct the second electronic device to load at least a first portion of the artwork into a second graphics application on the second electronic device.

22. The method of claim 21 further comprising, prior to the sending, receiving a first instruction at the first electronic device that defines the at least a first portion of the artwork.

23. The method of claim 22, wherein the receiving the first instruction comprises receiving the first instruction at the first electronic device from a user of the first electronic device.

24. The method of claim 22, wherein:

the loading the artwork into the first graphics application on the first electronic device comprises presenting at least a second portion of the artwork on a display of the first electronic device; and

the receiving the first instruction comprises receiving from a user of the first electronic device a selection of the at least a first portion of the artwork from the at least a second portion of the artwork on the display.

25. The method of claim 24, wherein the receiving the selection comprises receiving from the user a definition of an outline on the display.

26. The method of claim 22, wherein the receiving the first instruction comprises receiving the first instruction at the first electronic device from the second electronic device.

27. The method of claim 26, wherein the first instruction is indicative of at least one of the dimensions of a canvas displayed by the second electronic device and a resolution of a display of the second electronic device.

28. The method of claim 26, wherein the first instruction is indicative of a selection of the at least a first portion of the artwork by a user of the second electronic device.

29. The method of claim 21, wherein the loading the artwork into the first graphics application on the first electronic device comprises loading the artwork from a server into the first graphics application on the first electronic device.

30. The method of claim 29, wherein the first information is configured to instruct the second electronic device to load the at least a first portion of the artwork into the second graphics application on the second electronic device from the server.

31. The method of claim 21, wherein the first information comprises pixel array data.

32. The method of claim 21, wherein the first information comprises at least one graphical object input command.

33. The method of claim 32 further comprising processing the at least one graphical object input command with the second graphics application on the second electronic device to generate first pixel array data in a canvas of the second electronic device, wherein the first pixel array data is indicative of the at least a first portion of the artwork.

34. The method of claim 21, wherein:

the loading the artwork into the first graphics application on the first electronic device comprises presenting a second portion of the artwork on a display of the first electronic device; and

the second portion comprises the at least a first portion of the artwork and an additional portion of the artwork.

35. The method of claim 21, wherein:

the loading the artwork into the first graphics application on the first electronic device comprises presenting a second portion of the artwork on a display of the first electronic device;

the second portion comprises the entirety of the artwork; and

the at least a first portion of the artwork comprises only a portion of the entirety of the artwork.

36. The method of claim 21, wherein the first graphics application and the second graphics application share a semantic command set.

37. An electronic device comprising:

a display;

a user input component;

communications circuitry; and

a processor configured to;

receive a first user instruction from the user input component;

generate a first input command based on the received first user instruction;

provide the first input command to the communications circuitry for transmission of the first input command to an other electronic device;

process first pixel array data from the first input command; and

present at least a portion of the first pixel array data on the display.

38. The electronic device of claim 37, wherein:

the first pixel array data presents at least a portion of a drawing stroke graphical object on the display; and

the first input command comprises at least one of;

data defining at least one point of a trail of the drawing stroke graphical object; and

data defining an input tool of the drawing stroke graphical object.

39. The electronic device of claim 37, wherein:

the first pixel array data presents at least a portion of a text graphical object on the display; and

the first input command comprises at least one of;

data defining a glyph of the text graphical object;

data defining a font of the text graphical object; and

data defining a position of the text graphical object.

40. The electronic device of claim 37, wherein:

the first pixel array data presents at least a portion of a shape graphical object on the display; and

the first input command comprises at least one of;

data defining a shape of the shape graphical object; and

data defining a position of the shape graphical object.

41. The electronic device of claim 37, wherein:

the first pixel array data presents at least a portion of an image graphical object on the display; and

the first input command comprises at least one of;

data defining a shape of the image graphical object; and

data defining a link to an image file.

42. The electronic device of claim 37, wherein:

the communications circuitry is configured to receive a second input command from the other electronic device; and

the processor is further configured to;

receive the second input command from the communications circuitry;

process a third input command from the second input command; and

provide the third input command to the communications circuitry for transmission of the third input command to the other electronic device, wherein the third input command is configured to instruct the other electronic device to share pixel array data with the electronic device.

43. The electronic device of claim 37, wherein:

the communications circuitry is configured to receive a second input command from the other electronic device; and

the processor is further configured to;

receive the second input command from the communications circuitry;

process second pixel array data from the second input command; and

present at least a portion of the second pixel array data on the display.

44. The electronic device of claim 43, wherein:

the first pixel array data presents a first portion of a graphical object on the display; and

the second pixel array data presents a second portion of the graphical object on the display.

45. The electronic device of claim 44, wherein the processor is not configured to generate the second input command.

46. The electronic device of claim 44, wherein the second input command comprises data generated by a motion sensor.

47. The electronic device of claim 44, wherein:

the graphical object is a drawing stroke graphical object on the display;

the first input command comprises data defining at least one point of a trail of the drawing stroke graphical object; and

the second input command comprises data defining an effect imparted on an input tool of the drawing stroke graphical object.

48. The electronic device of claim 37, wherein:

the communications circuitry is configured to receive a second input command from the other electronic device; and

the processor is further configured to;

receive the second input command from the communications circuitry; and

process the first pixel array data from a combination of the first input command and the second input command.

49. The electronic device of claim 48, wherein the first input command and the second input command share a timestamp.

50. The electronic device of claim 48, wherein the first input command and the second input command are associated with the creation of a single graphical object.

51. Computer-readable media for controlling an electronic device, comprising computer-readable code recorded thereon for:

receiving first user instructions with a first user interface of the electronic device;

generating a first input command based on the received first user instructions;

transmitting the first input command from the electronic device to an other electronic device; and

processing the first input command on the electronic device to generate first pixel array data.

52. A data processing system, comprising:

a processor to execute instructions; and

a memory coupled with the processor to store instructions, which when executed by the processor, cause the processor to perform operations to generate an application programming interface (“

API”

) that allows an API-calling component to perform the following operations;

receive first user instructions with a first user interface of a first electronic device;

generate a first input command based on the received first user instructions;

transmit the first input command from the first electronic device to an other electronic device; and

process the first input command on the first electronic device to generate first pixel array data.

53-60. -60. (canceled)