US20150143421A1 - Method, server, client and software - Google Patents
Method, server, client and software Download PDFInfo
- Publication number
- US20150143421A1 US20150143421A1 US14/520,438 US201414520438A US2015143421A1 US 20150143421 A1 US20150143421 A1 US 20150143421A1 US 201414520438 A US201414520438 A US 201414520438A US 2015143421 A1 US2015143421 A1 US 2015143421A1
- Authority
- US
- United States
- Prior art keywords
- client device
- image
- segment
- segments
- parameters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234327—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into layers, e.g. base layer and one or more enhancement layers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/764—Media network packet handling at the destination
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/239—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
- H04N21/2393—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests involving handling client requests
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
- H04N21/25866—Management of end-user data
- H04N21/25891—Management of end-user data being end-user preferences
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
- H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/27—Server based end-user applications
- H04N21/274—Storing end-user multimedia data in response to end-user request, e.g. network recorder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/437—Interfacing the upstream path of the transmission network, e.g. for transmitting client requests to a VOD server
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440245—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/4728—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
- H04N21/64322—IP
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/65—Transmission of management data between client and server
- H04N21/658—Transmission by the client directed to the server
- H04N21/6587—Control parameters, e.g. trick play commands, viewpoint selection
Definitions
- the present-disclosure relates to a method, server, client and software
- This cut-out image can then be broadcast to many user devices over a network.
- This network may be a broadcast network or a local or wide area network.
- the user of each device would be able to only view the broadcast stream. In other words, the user has no control over the position of the cut out and so cannot control their virtual camera view.
- a client device comprising:
- a communication unit configured to receive a plurality of parameters, the parameters defining segments of an image stored on a server, wherein the segments partially overlap; a processing unit configured to allow selection of a segment of the image in accordance with a defined view on the client device and wherein upon selection the segment is downloadable from the server; and a display configured to display on the client device the cut out of the image taken from the receivable segment.
- the amount of overlap between the segments may be greater than the cut out of the image.
- the plurality of segments may be the same size.
- the plurality of segments may be high definition images.
- the resolution of the plurality of the segments may be less than the resolution of the image.
- the cut-out may be defined using the same parameters as those used to define the segments and the processing unit is further operable to select the segment by comparing the value of the parameters of the cut-out with the value of the parameters of the segment and selecting the segment having the closest value to that of the cut-out.
- the device may further comprise a storage unit configured to store the parameters of the cut out of the image, and to return the stored parameters to the server via the communication unit.
- the device may comprise charging advertising revenue on the basis of the returned parameters
- the segment may be defined by a Unique Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for download.
- the communication unit may be further configured to receive, at the client device, parameters of a second set of segments containing a different number of segments of the image, and to select the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
- a server comprising: a communication unit configured to send, to a client device, a plurality of parameters, the parameters defining segments of an image stored on the server, wherein the segments partially overlap; and a processing unit configured in use to receive, via the communication unit, a request for a selected segment of the image in accordance with a defined view on the client device and, in use, uploading the segment of the image to the client.
- the amount of overlap between the segments may be greater than the defined view on the client device.
- the plurality of segments may be the same size.
- the plurality of segments may be high definition images.
- the resolution of the plurality of the segments may be less than the resolution of the image.
- the defined view may use the same parameters as those used to define the segments and the processing unit is further configured to: select the segment by comparing the value of the parameters of the defined view with the value of the parameters of the segment and selecting the segment having the closest value to that of the defined view.
- the communication unit may be configured to receive the parameters of the desired view from the client device.
- the server may comprise charging advertising revenue on the basis of the returned parameters
- the segment may be defined by a Unique Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for upload.
- the communication unit may be configured to send, to the client device, parameters of a second set of segments containing a different number of segments of the image, and to send the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
- a method of displaying, on a client device, a cut out of an image stored on a server comprising the steps of: receiving, at the client device, a plurality of parameters, the parameters defining segments of the image stored on the server, wherein the segments partially overlap; selecting the segment of the image in accordance with a defined view on the client device and downloading the segment of the image from the server; and displaying on the client device the cut out of the image taken from the received segment.
- the amount of overlap between the segments may be greater than the cut out of the image.
- the plurality of segments may be the same size
- the plurality of segments may be high definition images.
- the resolution of the plurality of the segments may be less than the resolution of the image.
- the cut-out may be defined using the same parameters as those used to define the segments and the step of selecting the segment comprises the step of comparing the value of the parameters of the cut-out with the value of the parameters of the segment and selecting the segment having the closest value to that of the cut-out.
- the method may further comprise storing the parameters of the cut out of the image, and returning the stored parameters to the server.
- the method may further comprise charging advertising revenue on the basis of the returned parameters
- the segment may be defined by a Unique Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for download.
- the method may further comprise receiving, at the client device, parameters of a second set of segments containing a different number of segments of the image, and selecting the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
- a method of providing, to a client device, a segment of an image stored on at least one server comprising the steps of: sending, to the client device, a plurality of parameters, the parameters defining segments of the image stored on the server, wherein the segments partially overlap; and receiving a request for a selected segment of the image in accordance with a defined view on the client device and uploading the segment of the image to the client.
- the amount of overlap between the segments may be greater than the defined view on the client device.
- the plurality of segments may be the same size.
- the plurality of segments may be high definition images.
- the resolution of the plurality of the segments may be less than the resolution of the image.
- the defined view may use the same parameters as those used to define the segments and the step of selecting the segment comprises the step of comparing the value of the parameters of the defined view with the value of the parameters of the segment and selecting the segment having the closest value to that of the defined view.
- the method may further comprise receiving the parameters of the desired view from the client device.
- the method may further comprise charging advertising revenue on the basis of the returned parameters
- the segment may be defined by a Unique Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for upload.
- the method may further comprise sending, to the client device, parameters of a second set of segments containing a different number of segments of the image, and sending the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
- FIG. 1 shows a system according to embodiments of the disclosure
- FIG. 2 shows a client shown in FIG. 1 ;
- FIG. 3 shows a server shown in FIG. 1 ;
- FIG. 4 shows a simple ultra-high definition image with three landmark points
- FIG. 5A-5B shows an image tile taken from the image of FIG. 4 and an indicated virtual camera view provided to the client of FIG. 2 ;
- FIGS. 6A-6H shows a plurality of image tiles taken from the image of FIG. 4 and various indicated virtual camera view provided to the client of FIG. 2 ;
- FIGS. 7A-7F shows a plurality of image tiles taken from the image of FIG. 4 and various indicated virtual camera view provided to the client of FIG. 2 ;
- FIGS. 8A and 8B show a user zooming out of the image of FIG. 7F using the client of FIG. 2 ;
- FIG. 9 shows a tile map stored within the client of FIG. 2 ;
- FIG. 10 shows the association between the image tiles and the location thereof stored in the server
- FIG. 11 shows the association between image tiles and the location thereof stored in the client
- FIG. 12 shows a flow chart of an initial registration of the user of the client device
- FIG. 13 shows a flow chart of a the selection of an event to view on the client device.
- FIG. 14 shows a flow chart explaining the operation of the client device.
- a system 100 is shown.
- a server 300 is connected to the Internet 110 .
- a router 120 is also connected to the Internet 110 .
- the router 120 enables devices to connect to the Internet 110 .
- the router 120 is a home router located within a user's home.
- the router 120 may be any device that allows any apparatus to connect to the Internet 110 .
- the router 120 is connected to a device 200 and thus the device 200 connects to the Internet 110 .
- the device in this case, is a client device that connects to the server 300 via the Internet 110 .
- the client device 200 is a portable client device such as a tablet.
- the tablet may be a Sony® Xperia Z which runs the Android operating system.
- any tablet device, smartphone, laptop or indeed any client device is envisaged.
- a communication unit 210 is connected to an antenna 205 .
- the communication unit 210 is, in embodiments, communicates with the router 120 using WiFi, such as the IEEE802.11g/n standard.
- the antenna 205 may be a MIMO type antenna to reduce interference. Of course, any type of wired or wireless protocol is envisaged.
- the communication unit 210 is connected to a controller 220 .
- the controller 220 operates using a computer program which contains computer readable instructions.
- the computer program which controls the controller 230 is stored on a memory 225 which is also connected to the controller 220 .
- the memory 225 may be any kind of storage medium such as a solid state memory, or indeed, an optical or magnetic readable memory or the like is envisaged.
- Also connected to the controller 220 is a touch screen 230 .
- the touch screen 230 is a user interface to allow a user to control the tablet 200 .
- the tablet 200 has a “pinch to zoom” function. This will be explained later. Of course, any kind of user interface may be used.
- the controller 220 is also connected to a graphical processor unit (GPU) 215 .
- the GPU 215 is a processing unit that is specifically configured to process images.
- the GPU 215 is entirely optional in embodiments.
- a display 235 Further, connected to the GPU 215 is a display 235 .
- the size of the display may vary. For example, in the Xperia Z tablet, the display is 10.1 inches and has a resolution of 1920 ⁇ 1200 pixels.
- the server 300 according to embodiments of the disclosure is shown.
- storage 305 is provided.
- Stored within the storage 305 are video tiles. These will be explained later. Additionally, other video streams may be stored in the storage 305 . Moreover, advertisements or still images, or indeed any content may be stored in the storage 305 .
- the storage 305 may be solid state memory, or magnetic or optical readable storage or a combination thereof.
- a server controller 320 is provided.
- the server controller 320 is a processor that controls the operation of the server 300 . Further, the server controller 320 performs image processing to generate the image tiles (or segments) stored within the storage 305 .
- the server controller 320 is controlled using a computer program.
- the computer program contains computer readable instructions which when loaded onto the server controller 320 makes the server controller 320 perform certain steps as will be explained later.
- Also connected to the server controller 320 is a user database 310 .
- the user database 310 stores details about the users connected to the server 300 . This will be explained later.
- a video database 315 is connected to the server controller 320 .
- the video database 315 stores the address of the video tile and associates the address with the appropriate video tile (or any other content) stored within the storage 305 as will be explained later.
- the communication circuit 325 connects the server 300 to the Internet 110 and communicates with the client device 200 .
- FIG. 4 shows an image 400 .
- the image 400 contains a soccer pitch 410 and a surrounding area 420 .
- Within the image are three identified points 430 , 440 and 450 .
- the first identified point 430 (shown as an “X” in the Figure) is a user selectable icon.
- the second and third identified points 440 , 450 (shown as a “Y” and “Z” in the Figure) is user specific content. These identified points are selected in advance by the content provider and their locations are stored within the tile map shown in FIG. 9 .
- the tile map is a set of co-ordinates defining segments in the image.
- the image 400 is, in embodiments, an ultra-high definition image. That is, the image 400 is comprised of two images stitched together. Specifically, in embodiments, the image is comprised of two 4K images stitched together. This means that image 400 has a resolution of 8192 ⁇ 2160 pixels. Clearly, any kind of image or any size of image may be used.
- the video tiles are explained with reference to the video tile map.
- the video tile map of FIG. 9 is stored within the memory 225 of the client device 200 .
- This tile map is specific to any particular stream and is downloaded in advance of the content.
- the tile map is a set of pitch, zoom and yaw values that define segments (or tiles) of the image 400 .
- These pitch, yaw and zoom values are the values that would be applied to a virtual camera when cutting out the tile from the image 400 .
- the method for calculating the pitch, zoom and yaw are known and are disclosed in GB1306050.4 filed on 4 Apr. 2013 and also in GB1316102.1 filed 10 Sep. 2013. The contents of these documents are hereby incorporated by reference.
- the pitch, yaw and zoom values are examples of parameters which define the tiles (or segments) of the image 400 .
- the video tiles are arranged in layers. These tiles, irrespective of layer, and in embodiments, are all the same size and resolution and may be any size or resolution. However, this is not necessary for the entire breadth of the disclosure.
- the tiles could be high definition tiles. That is, each tile may have a resolution of 1920 ⁇ 1080 pixels.
- FIG. 9 shows tiles only overlapping in the horizontal direction, this is only for ease of explanation. It is also intended that tiles overlap both the horizontal and vertical directions and give the advantages explained below in all directions in the image.
- the size of the tile could depend on the display resolution of the of the client device as will become apparent. Although not necessary, it is preferable that the resolution of the tile conforms to an acceptable standard size to ensure high efficiency codecs for efficiency in storage and transmission. In other words, the size of the tiles is a size for which regular codecs exist.
- the high definition example is an acceptable standard size tile.
- layers correspond to different zoom levels within the image 400 . That is, in layer 1, one tile covers the image 400 and so the content of the tile is the entire image 400 albeit at a reduced resolution. In other words, the content of the image 400 is provided at a resolution of the tile rather than the ultra-high definition resolution.
- the image 400 is split into a plurality of overlapping tiles.
- the resolution of the tiles between the layers is the same, and because the image 400 is split into more tiles in layer 2 than in layer 1 , the content of each of the two tiles within layer 2 contains a different segment of the image 400 (except for an overlapping section which will be explained later).
- the content of tile A in layer 2 contains the left side of the image and the content of tile B in layer 2 contains the right side of the image 400 . Therefore, the content of each tile segment in layer 2 appears magnified when compared to the content of the tile in layer 1.
- the image 400 is split into six overlapping tiles all having the same resolution as the tiles in layer 1 and 2.
- the content of each of the tiles in layer 3 is therefore a more magnified version of the content in each of layer 1 or 2.
- each tile within each layer is analogous to a virtual camera capturing part or the image 400 .
- the amount of pitch, yaw and zoom which must be applied to the virtual camera in order to capture this field of view is determined using a known technique such as that described in GB1306050.4 filed on 4 Apr.
- zoom value, pitch value and yaw values are stored in the client device 200 in association with the tile.
- the tiles of the image are pre-processed and stored on the server.
- the client device 200 requests different tiles to be provided by the server 300 , the user of the client device can zoom into the image 400 .
- the client device 200 starts by displaying the content of the tile of layer 1 and if the user wishes to zoom into the image, the client device 200 requests the appropriate tile from layer 2 and then layer 3 in order to allow the user to zoom into the image and to thus control the zoom of a virtual camera around image 400 .
- the client device 200 also generates a virtual camera view of the tile provided to it by the server 300 .
- the technique for determining the client side virtual camera parameters is determined using a known technique such as that disclosed in GB1306050.4 filed on 4 Apr. 2013 and also in GB1316102.1 filed 10 Sep. 2013. Specifically, the pinch control on the client device 200 controls the zoom axis of the client side virtual camera and the swipe action controls both the pitch and yaw axes of the client side virtual camera.
- the image 400 is provided as one tile segment 500 .
- the name of the tile indicates a unique letter identifying each tile within a layer as well as the number of the layer. So, in FIG. 5A , the image 400 is contained within one tile 500 . This tile is “tile A1” (or is tile A in layer 1).
- a dotted box 510 is provided within the tile 500 . This dotted box 510 indicates the view that is being displayed on the client device 200 . As will be explained later, the server 300 provides the content of the tile to the client device 200 and the client device 200 will extract the appropriate section of the tile for display to the user. However, the dotted box 510 has been included to aid explanation.
- FIG. 5B the screen of the client device 200 is shown. As can be seen, the content of the dotted box 510 in FIG. 5A is displayed to the user of the client device 200 .
- the user of the client device 200 wishes to zoom in to the first identified point (point X) 430 of the image 400 .
- the user will perform a “pinch to zoom” action on the touch screen 230 around the first identified point 430 .
- the user will swipe the screen to pan around the image 400 to align the first identified point 430 with the centre of the view.
- the pinch zoom is applied which increases the zoom parameter of the client side virtual camera and thus zooms in on the centre of the view.
- the dotted box 605 shows the section of the image 400 that is being displayed on the client device 200 .
- the client device 200 determines that a tile from layer 2 is required.
- the user of the client device 200 wants a more detailed view of the first identified point (point X) 430 than was possible using tile A from layer 1. Therefore, the client device 200 requests a tile from layer 2 as the zoom action takes place.
- the client device 200 knowing from the tile map stored in the memory 225 the zoom, pitch and yaw values of each of the tiles. The client device 200 then compares the client side zoom, pitch and yaw of the image shown on its display to the zoom, pitch and yaw of the stored tiles. The client device 200 then retrieves a tile with the zoom, pitch and yaw closer to the client side value of zoom, pitch and yaw of the tile currently being displayed. It is important to note that the client device 200 will switch to a different tile when the view displayed on the client device 200 will not fit inside the tile view.
- the server 300 ensures that there is sufficient overlap between the tiles to allow the view of the client device 200 to fit within the overlap region. Additionally, the client device 200 will only ever have a zoom level higher than the zoom level associated with a particular tile. Therefore, when searching for a new tile, the client device 200 may only search tiles having a zoom level lower than the current zoom level associated with the client device 200 . This reduces processing requirements of the client device 200 .
- the client device 200 employs a predictive scheme such as a linear scheme or a Kalman filter in order to predict which tile needs downloading next.
- image 400 is split into two tiles, tile A2 and B2.
- tile A2 has two parts.
- a first part 610 of tile A2 covers the left side of the image 400 and a second part 615 of tile A2 covers a more central position.
- the second part of tile A2 overlaps with a second part of tile B2.
- the first part of tile B2 covers the right side of the image 400 . Therefore, the client device 200 retrieves tile A2 from the storage 305 within the server 300 .
- the dotted box 605 A is shown on the client device 200 .
- the first identified point (point X) 430 appears much larger on the display of the client device 200 .
- the two tiles are provided with zoom, pitch and yaw values. These values indicate the zoom, pitch and yaw values of a virtual camera capturing the image of the tile from the image 400 .
- the zoom, pitch and yaw values of the tiles are those values of a virtual camera capturing the image of the tile from image 400 .
- the relationship between two perspective views is defined by an homography.
- the tile view is defined by zoom, pitch and yaw with respect to the image 400 and the client device view is defined by zoom, pitch and yaw.
- the homography transform in this case is calculated from the differences in the zoom, pitch and yaw between the client device 200 view and the zoom, pitch and yaw of the specific tile.
- the user of the client device 200 swipes the touch screen, the user of the client device 200 can move around the image 400 at this zoom level. Therefore, if we assume the user of the client device 200 moves the view to the right, the view displayed to the user will change. In particular, we can assume that the user of the client device 200 has “swiped” the touch screen such that the view displayed to the user is new dotted box 605 in FIG. 6C . In other words, the user of the client device 200 is now viewing the second identified point (point Y) 440 . This view is shown in FIG. 6D .
- the client device 200 calculates the client side zoom, pitch and yaw of the new dotted box 605 . It should be noted here that the user has moved their view into the second area 615 of tile A2. In other words, the dotted box 605 which indicates the image displayed on the client device 200 is taken from the overlapping region 615 between tile A2 and tile B2.
- the client device 200 requests that tile B2 be provided to the client device 200 from the server 300 . It is important to note here that in order for the client device 200 to have time to download the next tile, the size of the dotted box 605 must be smaller than the overlap region 615 . If the dotted box were larger than the area of overlap then the client device 200 would not be able to display a complete image for intermediate views between tiles.
- the content of tile B2 is downloaded from the server 300 and the client device 200 can display the third identified point 450 as shown by the dotted line box 605 and shown in FIG. 6F . This is because the content of the dotted line box 605 in FIG. 6F is provided by tile B2 rather than tile A1.
- the user then continues their movement to the right of the image 400 .
- the dotted line box 605 is from the first (non-overlapping) part of tile B2.
- the displayed view is shown in FIG. 6H .
- FIG. 7A a view of the image 400 split into six tiles of layer 3 is shown. Specifically, the image 400 is split into tile A3, B3 . . . F3. Each of these tiles has two sections. The first section of each tile, similar to that described above with reference to FIG. 6A-6H , covers an area of the image 400 . The second section overlaps with the preceding tile so that as the user controls the view on the client device 200 and the dotted box 705 effectively moves across the image 400 , the next tile in the view can be downloaded from the server 300 to the client device 200 . The client device 200 then displays the appropriate cut out from the tile.
- the dotted box 705 indicates that the user of the client device 200 is viewing the first indicated point 430 .
- the position of the dotted box 705 in this Figure indicates that the user of the client device 200 performed a pinch to zoom operation on the position of the dotted box 605 in FIG. 6A .
- the co-ordinates of the dotted box 705 is shown in FIG. 7A .
- the zoom, pitch and yaw of the dotted box 705 are known to the client device 200 as the user performed a pinch to zoom operation.
- the dotted box 705 is smaller in the image 400 , this indicates that the user of the client device 200 has zoomed in more closely on the first indicative point 430 .
- the client device 200 retrieves tile A3 from the server 300 . That is, the client device 200 retrieves tile A from layer 3. This zoomed in view displayed to the user is shown in FIG. 7B .
- the dotted box 705 indicates that the user is located over the second indicated point 440 .
- the client device 200 will have requested tile D3 from the server 300 as the user moved across the image 400 .
- the second indicated point 440 is located in the first part of tile E3 and the second part of tile D3.
- the second indicated point 440 is located in the overlap between tile D3 and tile E3. Therefore, the client device 200 now retrieves tile E3 from the server 300 .
- the view on the client device 200 is shown in FIG. 7D Again, the user continues to scroll to the right at the same zoom level.
- the dotted box 705 indicates that the user is located over the third indicated point 450 .
- the client device 200 will have requested tile E3 from the server 300 as the user moved across the image 400 .
- the third indicated point 450 is located in the first part of tile F3 and the second part of tile E3. In other words, the third indicated point 450 is located in the overlap between tile E3 and tile F3. Therefore, the client device 200 now retrieves tile F3 from the server 300 .
- the view on the client device 200 is shown in FIG. 7F .
- FIG. 8A shows the view on the client device 200 of FIG. 7F with a user of the client device 200 wishing to zoom out.
- the user's hand 805 is located over the touch screen 230 .
- the user's hand 805 touches the screen at points 815 and 820 .
- Point 810 is equidistant between point 815 and point 820 .
- FIG. 8B shows the image displayed on the client device 200 .
- FIG. 10 shows a table that is stored within the server 300 .
- the table shows the association between the location of the video tile stored within the storage 305 and the name of the video file.
- the table shows a Unique Resource Locator (URL) identifying the unique address of the storage 305 in which the video tile is stored.
- the URL may also be a Unique Resource Identifier (URI).
- the table is stored within the video database 315 of the server 300 .
- this embodiment mentions only a single server for brevity, it is possible that the content will be stored on multiple servers. These servers may form part of a so-called “Cloud Service”. Although the content may be distributed in any fashion amongst the servers, it is possible that each server will provide a subset of the tile streams. By providing the tile streams from multiple servers, the number of client devices 200 connected to the service is scalable.
- the client device 200 downloads the tile from the appropriate URL.
- the tile is stored as an MPEG4-AVC elementary stream with an accompanying index file that specifies the location in the stream of each key-frame.
- the tile may be stored as any appropriate image file.
- the content of the video is stored for the duration of the program. This allows the user to not only control the virtual camera position but that the user can also rewind, pause and fast forward through the content. By uniquely identifying each frame (or Group of Pictures) within the content, each user is allowed to have frame accurate control of the pause, fast forward and rewind through the content.
- the frame (or Group of Pictures) number is provided after the “:” in the URL.
- FIG. 11 shows a table stored within the client device 200 .
- the table of FIG. 11 is stored within the memory 225 of the client device 200 .
- the table of FIG. 11 is downloaded from the video database 315 of the server 300 before the program is to be viewed on the client device 200 .
- the table shows the tile name and the URL where the tile is stored. Further, as will be noted, the URL includes the query ⁇ insert-time>.
- the client device 200 retrieves this value from a counter.
- the server 300 therefore knows which Group of Pictures to retrieve from the server 300 .
- a Group of Pictures is retrieved because it is only possible to switch tile streams on a Group of Picture/Key frame boundary.
- the Group of Picture length is 15 frames (or 0.5 seconds), in embodiments. Therefore, as the switch between title streams can only happen on a Group of Picture/Key Frame boundary, there is a switching latency of at least 0.5 seconds.
- the zoom, pitch and yaw values associated with each tile are required so that the client device 200 can determine which tile to request from server 300 .
- the co-ordinates of the first to third indicated points are also provided.
- the co-ordinates may be either real world 3D positions or 2D co-ordinates in the image 400 . Indeed any invariant co-ordinates that may be easily transformed into the view of the client device 200 may be used.
- a URL to the icon is provided so that when the first indicated point is being displayed to the user, the client device 200 can retrieve the icon from the server 300 .
- the URL may re-direct the user to a different stream of content (for example a broadcast stream of content following a particular player within the content of the image 400 ) for display on the client device 200 .
- This view of the player may be provided as an virtual camera view from the image 400 using player tracking information. Alternatively, this view may be from a dedicated camera.
- a user may be provided with a stadium view. The user can then navigate and/or zoom around the stadium identifying the different events taking place in the stadium. The user may then select a particular indicated point within the client view. This selection will then re-route the user to a particular broadcast video stream using the URL associated with the identified point. Indeed, if there is no available current live broadcast, then either the user of the client device 200 is informed of this, or the user of the client device 200 is re-routed to alternative content based on either the user or the particular selected event.
- the selection of the particular identified point may be either touching the identified point on the touch screen or may be selected by zooming in to the identified point beyond a certain level of zoom.
- the second and third indicated point may include advertisement content specifically targeted to the user of the client device 200 .
- the user of the client device 200 may then select such advertisement which may redirect the user to a different URL such as the website of the advertiser. This may lead to the generation of user specific metadata which could be useful for an advertiser for obtaining data related to the impact of a particular advert or for determining the amount of revenue and advertiser has to pay the broadcaster or service provider.
- FIG. 12 shows a flow chart 1200 explaining the initial operation of the system.
- the process starts in step S 1205 .
- the user of the client device 200 registers with the system.
- the user provides a unique reference that uniquely identifies one user from another user. This may be an Email address, a unique username or the like.
- the user may be registered using their Sony Network username or their Android username, for example.
- details of the user may be collected such as payment details, loyalty card details or the like. Indeed, the user may be asked questions to determine their personal preferences and shopping habits. These may be taken at the time of registration or from any loyalty schemes that the user may subscribe to or any social media accounts which are linked to the user during registration.
- This information is stored within the user database 310 located in the server 300 .
- the application is downloaded to the client device 200 in step 1220 . In other words, the software required to configure the client device 200 is downloaded and stored within the memory 225 of the client device 200 .
- the initial registration process is then completed in step 1225 .
- FIG. 13 shows a flow chart 1300 explaining the operation of the system when a user of the client device wishes to view a program.
- the flow chart starts in step 1305 .
- the user of the client device 200 logs in to their account set up in the initial registration process of FIG. 12 .
- the user is then permitted to select an event to watch in step 1315 .
- the selection may be a recommendation according to the user's previous events or may be restricted to a certain age profile or the like.
- the server 300 selects the appropriate tile map from the video database 315 .
- the tile map is downloaded to the client device 200 in step 1320 .
- the tile map is the table shown in FIG. 11 . This is stored in the memory 225 of the client device 200 .
- the process then ends at step 1330 .
- FIG. 14 shows a flow chart 1400 explaining the operation of the client device 200 .
- the flow chart starts at step 1405 .
- the client device 200 starts a frame counter in step 1410 .
- the GOP counter is incremented every time the client device displays a new GOP. This allows the client device 200 to request the correct tile with frame accuracy from the server 300 .
- the client device 200 downloads the first tile from the server 300 in step 1415 .
- the first tile in this example is tile A1 although the disclosure is not so limited and any tile may be selected as the initial tile either automatically or through user preference.
- the client device 200 determines whether the user has touched the touch screen 230 . This is step 1420 .
- the client device 200 will determine if the user has performed any input operation. If the user does not perform an input, the client device 200 waits until such an input is received. In reality, although the Figure shows a loop back to step 1420 , the client device will increment the GOP counter and will retrieve the next GOP of that tile from the server 300 . This means that a server request is made every 0.5 seconds or so.
- the client device 200 determines whether the user has touched (that is, selected) an indicated point in step 1425 . If the user has selected an indicated point, the yes path is followed and the client device 200 goes to the URL for that indicated point stored in memory 225 . This occurs in step 1430 .
- the value of frame counter is stored in step 1435 so that should the user wish to return to the application, the client device 200 will retrieve the last frame from the server 300 . It is optional whether the value of the tile is also stored, or whether the client device 200 starts from the initial tile should a user return to the application.
- the client device 200 stores the co-ordinates of the view shown to the user of the client device 200 .
- the client device 200 for each frame, stores the co-ordinates of the dotted boxes shown in FIGS. 4-6 .
- This zoom, pitch, yaw data may be uploaded to the server 300 and, if uploaded, this is stored in the user database 310 .
- the co-ordinate data is analysed to determine for each user, their preferred view as well as any occasions where the indicated points were displayed to the user. This enables any advertising revenue to be collected from advertisers. The process will then end in step 1445 .
- the co-ordinate data may be used to generate user driven metadata such as generating a pseudo-broadcast feed defined as the average of the views being watched.
- step 1425 if the user has not selected an indicated point, the “no” step is followed. In other words, the user has changed the view displayed on the client device 200 . So, the co-ordinates of the moved dotted box are calculated according to amount of movement provided by the user during the swipe or the pinch to zoom. This is step 1450 . The user in this step effectively changes the zoom of the virtual camera (with the pinch to zoom) and the pitch and yaw of the virtual camera (with the swipe).
- the co-ordinates of the moved dotted box are then stored in step 1455 .
- the client device 200 determines whether a new tile is needed in step 1460 .
- the client device 200 in step 1465 then adds the value of the frame counter into the URL of the tile from the memory 225 .
- the client device 200 then requests the tile from the server 300 in step 1475 .
- the client device 200 then extracts the view from the received tile. In other words, the client device 200 extracts the content of the dotted box from the received tile and displays this on the display of the client device. This is step 1480 .
- the client device 200 determines whether the user wishes to quit the application by closing the application or pressing an appropriate icon on the touch screen. If the user does not wish to quit the application, the client device 200 returns to step 1425 .
- the coordinates of dotted box during the user's viewing experience are uploaded to the server 300 similarly to that described with reference to steps 1440 .
- the frame counter may be stored to allow a user to return to the same frame and/or the tile may also be stored.
- the process ends at step 1495 .
- step 1460 if a new tile is not required, the no path is followed.
- the client device 200 then extracts the view from the current tile. In other words, the client device 200 extracts the content of the dotted box from the current tile and displays this on the display of the client device. This is step 14951 .
- the client device 200 determines whether the user wishes to quit the application by closing the application or pressing an appropriate icon the touch screen. If the user does not wish to quit the application, the client device 200 returns to step 1450 .
- the coordinates of dotted box during the user's viewing experience are uploaded to the server 300 similarly to that described with reference to step 1440 .
- the frame counter may be stored to allow a user to return to the same frame and/or the tile may also be stored.
- the disclosure is no way limited to this.
- rectangular shaped tiles are described other shapes are envisaged.
- GB1306050.4 filed on 4 Apr. 2013 and also in GB1316102.1 filed 10 Sep. 2013 a method for applying perspective correction to a view of a panoramic scene is described.
- the entire content of this disclosure is hereby incorporated by reference.
- at least the section in this previous application describing the perspective correction is incorporated.
- the perspective correction may be applied to the entire image 400 .
- the rectangular tiles may then be applied to the perspective corrected image 400 .
- the shape of the tile may be selected to apply the perspective correction thereto.
- the client device 200 may perform the perspective correction knowing the co-ordinates of the view (i.e. the dotted box) using the techniques described in GB1306050.4 filed on 4 Apr. 2013 and also in GB1316102.1 filed 10 Sep. 2013.
- the server 300 may be configured to map the image 400 onto a non-rectilinear surface.
- the client 200 may then only supply the co-ordinates of the view required.
- the server 300 can then extract the view at the required co-ordinates (which is a rectangular cut-out) and provide this via the Internet. This allows the client device 200 to have very low processing power such as a set-top box and as the server 300 is only performing a rectangular cut-out of the mapped image (which is very low on processing power), the number of client devices 200 requesting these views is scalable.
- a client device comprising: a communication unit configured to receive a plurality of parameters, the parameters defining segments of an image stored on a server, wherein the segments partially overlap; a processing unit configured to allow selection of a segment of the image in accordance with a defined view on the client device and wherein upon selection the segment is downloadable from the server; and a display configured to display on the client device the cut out of the image taken from the receivable segment.
- a communication unit configured to receive a plurality of parameters, the parameters defining segments of an image stored on a server, wherein the segments partially overlap
- a processing unit configured to allow selection of a segment of the image in accordance with a defined view on the client device and wherein upon selection the segment is downloadable from the server
- a display configured to display on the client device the cut out of the image taken from the receivable segment.
- a device according to any one of clauses 1 to 4, wherein the resolution of the plurality of the segments is less than the resolution of the image. 6.
- a device according to any one of clauses 1 to 5 wherein the cut-out is defined using the same parameters as those used to define the segments and the processing unit is further operable to select the segment by comparing the value of the parameters of the cut-out with the value of the parameters of the segment and selecting the segment having the closest value to that of the cut-out.
- a device according to any one of clauses 1 to 6 further comprising a storage unit configured to store the parameters of the cut out of the image, and to return the stored parameters to the server via the communication unit. 8.
- a device comprising charging advertising revenue on the basis of the returned parameters 9.
- a server comprising: a communication unit configured to send, to a client device, a plurality of parameters, the parameters defining segments of an image stored on the server, wherein the segments partially overlap; and a processing unit configured in use to receive, via the communication unit, a request for a selected segment of the image in accordance with a defined view on the client device and, in use, uploading the segment of the image to the client.
- a server according to clause 11, wherein the amount of overlap between the segments is greater than the defined view on the client device.
- a server according to any one of clauses 11 to 13, wherein the plurality of segments are high definition images.
- a server according to any one of clauses 11 to 14, wherein the resolution of the plurality of the segments is less than the resolution of the image.
- a server uses the same parameters as those used to define the segments and the processing unit is further configured to: select the segment by comparing the value of the parameters of the defined view with the value of the parameters of the segment and selecting the segment having the closest value to that of the defined view.
- the communication unit is configured to receive the parameters of the desired view from the client device.
- a server comprising charging advertising revenue on the basis of the returned parameters 19.
- a server according to any one of clauses 11 to 38 wherein the segment is defined by a Unique Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for upload.
- the communication unit is configured to send, to the client device, parameters of a second set of segments containing a different number of segments of the image, and to send the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
- a method of displaying, on a client device, a cut out of an image stored on a server comprising the steps of: receiving, at the client device, a plurality of parameters, the parameters defining segments of the image stored on the server, wherein the segments partially overlap; selecting the segment of the image in accordance with a defined view on the client device and downloading the segment of the image from the server; and displaying on the client device the cut out of the image taken from the received segment. 22.
- 23. A method according to either clause 21 or clause 22, wherein the plurality of segments are the same size; 24.
- 26. A method according to any one of clauses 21 to 25 wherein the cut-out is defined using the same parameters as those used to define the segments and the step of selecting the segment comprises the step of comparing the value of the parameters of the cut-out with the value of the parameters of the segment and selecting the segment having the closest value to that of the cut-out.
- 27. A method according to any one of clauses 21 to 26 further comprising storing the parameters of the cut out of the image, and returning the stored parameters to the server. 28.
- a method according to any one of clauses 21 to 29 further comprising receiving, at the client device, parameters of a second set of segments containing a different number of segments of the image, and selecting the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
- a method of providing, to a client device, a segment of an image stored on at least one server comprising the steps of: sending, to the client device, a plurality of parameters, the parameters defining segments of the image stored on the server, wherein the segments partially overlap; and receiving a request for a selected segment of the image in accordance with a defined view on the client device and uploading the segment of the image to the client.
- 32. A method according to clause 31, wherein the amount of overlap between the segments is greater than the defined view on the client device.
- 33. A method according to either clause 31 or clause 32, wherein the plurality of segments are the same size.
- 34 A method according to any one of clauses 31 to 33, wherein the plurality of segments are high definition images. 35.
- 36. A method according to any one of clauses 31 to 35 wherein the defined view uses the same parameters as those used to define the segments and the step of selecting the segment comprises the step of comparing the value of the parameters of the defined view with the value of the parameters of the segment and selecting the segment having the closest value to that of the defined view.
- 37. A method according to any one of clauses 31 to 36 further comprising receiving the parameters of the desired view from the client device.
- 39. A method according to any one of clauses 31 to 38 wherein the segment is defined by a Unique
- Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for upload.
- a method according to any one of clauses 31 to 39 further comprising sending, to the client device, parameters of a second set of segments containing a different number of segments of the image, and sending the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
- 41. A computer program comprising computer readable instructions which, when loaded onto a computer, configures the computer to perform a method according to any one of clauses 21 to 40.
- 42. A computer program product configured to store the computer program of clause 41 therein or thereon.
- 43. A method, device, server, computer program or computer program product as substantially hereinbefore described with reference to the accompanying drawings.
Abstract
A client device, comprising:
a communication unit configured to receive a plurality of parameters, the parameters defining segments of an image stored on a server, wherein the segments partially overlap;
a processing unit configured to allow selection of a segment of the image in accordance with a defined view on the client device and wherein upon selection the segment is downloadable from the server; and
a display configured to display on the client device the cut out of the image taken from the receivable segment is disclosed.
a processing unit configured to allow selection of a segment of the image in accordance with a defined view on the client device and wherein upon selection the segment is downloadable from the server; and
a display configured to display on the client device the cut out of the image taken from the receivable segment is disclosed.
Description
- The present application claims priority to United Kingdom patent application GB1320217.1 filed on 15 Nov. 2013, the contents of which being incorporated herein by reference in its entirety.
- 1. Field of the Disclosure
- The present-disclosure relates to a method, server, client and software
- 2. Description of the Related Art
- The “background” description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in the background section, as well as aspects of the description which may not otherwise qualify as prior art at the time of filing, are neither expressly or impliedly admitted as prior art against the present disclosure.
- It is known to generate a cut out having a lower resolution from a higher resolution image. This cut-out image can then be broadcast to many user devices over a network. This network may be a broadcast network or a local or wide area network. However, in this case, the user of each device would be able to only view the broadcast stream. In other words, the user has no control over the position of the cut out and so cannot control their virtual camera view.
- It is an aim of embodiments of the disclosure to address this.
- According to embodiments of the disclosure, there is provided a client device, comprising:
- a communication unit configured to receive a plurality of parameters, the parameters defining segments of an image stored on a server, wherein the segments partially overlap;
a processing unit configured to allow selection of a segment of the image in accordance with a defined view on the client device and wherein upon selection the segment is downloadable from the server; and
a display configured to display on the client device the cut out of the image taken from the receivable segment. - The amount of overlap between the segments may be greater than the cut out of the image.
- The plurality of segments may be the same size.
- The plurality of segments may be high definition images.
- The resolution of the plurality of the segments may be less than the resolution of the image.
- The cut-out may be defined using the same parameters as those used to define the segments and the processing unit is further operable to select the segment by comparing the value of the parameters of the cut-out with the value of the parameters of the segment and selecting the segment having the closest value to that of the cut-out.
- The device may further comprise a storage unit configured to store the parameters of the cut out of the image, and to return the stored parameters to the server via the communication unit.
- The device may comprise charging advertising revenue on the basis of the returned parameters
- The segment may be defined by a Unique Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for download.
- The communication unit may be further configured to receive, at the client device, parameters of a second set of segments containing a different number of segments of the image, and to select the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
- According to other embodiments, there is provided a server comprising: a communication unit configured to send, to a client device, a plurality of parameters, the parameters defining segments of an image stored on the server, wherein the segments partially overlap; and a processing unit configured in use to receive, via the communication unit, a request for a selected segment of the image in accordance with a defined view on the client device and, in use, uploading the segment of the image to the client.
- The amount of overlap between the segments may be greater than the defined view on the client device.
- The plurality of segments may be the same size.
- The plurality of segments may be high definition images.
- The resolution of the plurality of the segments may be less than the resolution of the image.
- The defined view may use the same parameters as those used to define the segments and the processing unit is further configured to: select the segment by comparing the value of the parameters of the defined view with the value of the parameters of the segment and selecting the segment having the closest value to that of the defined view.
- The communication unit may be configured to receive the parameters of the desired view from the client device.
- The server may comprise charging advertising revenue on the basis of the returned parameters
- The segment may be defined by a Unique Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for upload.
- The communication unit may be configured to send, to the client device, parameters of a second set of segments containing a different number of segments of the image, and to send the segment of the image from either the first or second set of segments in accordance with the defined view on the client device. According to other embodiments, there is disclosed a method of displaying, on a client device, a cut out of an image stored on a server, comprising the steps of: receiving, at the client device, a plurality of parameters, the parameters defining segments of the image stored on the server, wherein the segments partially overlap; selecting the segment of the image in accordance with a defined view on the client device and downloading the segment of the image from the server; and displaying on the client device the cut out of the image taken from the received segment.
- The amount of overlap between the segments may be greater than the cut out of the image.
- The plurality of segments may be the same size;
- The plurality of segments may be high definition images.
- The resolution of the plurality of the segments may be less than the resolution of the image.
- The cut-out may be defined using the same parameters as those used to define the segments and the step of selecting the segment comprises the step of comparing the value of the parameters of the cut-out with the value of the parameters of the segment and selecting the segment having the closest value to that of the cut-out.
- The method may further comprise storing the parameters of the cut out of the image, and returning the stored parameters to the server.
- The method may further comprise charging advertising revenue on the basis of the returned parameters
- The segment may be defined by a Unique Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for download.
- The method may further comprise receiving, at the client device, parameters of a second set of segments containing a different number of segments of the image, and selecting the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
- According to other embodiments, a method of providing, to a client device, a segment of an image stored on at least one server, comprising the steps of: sending, to the client device, a plurality of parameters, the parameters defining segments of the image stored on the server, wherein the segments partially overlap; and receiving a request for a selected segment of the image in accordance with a defined view on the client device and uploading the segment of the image to the client.
- The amount of overlap between the segments may be greater than the defined view on the client device.
- The plurality of segments may be the same size.
- The plurality of segments may be high definition images.
- The resolution of the plurality of the segments may be less than the resolution of the image.
- The defined view may use the same parameters as those used to define the segments and the step of selecting the segment comprises the step of comparing the value of the parameters of the defined view with the value of the parameters of the segment and selecting the segment having the closest value to that of the defined view.
- The method may further comprise receiving the parameters of the desired view from the client device.
- The method may further comprise charging advertising revenue on the basis of the returned parameters
- The segment may be defined by a Unique Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for upload.
- The method may further comprise sending, to the client device, parameters of a second set of segments containing a different number of segments of the image, and sending the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
- The foregoing paragraphs have been provided by way of general introduction, and are not intended to limit the scope of the following claims. The described embodiments, together with further advantages, will be best understood by reference to the following detailed description taken in conjunction with the accompanying drawings.
- A more complete appreciation of the disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein:
-
FIG. 1 shows a system according to embodiments of the disclosure; -
FIG. 2 shows a client shown inFIG. 1 ; -
FIG. 3 shows a server shown inFIG. 1 ; -
FIG. 4 shows a simple ultra-high definition image with three landmark points; -
FIG. 5A-5B shows an image tile taken from the image ofFIG. 4 and an indicated virtual camera view provided to the client ofFIG. 2 ; -
FIGS. 6A-6H shows a plurality of image tiles taken from the image ofFIG. 4 and various indicated virtual camera view provided to the client ofFIG. 2 ; -
FIGS. 7A-7F shows a plurality of image tiles taken from the image ofFIG. 4 and various indicated virtual camera view provided to the client ofFIG. 2 ; -
FIGS. 8A and 8B show a user zooming out of the image ofFIG. 7F using the client ofFIG. 2 ; -
FIG. 9 shows a tile map stored within the client ofFIG. 2 ; -
FIG. 10 shows the association between the image tiles and the location thereof stored in the server; -
FIG. 11 shows the association between image tiles and the location thereof stored in the client; -
FIG. 12 shows a flow chart of an initial registration of the user of the client device; -
FIG. 13 shows a flow chart of a the selection of an event to view on the client device; and -
FIG. 14 shows a flow chart explaining the operation of the client device. - Referring now to the drawings, wherein like reference numerals designate identical or corresponding parts throughout the several views.
- Referring to
FIG. 1 , asystem 100 according to embodiments of the disclosure is shown. In thesystem 100, aserver 300 is connected to theInternet 110. Additionally, arouter 120 is also connected to theInternet 110. As would be appreciated, therouter 120 enables devices to connect to theInternet 110. In this case, therouter 120 is a home router located within a user's home. Of course, therouter 120 may be any device that allows any apparatus to connect to theInternet 110. In this case, therouter 120 is connected to adevice 200 and thus thedevice 200 connects to theInternet 110. The device, in this case, is a client device that connects to theserver 300 via theInternet 110. Although any client device is envisaged, in this case, theclient device 200 is a portable client device such as a tablet. The tablet may be a Sony® Xperia Z which runs the Android operating system. However, any tablet device, smartphone, laptop or indeed any client device is envisaged. - Referring to
FIG. 2 , a block diagram of the tablet 200 (client device) is shown. Acommunication unit 210 is connected to anantenna 205. Thecommunication unit 210 is, in embodiments, communicates with therouter 120 using WiFi, such as the IEEE802.11g/n standard. Theantenna 205 may be a MIMO type antenna to reduce interference. Of course, any type of wired or wireless protocol is envisaged. - Indeed, anything that enables the
client device 200 to communicate with theserver 300, either directly or via a network, such as the Internet is envisaged. - The
communication unit 210 is connected to acontroller 220. Thecontroller 220 operates using a computer program which contains computer readable instructions. The computer program which controls thecontroller 230 is stored on amemory 225 which is also connected to thecontroller 220. Thememory 225 may be any kind of storage medium such as a solid state memory, or indeed, an optical or magnetic readable memory or the like is envisaged. Also connected to thecontroller 220 is atouch screen 230. Thetouch screen 230 is a user interface to allow a user to control thetablet 200. In embodiments, thetablet 200 has a “pinch to zoom” function. This will be explained later. Of course, any kind of user interface may be used. - The
controller 220 is also connected to a graphical processor unit (GPU) 215. TheGPU 215 is a processing unit that is specifically configured to process images. TheGPU 215 is entirely optional in embodiments. Further, connected to theGPU 215 is adisplay 235. The size of the display may vary. For example, in the Xperia Z tablet, the display is 10.1 inches and has a resolution of 1920×1200 pixels. - Referring to
FIG. 3 , theserver 300 according to embodiments of the disclosure is shown. In theserver 300storage 305 is provided. Stored within thestorage 305 are video tiles. These will be explained later. Additionally, other video streams may be stored in thestorage 305. Moreover, advertisements or still images, or indeed any content may be stored in thestorage 305. Thestorage 305 may be solid state memory, or magnetic or optical readable storage or a combination thereof. - A
server controller 320 is provided. Theserver controller 320 is a processor that controls the operation of theserver 300. Further, theserver controller 320 performs image processing to generate the image tiles (or segments) stored within thestorage 305. Theserver controller 320 is controlled using a computer program. The computer program contains computer readable instructions which when loaded onto theserver controller 320 makes theserver controller 320 perform certain steps as will be explained later. Also connected to theserver controller 320 is auser database 310. Theuser database 310 stores details about the users connected to theserver 300. This will be explained later. Additionally, avideo database 315 is connected to theserver controller 320. Thevideo database 315 stores the address of the video tile and associates the address with the appropriate video tile (or any other content) stored within thestorage 305 as will be explained later. - Also connected to the
server controller 320 is acommunication circuit 325. Thecommunication circuit 325 connects theserver 300 to theInternet 110 and communicates with theclient device 200. -
FIG. 4 shows animage 400. Theimage 400 contains asoccer pitch 410 and a surroundingarea 420. Within the image are three identifiedpoints identified points 440, 450 (shown as a “Y” and “Z” in the Figure) is user specific content. These identified points are selected in advance by the content provider and their locations are stored within the tile map shown inFIG. 9 . The tile map, as will become apparent, is a set of co-ordinates defining segments in the image. - The
image 400 is, in embodiments, an ultra-high definition image. That is, theimage 400 is comprised of two images stitched together. Specifically, in embodiments, the image is comprised of two 4K images stitched together. This means thatimage 400 has a resolution of 8192×2160 pixels. Clearly, any kind of image or any size of image may be used. - Referring to
FIG. 9 , the video tiles are explained with reference to the video tile map. The video tile map ofFIG. 9 is stored within thememory 225 of theclient device 200. This tile map is specific to any particular stream and is downloaded in advance of the content. The tile map is a set of pitch, zoom and yaw values that define segments (or tiles) of theimage 400. These pitch, yaw and zoom values are the values that would be applied to a virtual camera when cutting out the tile from theimage 400. The method for calculating the pitch, zoom and yaw are known and are disclosed in GB1306050.4 filed on 4 Apr. 2013 and also in GB1316102.1 filed 10 Sep. 2013. The contents of these documents are hereby incorporated by reference. The pitch, yaw and zoom values are examples of parameters which define the tiles (or segments) of theimage 400. - As noted above, the pitch, yaw and zoom of the first, second and third
identified points FIG. 9 shows tiles only overlapping in the horizontal direction, this is only for ease of explanation. It is also intended that tiles overlap both the horizontal and vertical directions and give the advantages explained below in all directions in the image. Alternatively, the size of the tile could depend on the display resolution of the of the client device as will become apparent. Although not necessary, it is preferable that the resolution of the tile conforms to an acceptable standard size to ensure high efficiency codecs for efficiency in storage and transmission. In other words, the size of the tiles is a size for which regular codecs exist. The high definition example is an acceptable standard size tile. - These layers correspond to different zoom levels within the
image 400. That is, inlayer 1, one tile covers theimage 400 and so the content of the tile is theentire image 400 albeit at a reduced resolution. In other words, the content of theimage 400 is provided at a resolution of the tile rather than the ultra-high definition resolution. - In
layer 2, theimage 400 is split into a plurality of overlapping tiles. As the resolution of the tiles between the layers is the same, and because theimage 400 is split into more tiles inlayer 2 than inlayer 1, the content of each of the two tiles withinlayer 2 contains a different segment of the image 400 (except for an overlapping section which will be explained later). This means that the content of tile A inlayer 2 contains the left side of the image and the content of tile B inlayer 2 contains the right side of theimage 400. Therefore, the content of each tile segment inlayer 2 appears magnified when compared to the content of the tile inlayer 1. - Similarly, in
layer 3, theimage 400 is split into six overlapping tiles all having the same resolution as the tiles inlayer layer 3 is therefore a more magnified version of the content in each oflayer - To summarise, therefore, the different layers all contain a different number of tiles. The resolution of all the tiles, irrespective of layer, is the same. Therefore, the more tiles contained in a layer, the more “zoomed” into the
image 400 the content within that tile appears. As explained above, it is also important to note here that the view of each tile has a zoom value, a pitch value and a yaw value associated with it. In other words, each tile within each layer is analogous to a virtual camera capturing part or theimage 400. The amount of pitch, yaw and zoom which must be applied to the virtual camera in order to capture this field of view is determined using a known technique such as that described in GB1306050.4 filed on 4 Apr. 2013 and also in GB1316102.1 filed 10 Sep. 2013, the entire contents of which is hereby incorporated by reference. The zoom value, pitch value and yaw values are stored in theclient device 200 in association with the tile. The tiles of the image are pre-processed and stored on the server. - As will be explained later, as the
client device 200 requests different tiles to be provided by theserver 300, the user of the client device can zoom into theimage 400. In other words, theclient device 200 starts by displaying the content of the tile oflayer 1 and if the user wishes to zoom into the image, theclient device 200 requests the appropriate tile fromlayer 2 and thenlayer 3 in order to allow the user to zoom into the image and to thus control the zoom of a virtual camera aroundimage 400. In other words, theclient device 200 also generates a virtual camera view of the tile provided to it by theserver 300. - Again, the technique for determining the client side virtual camera parameters (the zoom, pitch and yaw) is determined using a known technique such as that disclosed in GB1306050.4 filed on 4 Apr. 2013 and also in GB1316102.1 filed 10 Sep. 2013. Specifically, the pinch control on the
client device 200 controls the zoom axis of the client side virtual camera and the swipe action controls both the pitch and yaw axes of the client side virtual camera. - The tiles will be explained in more detail with reference to
FIGS. 5 , 6 and 7. - Referring to
FIG. 5A , theimage 400 is provided as onetile segment 500. In the numbering of the Figures, the name of the tile indicates a unique letter identifying each tile within a layer as well as the number of the layer. So, inFIG. 5A , theimage 400 is contained within onetile 500. This tile is “tile A1” (or is tile A in layer 1). InFIG. 5A , a dottedbox 510 is provided within thetile 500. Thisdotted box 510 indicates the view that is being displayed on theclient device 200. As will be explained later, theserver 300 provides the content of the tile to theclient device 200 and theclient device 200 will extract the appropriate section of the tile for display to the user. However, the dottedbox 510 has been included to aid explanation. - Referring to
FIG. 5B , the screen of theclient device 200 is shown. As can be seen, the content of the dottedbox 510 inFIG. 5A is displayed to the user of theclient device 200. - Imagine that the user of the
client device 200 wishes to zoom in to the first identified point (point X) 430 of theimage 400. The user will perform a “pinch to zoom” action on thetouch screen 230 around the first identifiedpoint 430. In the specific case, the user will swipe the screen to pan around theimage 400 to align the first identifiedpoint 430 with the centre of the view. Once the desired view is reached, the pinch zoom is applied which increases the zoom parameter of the client side virtual camera and thus zooms in on the centre of the view. - Referring to
FIG. 6A , the dottedbox 605 shows the section of theimage 400 that is being displayed on theclient device 200. However, as the user of theclient device 200 performs such a zoom, and as will be explained later, theclient device 200 determines that a tile fromlayer 2 is required. In other words, the user of theclient device 200 wants a more detailed view of the first identified point (point X) 430 than was possible using tile A fromlayer 1. Therefore, theclient device 200 requests a tile fromlayer 2 as the zoom action takes place. - This is achieved, in embodiments, by the
client device 200 knowing from the tile map stored in thememory 225 the zoom, pitch and yaw values of each of the tiles. Theclient device 200 then compares the client side zoom, pitch and yaw of the image shown on its display to the zoom, pitch and yaw of the stored tiles. Theclient device 200 then retrieves a tile with the zoom, pitch and yaw closer to the client side value of zoom, pitch and yaw of the tile currently being displayed. It is important to note that theclient device 200 will switch to a different tile when the view displayed on theclient device 200 will not fit inside the tile view. - It is also important to note that at the time of tile generation, the
server 300 ensures that there is sufficient overlap between the tiles to allow the view of theclient device 200 to fit within the overlap region. Additionally, theclient device 200 will only ever have a zoom level higher than the zoom level associated with a particular tile. Therefore, when searching for a new tile, theclient device 200 may only search tiles having a zoom level lower than the current zoom level associated with theclient device 200. This reduces processing requirements of theclient device 200. - Further, the
client device 200 employs a predictive scheme such as a linear scheme or a Kalman filter in order to predict which tile needs downloading next. Specifically, inFIG. 6A and inlayer 2,image 400 is split into two tiles, tile A2 and B2. In this, tile A2 has two parts. Afirst part 610 of tile A2 covers the left side of theimage 400 and asecond part 615 of tile A2 covers a more central position. The second part of tile A2 overlaps with a second part of tile B2. The first part of tile B2 covers the right side of theimage 400. Therefore, theclient device 200 retrieves tile A2 from thestorage 305 within theserver 300. As shown inFIG. 6B , the dottedbox 605A is shown on theclient device 200. As can be seen, the first identified point (point X) 430 appears much larger on the display of theclient device 200. - Although not explicitly shown in
FIG. 6A , but as noted above, the two tiles (Tile A2 and Tile B2) are provided with zoom, pitch and yaw values. These values indicate the zoom, pitch and yaw values of a virtual camera capturing the image of the tile from theimage 400. In other words, the zoom, pitch and yaw values of the tiles are those values of a virtual camera capturing the image of the tile fromimage 400. As understood by the skilled person, the relationship between two perspective views is defined by an homography. The tile view is defined by zoom, pitch and yaw with respect to theimage 400 and the client device view is defined by zoom, pitch and yaw. The homography transform in this case is calculated from the differences in the zoom, pitch and yaw between theclient device 200 view and the zoom, pitch and yaw of the specific tile. - Next, if the user of the
client device 200 swipes the touch screen, the user of theclient device 200 can move around theimage 400 at this zoom level. Therefore, if we assume the user of theclient device 200 moves the view to the right, the view displayed to the user will change. In particular, we can assume that the user of theclient device 200 has “swiped” the touch screen such that the view displayed to the user is new dottedbox 605 inFIG. 6C . In other words, the user of theclient device 200 is now viewing the second identified point (point Y) 440. This view is shown inFIG. 6D . - As the
client device 200 knows the starting zoom, pitch and yaw client side parameters of the dotted box (the position inFIG. 6A ), theclient device 200 calculates the client side zoom, pitch and yaw of the new dottedbox 605. It should be noted here that the user has moved their view into thesecond area 615 of tile A2. In other words, the dottedbox 605 which indicates the image displayed on theclient device 200 is taken from the overlappingregion 615 between tile A2 and tile B2. - In order to ensure that the user of the
client device 200 may continue scrolling to the right smoothly, theclient device 200 requests that tile B2 be provided to theclient device 200 from theserver 300. It is important to note here that in order for theclient device 200 to have time to download the next tile, the size of the dottedbox 605 must be smaller than theoverlap region 615. If the dotted box were larger than the area of overlap then theclient device 200 would not be able to display a complete image for intermediate views between tiles. - When the user of the
client device 200 scrolls to the area shown inFIG. 6E , the content of tile B2 is downloaded from theserver 300 and theclient device 200 can display the thirdidentified point 450 as shown by the dottedline box 605 and shown inFIG. 6F . This is because the content of the dottedline box 605 inFIG. 6F is provided by tile B2 rather than tile A1. - The user then continues their movement to the right of the
image 400. As can be seen inFIG. 6G , the dottedline box 605 is from the first (non-overlapping) part of tile B2. The displayed view is shown inFIG. 6H . - Referring now to
FIG. 7A , a view of theimage 400 split into six tiles oflayer 3 is shown. Specifically, theimage 400 is split into tile A3, B3 . . . F3. Each of these tiles has two sections. The first section of each tile, similar to that described above with reference toFIG. 6A-6H , covers an area of theimage 400. The second section overlaps with the preceding tile so that as the user controls the view on theclient device 200 and the dottedbox 705 effectively moves across theimage 400, the next tile in the view can be downloaded from theserver 300 to theclient device 200. Theclient device 200 then displays the appropriate cut out from the tile. - So, in
FIG. 7A , the dottedbox 705 indicates that the user of theclient device 200 is viewing the first indicatedpoint 430. Importantly, the position of the dottedbox 705 in this Figure indicates that the user of theclient device 200 performed a pinch to zoom operation on the position of the dottedbox 605 inFIG. 6A . For ease of reference, the co-ordinates of the dottedbox 705 is shown inFIG. 7A . - Specifically, the zoom, pitch and yaw of the dotted
box 705 are known to theclient device 200 as the user performed a pinch to zoom operation. Illustratively, as the dottedbox 705 is smaller in theimage 400, this indicates that the user of theclient device 200 has zoomed in more closely on the firstindicative point 430. In other words, as the user has zoomed in on the first indicatedpoint 430, theclient device 200 retrieves tile A3 from theserver 300. That is, theclient device 200 retrieves tile A fromlayer 3. This zoomed in view displayed to the user is shown inFIG. 7B . - Again, the user continues to scroll to the right at the same zoom level. The dotted
box 705 indicates that the user is located over the second indicatedpoint 440. As will be apparent, theclient device 200 will have requested tile D3 from theserver 300 as the user moved across theimage 400. The second indicatedpoint 440 is located in the first part of tile E3 and the second part of tile D3. In other words, the second indicatedpoint 440 is located in the overlap between tile D3 and tile E3. Therefore, theclient device 200 now retrieves tile E3 from theserver 300. The view on theclient device 200 is shown inFIG. 7D Again, the user continues to scroll to the right at the same zoom level. The dottedbox 705 indicates that the user is located over the thirdindicated point 450. As will be apparent, theclient device 200 will have requested tile E3 from theserver 300 as the user moved across theimage 400. The third indicatedpoint 450 is located in the first part of tile F3 and the second part of tile E3. In other words, the thirdindicated point 450 is located in the overlap between tile E3 and tile F3. Therefore, theclient device 200 now retrieves tile F3 from theserver 300. The view on theclient device 200 is shown inFIG. 7F . -
FIG. 8A shows the view on theclient device 200 ofFIG. 7F with a user of theclient device 200 wishing to zoom out. The user'shand 805 is located over thetouch screen 230. The user'shand 805 touches the screen atpoints Point 810 is equidistant betweenpoint 815 andpoint 820. As the user moves his or her finger in the direction of the arrows (i.e. towards point 810), the displayed image will appear to zoom out andFIG. 8B shows the image displayed on theclient device 200. -
FIG. 10 shows a table that is stored within theserver 300. Specifically, the table shows the association between the location of the video tile stored within thestorage 305 and the name of the video file. The table shows a Unique Resource Locator (URL) identifying the unique address of thestorage 305 in which the video tile is stored. In more general terms, the URL may also be a Unique Resource Identifier (URI). The table is stored within thevideo database 315 of theserver 300. Of course, although this embodiment mentions only a single server for brevity, it is possible that the content will be stored on multiple servers. These servers may form part of a so-called “Cloud Service”. Although the content may be distributed in any fashion amongst the servers, it is possible that each server will provide a subset of the tile streams. By providing the tile streams from multiple servers, the number ofclient devices 200 connected to the service is scalable. - During operation of the
client device 200, theclient device 200 downloads the tile from the appropriate URL. The tile is stored as an MPEG4-AVC elementary stream with an accompanying index file that specifies the location in the stream of each key-frame. Although, of course, the tile may be stored as any appropriate image file. Furthermore, it should be noted that the content of the video is stored for the duration of the program. This allows the user to not only control the virtual camera position but that the user can also rewind, pause and fast forward through the content. By uniquely identifying each frame (or Group of Pictures) within the content, each user is allowed to have frame accurate control of the pause, fast forward and rewind through the content. The frame (or Group of Pictures) number is provided after the “:” in the URL. -
FIG. 11 shows a table stored within theclient device 200. Specifically, the table ofFIG. 11 is stored within thememory 225 of theclient device 200. The table ofFIG. 11 is downloaded from thevideo database 315 of theserver 300 before the program is to be viewed on theclient device 200. The table shows the tile name and the URL where the tile is stored. Further, as will be noted, the URL includes the query <insert-time>. As will be explained later, theclient device 200 retrieves this value from a counter. Theserver 300 therefore knows which Group of Pictures to retrieve from theserver 300. Specifically, a Group of Pictures is retrieved because it is only possible to switch tile streams on a Group of Picture/Key frame boundary. The Group of Picture length is 15 frames (or 0.5 seconds), in embodiments. Therefore, as the switch between title streams can only happen on a Group of Picture/Key Frame boundary, there is a switching latency of at least 0.5 seconds. - Additionally provided in the table are the zoom, pitch and yaw values associated with each tile. These parameter values of the tile are required so that the
client device 200 can determine which tile to request fromserver 300. - Although not specifically shown in the Figure, the co-ordinates of the first to third indicated points are also provided. The co-ordinates may be either real world 3D positions or 2D co-ordinates in the
image 400. Indeed any invariant co-ordinates that may be easily transformed into the view of theclient device 200 may be used. Additionally, in the case of the first indicated point (which is a user selectable icon), a URL to the icon is provided so that when the first indicated point is being displayed to the user, theclient device 200 can retrieve the icon from theserver 300. Additionally, the URL may re-direct the user to a different stream of content (for example a broadcast stream of content following a particular player within the content of the image 400) for display on theclient device 200. This view of the player may be provided as an virtual camera view from theimage 400 using player tracking information. Alternatively, this view may be from a dedicated camera. As another example, in a multi-disciplinary event such as the Olympics, a user may be provided with a stadium view. The user can then navigate and/or zoom around the stadium identifying the different events taking place in the stadium. The user may then select a particular indicated point within the client view. This selection will then re-route the user to a particular broadcast video stream using the URL associated with the identified point. Indeed, if there is no available current live broadcast, then either the user of theclient device 200 is informed of this, or the user of theclient device 200 is re-routed to alternative content based on either the user or the particular selected event. In some embodiments, the selection of the particular identified point may be either touching the identified point on the touch screen or may be selected by zooming in to the identified point beyond a certain level of zoom. - Further, with regard to the second and third indicated point, this is content that is provided specifically to the user. As will be explained later, the second and third indicated point may include advertisement content specifically targeted to the user of the
client device 200. The user of theclient device 200 may then select such advertisement which may redirect the user to a different URL such as the website of the advertiser. This may lead to the generation of user specific metadata which could be useful for an advertiser for obtaining data related to the impact of a particular advert or for determining the amount of revenue and advertiser has to pay the broadcaster or service provider. -
FIG. 12 shows aflow chart 1200 explaining the initial operation of the system. The process starts instep S 1205. The user of theclient device 200 then registers with the system. The user provides a unique reference that uniquely identifies one user from another user. This may be an Email address, a unique username or the like. The user may be registered using their Sony Network username or their Android username, for example. During the registration process details of the user may be collected such as payment details, loyalty card details or the like. Indeed, the user may be asked questions to determine their personal preferences and shopping habits. These may be taken at the time of registration or from any loyalty schemes that the user may subscribe to or any social media accounts which are linked to the user during registration. This information is stored within theuser database 310 located in theserver 300. After successful registration, the application is downloaded to theclient device 200 instep 1220. In other words, the software required to configure theclient device 200 is downloaded and stored within thememory 225 of theclient device 200. - The initial registration process is then completed in
step 1225. -
FIG. 13 shows aflow chart 1300 explaining the operation of the system when a user of the client device wishes to view a program. The flow chart starts instep 1305. The user of theclient device 200 logs in to their account set up in the initial registration process ofFIG. 12 . The user is then permitted to select an event to watch instep 1315. The selection may be a recommendation according to the user's previous events or may be restricted to a certain age profile or the like. - After the user has selected the event, the
server 300 selects the appropriate tile map from thevideo database 315. The tile map is downloaded to theclient device 200 instep 1320. As explained, the tile map is the table shown inFIG. 11 . This is stored in thememory 225 of theclient device 200. - The process then ends at
step 1330. -
FIG. 14 shows aflow chart 1400 explaining the operation of theclient device 200. The flow chart starts atstep 1405. Theclient device 200 starts a frame counter instep 1410. The GOP counter is incremented every time the client device displays a new GOP. This allows theclient device 200 to request the correct tile with frame accuracy from theserver 300. Theclient device 200 downloads the first tile from theserver 300 instep 1415. The first tile in this example is tile A1 although the disclosure is not so limited and any tile may be selected as the initial tile either automatically or through user preference. - The
client device 200 then determines whether the user has touched thetouch screen 230. This isstep 1420. Of course, if theclient device 200 is being operated using a mouse or other input device, theclient device 200 will determine if the user has performed any input operation. If the user does not perform an input, theclient device 200 waits until such an input is received. In reality, although the Figure shows a loop back tostep 1420, the client device will increment the GOP counter and will retrieve the next GOP of that tile from theserver 300. This means that a server request is made every 0.5 seconds or so. - If the
client device 200 does establish that the user has touched thetouch screen 230, theclient device 200 determines whether the user has touched (that is, selected) an indicated point instep 1425. If the user has selected an indicated point, the yes path is followed and theclient device 200 goes to the URL for that indicated point stored inmemory 225. This occurs instep 1430. The value of frame counter is stored instep 1435 so that should the user wish to return to the application, theclient device 200 will retrieve the last frame from theserver 300. It is optional whether the value of the tile is also stored, or whether theclient device 200 starts from the initial tile should a user return to the application. - Additionally, although not specifically shown in the Figure, the
client device 200 stores the co-ordinates of the view shown to the user of theclient device 200. In other words, theclient device 200, for each frame, stores the co-ordinates of the dotted boxes shown inFIGS. 4-6 . This zoom, pitch, yaw data may be uploaded to theserver 300 and, if uploaded, this is stored in theuser database 310. The co-ordinate data is analysed to determine for each user, their preferred view as well as any occasions where the indicated points were displayed to the user. This enables any advertising revenue to be collected from advertisers. The process will then end instep 1445. Additionally, the co-ordinate data may be used to generate user driven metadata such as generating a pseudo-broadcast feed defined as the average of the views being watched. - Returning to step 1425, if the user has not selected an indicated point, the “no” step is followed. In other words, the user has changed the view displayed on the
client device 200. So, the co-ordinates of the moved dotted box are calculated according to amount of movement provided by the user during the swipe or the pinch to zoom. This isstep 1450. The user in this step effectively changes the zoom of the virtual camera (with the pinch to zoom) and the pitch and yaw of the virtual camera (with the swipe). - The co-ordinates of the moved dotted box are then stored in
step 1455. - The
client device 200 then determines whether a new tile is needed instep 1460. - If a new tile is needed, the yes path is followed. The
client device 200 instep 1465 then adds the value of the frame counter into the URL of the tile from thememory 225. Theclient device 200 then requests the tile from theserver 300 instep 1475. - The
client device 200 then extracts the view from the received tile. In other words, theclient device 200 extracts the content of the dotted box from the received tile and displays this on the display of the client device. This isstep 1480. - The
client device 200 then determines whether the user wishes to quit the application by closing the application or pressing an appropriate icon on the touch screen. If the user does not wish to quit the application, theclient device 200 returns to step 1425. - Alternatively, if the user does decide to quit the application the coordinates of dotted box during the user's viewing experience are uploaded to the
server 300 similarly to that described with reference tosteps 1440. Optionally, the frame counter may be stored to allow a user to return to the same frame and/or the tile may also be stored. - The process ends at
step 1495. - Returning to step 1460, if a new tile is not required, the no path is followed. The
client device 200 then extracts the view from the current tile. In other words, theclient device 200 extracts the content of the dotted box from the current tile and displays this on the display of the client device. This isstep 14951. Theclient device 200 then determines whether the user wishes to quit the application by closing the application or pressing an appropriate icon the touch screen. If the user does not wish to quit the application, theclient device 200 returns to step 1450. - Alternatively, if the user does decide to quit the application the coordinates of dotted box during the user's viewing experience are uploaded to the
server 300 similarly to that described with reference to step 1440. Optionally, the frame counter may be stored to allow a user to return to the same frame and/or the tile may also be stored. - Obviously, the above is a non-limiting example and the scope of protection should be in no way limited to the example.
- Although the above is described with reference to certain shapes of tiles, the disclosure is no way limited to this. In fact, although rectangular shaped tiles are described other shapes are envisaged. For example, in GB1306050.4 filed on 4 Apr. 2013 and also in GB1316102.1 filed 10 Sep. 2013 a method for applying perspective correction to a view of a panoramic scene is described. The entire content of this disclosure is hereby incorporated by reference. In particular, at least the section in this previous application describing the perspective correction is incorporated. In this example, the perspective correction may be applied to the
entire image 400. The rectangular tiles may then be applied to the perspective correctedimage 400. Alternatively, if theimage 400 does not have the perspective correction applied thereto, the shape of the tile may be selected to apply the perspective correction thereto. Additionally, if theimage 400 is not perspective corrected, and the tile shape is rectangular, then theclient device 200 may perform the perspective correction knowing the co-ordinates of the view (i.e. the dotted box) using the techniques described in GB1306050.4 filed on 4 Apr. 2013 and also in GB1316102.1 filed 10 Sep. 2013. - Additionally, the
server 300 may be configured to map theimage 400 onto a non-rectilinear surface. In this case, theclient 200 may then only supply the co-ordinates of the view required. Theserver 300 can then extract the view at the required co-ordinates (which is a rectangular cut-out) and provide this via the Internet. This allows theclient device 200 to have very low processing power such as a set-top box and as theserver 300 is only performing a rectangular cut-out of the mapped image (which is very low on processing power), the number ofclient devices 200 requesting these views is scalable. - Obviously, numerous modifications and variations of the present disclosure are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims, the disclosure may be practiced otherwise than as specifically described herein.
- In so far as embodiments of the disclosure have been described as being implemented, at least in part, by software-controlled data processing apparatus, it will be appreciated that a non-transitory machine-readable medium carrying such software, such as an optical disk, a magnetic disk, semiconductor memory or the like, is also considered to represent an embodiment of the present disclosure.
- Embodiments of the disclosure can be generalised into the following numbered clauses:
- 1. A client device, comprising:
a communication unit configured to receive a plurality of parameters, the parameters defining segments of an image stored on a server, wherein the segments partially overlap;
a processing unit configured to allow selection of a segment of the image in accordance with a defined view on the client device and wherein upon selection the segment is downloadable from the server; and
a display configured to display on the client device the cut out of the image taken from the receivable segment.
2. A device according toclause 1, wherein the amount of overlap between the segments is greater than the cut out of the image.
3. A device according to eitherclause 1 orclause 2, wherein the plurality of segments are the same size.
4. A device according to any one ofclauses 1 to 3, wherein the plurality of segments are high definition images.
5. A device according to any one ofclauses 1 to 4, wherein the resolution of the plurality of the segments is less than the resolution of the image.
6. A device according to any one ofclauses 1 to 5 wherein the cut-out is defined using the same parameters as those used to define the segments and the processing unit is further operable to select the segment by comparing the value of the parameters of the cut-out with the value of the parameters of the segment and selecting the segment having the closest value to that of the cut-out.
7. A device according to any one ofclauses 1 to 6 further comprising a storage unit configured to store the parameters of the cut out of the image, and to return the stored parameters to the server via the communication unit.
8. A device according to clause 7, comprising charging advertising revenue on the basis of the returned parameters
9. A device according to any one ofclauses 1 to 8 wherein the segment is defined by a Unique Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for download.
10. A device according to any one ofclauses 1 to 9 wherein the communication unit is further configured to receive, at the client device, parameters of a second set of segments containing a different number of segments of the image, and to select the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
11. A server comprising:
a communication unit configured to send, to a client device, a plurality of parameters, the parameters defining segments of an image stored on the server, wherein the segments partially overlap; and
a processing unit configured in use to receive, via the communication unit, a request for a selected segment of the image in accordance with a defined view on the client device and, in use, uploading the segment of the image to the client.
12. A server according to clause 11, wherein the amount of overlap between the segments is greater than the defined view on the client device.
13. A server according to either clause 11 or clause 12, wherein the plurality of segments are the same size.
14. A server according to any one of clauses 11 to 13, wherein the plurality of segments are high definition images.
15. A server according to any one of clauses 11 to 14, wherein the resolution of the plurality of the segments is less than the resolution of the image.
16. A server according to any one of clauses 11 to 15 wherein the defined view uses the same parameters as those used to define the segments and the processing unit is further configured to: select the segment by comparing the value of the parameters of the defined view with the value of the parameters of the segment and selecting the segment having the closest value to that of the defined view.
17. A server according to any one of clauses 11 to 16 further wherein the communication unit is configured to receive the parameters of the desired view from the client device.
18. A server according to clause 17, comprising charging advertising revenue on the basis of the returned parameters
19. A server according to any one of clauses 11 to 38 wherein the segment is defined by a Unique Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for upload.
20. A server according to any one of clauses 11 to 19 further wherein the communication unit is configured to send, to the client device, parameters of a second set of segments containing a different number of segments of the image, and to send the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
21. A method of displaying, on a client device, a cut out of an image stored on a server, comprising the steps of:
receiving, at the client device, a plurality of parameters, the parameters defining segments of the image stored on the server, wherein the segments partially overlap;
selecting the segment of the image in accordance with a defined view on the client device and downloading the segment of the image from the server; and
displaying on the client device the cut out of the image taken from the received segment.
22. A method according to clause 21, wherein the amount of overlap between the segments is greater than the cut out of the image.
23. A method according to either clause 21 or clause 22, wherein the plurality of segments are the same size;
24. A method according to any one of clauses 21 to 23, wherein the plurality of segments are high definition images.
25. A method according to any one of clauses 21 to 24, wherein the resolution of the plurality of the segments is less than the resolution of the image.
26. A method according to any one of clauses 21 to 25 wherein the cut-out is defined using the same parameters as those used to define the segments and the step of selecting the segment comprises the step of comparing the value of the parameters of the cut-out with the value of the parameters of the segment and selecting the segment having the closest value to that of the cut-out.
27. A method according to any one of clauses 21 to 26 further comprising storing the parameters of the cut out of the image, and returning the stored parameters to the server.
28. A method according to clause 27, comprising charging advertising revenue on the basis of the returned parameters
29. A method according to any one of clauses 21 to 28 wherein the segment is defined by a Unique Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for download.
30. A method according to any one of clauses 21 to 29 further comprising receiving, at the client device, parameters of a second set of segments containing a different number of segments of the image, and selecting the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
31. A method of providing, to a client device, a segment of an image stored on at least one server, comprising the steps of:
sending, to the client device, a plurality of parameters, the parameters defining segments of the image stored on the server, wherein the segments partially overlap; and
receiving a request for a selected segment of the image in accordance with a defined view on the client device and uploading the segment of the image to the client.
32. A method according to clause 31, wherein the amount of overlap between the segments is greater than the defined view on the client device.
33. A method according to either clause 31 or clause 32, wherein the plurality of segments are the same size.
34. A method according to any one of clauses 31 to 33, wherein the plurality of segments are high definition images.
35. A method according to any one of clauses 31 to 34, wherein the resolution of the plurality of the segments is less than the resolution of the image.
36. A method according to any one of clauses 31 to 35 wherein the defined view uses the same parameters as those used to define the segments and the step of selecting the segment comprises the step of comparing the value of the parameters of the defined view with the value of the parameters of the segment and selecting the segment having the closest value to that of the defined view.
37. A method according to any one of clauses 31 to 36 further comprising receiving the parameters of the desired view from the client device.
38. A method according to clause 37, comprising charging advertising revenue on the basis of the returned parameters
39. A method according to any one of clauses 31 to 38 wherein the segment is defined by a Unique - Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for upload.
- 40. A method according to any one of clauses 31 to 39 further comprising sending, to the client device, parameters of a second set of segments containing a different number of segments of the image, and sending the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
41. A computer program comprising computer readable instructions which, when loaded onto a computer, configures the computer to perform a method according to any one of clauses 21 to 40.
42. A computer program product configured to store the computer program of clause 41 therein or thereon.
43. A method, device, server, computer program or computer program product as substantially hereinbefore described with reference to the accompanying drawings.
Claims (20)
1. A client device, comprising:
a communication circuitry configured to receive a plurality of parameters, the parameters defining segments of an image stored on a server, wherein the segments partially overlap;
a processing circuitry configured to allow selection of a segment of the image in accordance with a defined view on the client device and wherein upon selection the segment is downloadable from the server; and
a display circuitry configured to display on the client device the cut out of the image taken from the receivable segment;
a storage circuitry configured to store the parameters of the cut out of the image, and to return the stored parameters to the server via the communication unit.
2. The device according to claim 1 , wherein the amount of overlap between the segments is greater than the cut out of the image.
3. The device according to claim 1 , wherein the plurality of segments are the same size.
4. The device according to claim 1 , wherein the plurality of segments are high definition images.
5. The device according to claim 1 , wherein the resolution of the plurality of the segments is less than the resolution of the image.
6. The device according to claim 1 wherein the cut-out is defined using the same parameters as those used to define the segments and the processing unit is further operable to select the segment by comparing the value of the parameters of the cut-out with the value of the parameters of the segment and selecting the segment having the closest value to that of the cut-out.
7. The device according to claim 1 , comprising charging advertising revenue on the basis of the returned parameters
8. The device according to claim 1 wherein the segment is defined by a Unique Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for download.
9. The device according to claim 1 wherein the communication circuitry is further configured to receive, at the client device, parameters of a second set of segments containing a different number of segments of the image, and to select the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
10. A server comprising:
a communication circuitry is configured to send, to a client device, a plurality of parameters, the parameters defining segments of an image stored on the server, wherein the segments partially overlap; and
a processing circuitry is configured to receive, via the communication circuitry, a request for a selected segment of the image in accordance with a defined view on the client device and uploading the segment of the image to the client, wherein the communication circuitry is configured to receive the parameters of the desired view from the client device.
11. The server according to claim 10 , wherein the amount of overlap between the segments is greater than the defined view on the client device.
12. The server according to claim 10 , wherein the plurality of segments are the same size.
13. The server according to claim 10 , wherein the plurality of segments are high definition images.
14. The server according to claim 10 , wherein the resolution of the plurality of the segments is less than the resolution of the image.
15. The server according to claim 10 wherein the defined view uses the same parameters as those used to define the segments and the processing circuitry is further configured to: select the segment by comparing the value of the parameters of the defined view with the value of the parameters of the segment and selecting the segment having the closest value to that of the defined view.
16. The server according to claim 10 , comprising charging advertising revenue on the basis of the returned parameters
17. The server according to claim 10 wherein the segment is defined by a Unique Resource Identifier identifying the location of the segment and a time identifier uniquely identifying the Group of Pictures of the segment required for upload.
18. The server according to claim 10 further wherein the communication circuitry is configured to send, to the client device, parameters of a second set of segments containing a different number of segments of the image, and to send the segment of the image from either the first or second set of segments in accordance with the defined view on the client device.
19. A method of displaying, on a client device, a cut out of an image stored on a server, comprising the steps of:
receiving, at the client device, a plurality of parameters, the parameters defining segments of the image stored on the server, wherein the segments partially overlap;
selecting the segment of the image in accordance with a defined view on the client device and downloading the segment of the image from the server; and
displaying on the client device the cut out of the image taken from the received segment; and
storing the parameters of the cut out of the image and returning the stored parameters to the server.
20. A non-transitory computer readable medium including computer program instructions, which when executed by a computer causes the computer to perform the method of claim 19 .
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB1320217.1A GB2520310A (en) | 2013-11-15 | 2013-11-15 | A method, server, client and software |
GB1320217.1 | 2013-11-15 |
Publications (1)
Publication Number | Publication Date |
---|---|
US20150143421A1 true US20150143421A1 (en) | 2015-05-21 |
Family
ID=49883689
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/520,438 Abandoned US20150143421A1 (en) | 2013-11-15 | 2014-10-22 | Method, server, client and software |
US14/535,712 Active 2035-07-04 US10091277B2 (en) | 2013-11-15 | 2014-11-07 | Method, server, client and software for image processing |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/535,712 Active 2035-07-04 US10091277B2 (en) | 2013-11-15 | 2014-11-07 | Method, server, client and software for image processing |
Country Status (4)
Country | Link |
---|---|
US (2) | US20150143421A1 (en) |
EP (1) | EP2874405B1 (en) |
CN (1) | CN104661098B (en) |
GB (1) | GB2520310A (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160378290A1 (en) * | 2015-06-26 | 2016-12-29 | Sharp Kabushiki Kaisha | Content display device, content display method and program |
US10015527B1 (en) * | 2013-12-16 | 2018-07-03 | Amazon Technologies, Inc. | Panoramic video distribution and viewing |
US10873781B2 (en) * | 2017-06-13 | 2020-12-22 | Comcast Cable Communications, Llc | Video fragment file processing |
US11508125B1 (en) * | 2014-05-28 | 2022-11-22 | Lucasfilm Entertainment Company Ltd. | Navigating a virtual environment of a media content item |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
RU2653229C2 (en) * | 2016-03-14 | 2018-05-07 | Валерий Валерьевич Слепичко | Compact autonomous network data storage and viewing device |
Citations (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6192393B1 (en) * | 1998-04-07 | 2001-02-20 | Mgi Software Corporation | Method and system for panorama viewing |
US6331869B1 (en) * | 1998-08-07 | 2001-12-18 | Be Here Corporation | Method and apparatus for electronically distributing motion panoramic images |
US20020021353A1 (en) * | 2000-06-09 | 2002-02-21 | Denies Mark | Streaming panoramic video |
US20040172650A1 (en) * | 2003-02-28 | 2004-09-02 | Hawkins William J. | Targeted content delivery system in an interactive television network |
US20040175059A1 (en) * | 2003-03-07 | 2004-09-09 | Nokia Corporation | Method and device for image surfing |
US20070027949A1 (en) * | 2005-07-21 | 2007-02-01 | Lg Electronics Inc. | Mobile terminal having bookmark function of contents service and operation method thereof |
US20070188653A1 (en) * | 2006-02-13 | 2007-08-16 | Pollock David B | Multi-lens array system and method |
US20090300692A1 (en) * | 2008-06-02 | 2009-12-03 | Mavlankar Aditya A | Systems and methods for video streaming and display |
US20120272269A1 (en) * | 2010-02-06 | 2012-10-25 | Frontier Communications Corporation | Management and delivery of audiovisual content items that correspond to scheduled programs |
US20130031204A1 (en) * | 2011-07-28 | 2013-01-31 | Graham Christoph J | Systems and methods of accelerating delivery of remote content |
US20140059166A1 (en) * | 2012-08-21 | 2014-02-27 | Skybox Imaging, Inc. | Multi-resolution pyramid for georeferenced video |
US20150015789A1 (en) * | 2013-07-09 | 2015-01-15 | Samsung Electronics Co., Ltd. | Method and device for rendering selected portions of video in high resolution |
US9049362B2 (en) * | 2009-11-13 | 2015-06-02 | Samsung Electronics Co., Ltd. | Photographing apparatus and method of providing photographed video |
US20160165309A1 (en) * | 2013-07-29 | 2016-06-09 | Koninklijke Kpn N.V. | Providing tile video streams to a client |
US9449109B1 (en) * | 2004-04-29 | 2016-09-20 | Eversitas, LLC | Visualizing, sharing and monetizing multimedia content |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6043837A (en) * | 1997-05-08 | 2000-03-28 | Be Here Corporation | Method and apparatus for electronically distributing images from a panoptic camera system |
US6532311B1 (en) * | 1999-02-26 | 2003-03-11 | Lockheed Martin Corporation | Image browser |
US6859557B1 (en) * | 2000-07-07 | 2005-02-22 | Microsoft Corp. | System and method for selective decoding and decompression |
US7660352B2 (en) * | 2003-04-04 | 2010-02-09 | Sony Corporation | Apparatus and method of parallel processing an MPEG-4 data stream |
KR101255226B1 (en) * | 2005-09-26 | 2013-04-16 | 한국과학기술원 | Method and Apparatus for defining and reconstructing ROIs in Scalable Video Coding |
US8270469B2 (en) * | 2006-12-15 | 2012-09-18 | Precoad Inc. | Encoding video at multiple resolution levels |
CN102957914B (en) * | 2008-05-23 | 2016-01-06 | 松下知识产权经营株式会社 | Picture decoding apparatus, picture decoding method, picture coding device and method for encoding images |
US20100054578A1 (en) * | 2008-08-26 | 2010-03-04 | Frank Edughom Ekpar | Method and apparatus for interactive visualization and distribution of very large image data sets |
US20100060547A1 (en) * | 2008-09-11 | 2010-03-11 | Sony Ericsson Mobile Communications Ab | Display Device and Method for Displaying Images in a Variable Size Display Area |
JP5067361B2 (en) * | 2008-12-25 | 2012-11-07 | ソニー株式会社 | Encoder and decoder, encoding method and decoding method, and recording medium |
US8463741B2 (en) * | 2009-09-04 | 2013-06-11 | Omnyx, LLC | Digital pathology system |
JPWO2012046435A1 (en) * | 2010-10-04 | 2014-02-24 | パナソニック株式会社 | Image processing apparatus, image encoding method, and image processing method |
US8468262B2 (en) * | 2010-11-01 | 2013-06-18 | Research In Motion Limited | Method and apparatus for updating http content descriptions |
US8515990B2 (en) * | 2010-11-19 | 2013-08-20 | Lg Electronics Inc. | Mobile terminal and method of managing video using metadata therein |
WO2012168365A1 (en) * | 2011-06-08 | 2012-12-13 | Koninklijke Kpn N.V. | Spatially-segmented content delivery |
US9621905B2 (en) * | 2012-06-29 | 2017-04-11 | Qualcomm Incorporated | Tiles and wavefront parallel processing |
GB2512621A (en) * | 2013-04-04 | 2014-10-08 | Sony Corp | A method and apparatus |
-
2013
- 2013-11-15 GB GB1320217.1A patent/GB2520310A/en not_active Withdrawn
-
2014
- 2014-10-21 EP EP14189778.5A patent/EP2874405B1/en active Active
- 2014-10-22 US US14/520,438 patent/US20150143421A1/en not_active Abandoned
- 2014-11-05 CN CN201410636159.2A patent/CN104661098B/en active Active
- 2014-11-07 US US14/535,712 patent/US10091277B2/en active Active
Patent Citations (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6192393B1 (en) * | 1998-04-07 | 2001-02-20 | Mgi Software Corporation | Method and system for panorama viewing |
US6331869B1 (en) * | 1998-08-07 | 2001-12-18 | Be Here Corporation | Method and apparatus for electronically distributing motion panoramic images |
US20020021353A1 (en) * | 2000-06-09 | 2002-02-21 | Denies Mark | Streaming panoramic video |
US20040172650A1 (en) * | 2003-02-28 | 2004-09-02 | Hawkins William J. | Targeted content delivery system in an interactive television network |
US20040175059A1 (en) * | 2003-03-07 | 2004-09-09 | Nokia Corporation | Method and device for image surfing |
US9449109B1 (en) * | 2004-04-29 | 2016-09-20 | Eversitas, LLC | Visualizing, sharing and monetizing multimedia content |
US20070027949A1 (en) * | 2005-07-21 | 2007-02-01 | Lg Electronics Inc. | Mobile terminal having bookmark function of contents service and operation method thereof |
US20070188653A1 (en) * | 2006-02-13 | 2007-08-16 | Pollock David B | Multi-lens array system and method |
US20090300692A1 (en) * | 2008-06-02 | 2009-12-03 | Mavlankar Aditya A | Systems and methods for video streaming and display |
US9049362B2 (en) * | 2009-11-13 | 2015-06-02 | Samsung Electronics Co., Ltd. | Photographing apparatus and method of providing photographed video |
US20120272269A1 (en) * | 2010-02-06 | 2012-10-25 | Frontier Communications Corporation | Management and delivery of audiovisual content items that correspond to scheduled programs |
US20130031204A1 (en) * | 2011-07-28 | 2013-01-31 | Graham Christoph J | Systems and methods of accelerating delivery of remote content |
US20140059166A1 (en) * | 2012-08-21 | 2014-02-27 | Skybox Imaging, Inc. | Multi-resolution pyramid for georeferenced video |
US20150015789A1 (en) * | 2013-07-09 | 2015-01-15 | Samsung Electronics Co., Ltd. | Method and device for rendering selected portions of video in high resolution |
US20160165309A1 (en) * | 2013-07-29 | 2016-06-09 | Koninklijke Kpn N.V. | Providing tile video streams to a client |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10015527B1 (en) * | 2013-12-16 | 2018-07-03 | Amazon Technologies, Inc. | Panoramic video distribution and viewing |
US11508125B1 (en) * | 2014-05-28 | 2022-11-22 | Lucasfilm Entertainment Company Ltd. | Navigating a virtual environment of a media content item |
US20160378290A1 (en) * | 2015-06-26 | 2016-12-29 | Sharp Kabushiki Kaisha | Content display device, content display method and program |
US10620818B2 (en) * | 2015-06-26 | 2020-04-14 | Sharp Kabushiki Kaisha | Content display device, content display method and program |
US11068151B2 (en) | 2015-06-26 | 2021-07-20 | Sharp Kabushiki Kaisha | Content display device, content display method and program |
US10873781B2 (en) * | 2017-06-13 | 2020-12-22 | Comcast Cable Communications, Llc | Video fragment file processing |
US11432038B2 (en) * | 2017-06-13 | 2022-08-30 | Comcast Cable Communications, Llc | Video fragment file processing |
US11743535B2 (en) | 2017-06-13 | 2023-08-29 | Comcast Cable Communications, Llc | Video fragment file processing |
Also Published As
Publication number | Publication date |
---|---|
GB201320217D0 (en) | 2014-01-01 |
US10091277B2 (en) | 2018-10-02 |
EP2874405A3 (en) | 2015-06-10 |
CN104661098B (en) | 2018-12-04 |
EP2874405A2 (en) | 2015-05-20 |
EP2874405B1 (en) | 2020-08-12 |
GB2520310A (en) | 2015-05-20 |
US20150142875A1 (en) | 2015-05-21 |
CN104661098A (en) | 2015-05-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11443511B2 (en) | Systems and methods for presenting supplemental content in augmented reality | |
US11507258B2 (en) | Methods and systems for presenting direction-specific media assets | |
US8918737B2 (en) | Zoom display navigation | |
JP6335448B2 (en) | Information display method for mobile terminal, information providing method for display device, control signal generation method for mobile terminal | |
KR102071579B1 (en) | Method for providing services using screen mirroring and apparatus thereof | |
US10449732B2 (en) | Customized three dimensional (3D) printing of media-related objects | |
CN103299645B (en) | Video player with assisted seek | |
WO2017096817A1 (en) | Method and device for implementing synchronous playing of multi-camera video | |
EP2874405B1 (en) | A client device, system and software for providing a defined view of an image on a client device | |
US20150026718A1 (en) | Systems and methods for displaying a selectable advertisement when video has a background advertisement | |
US20190138194A1 (en) | Apparatus and methods for a user interface | |
US20090300004A1 (en) | Contents display device and contents display method | |
US20160132278A1 (en) | Method, server, client and software | |
US20130174201A1 (en) | Systems and methods for presenting three-dimensional objects in an interactive media guidance application | |
CN109074828A (en) | For providing the system and method for the playlist for the user's related content for replacing ad content to be played back | |
KR20140118604A (en) | Server and method for transmitting augmented reality object to personalized | |
US10061482B1 (en) | Methods, systems, and media for presenting annotations across multiple videos | |
CN112486364A (en) | System and method for media applications including interactive grid displays | |
KR102060902B1 (en) | System for providing video image data based on social network using mobile device and the method thereof | |
KR101805302B1 (en) | Apparatus and method for displaying multimedia contents | |
WO2014193427A1 (en) | Method and system for dynamic discovery of related media assets | |
KR20150065320A (en) | System and method for providing personalized formation for content, and device therefor |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SONY CORPORATION, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WILLIAMS, MICHAEL JOHN;WAGG, DAVID;GILLARD, CLIVE HENRY;AND OTHERS;SIGNING DATES FROM 20141105 TO 20141110;REEL/FRAME:034667/0277 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |