WO2010024938A3 - View synthesis with heuristic view blending - Google Patents

View synthesis with heuristic view blending Download PDF

Info

Publication number
WO2010024938A3
WO2010024938A3 PCT/US2009/004924 US2009004924W WO2010024938A3 WO 2010024938 A3 WO2010024938 A3 WO 2010024938A3 US 2009004924 W US2009004924 W US 2009004924W WO 2010024938 A3 WO2010024938 A3 WO 2010024938A3
Authority
WO
WIPO (PCT)
Prior art keywords
view
pixel
location
heuristic
blending
Prior art date
Application number
PCT/US2009/004924
Other languages
French (fr)
Other versions
WO2010024938A2 (en
Inventor
Zefeng Ni
Dong Tian
Sitaram Bhagavathy
Joan Llach
Original Assignee
Thomson Licensing
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thomson Licensing filed Critical Thomson Licensing
Priority to CN200980134021.XA priority Critical patent/CN102138333B/en
Priority to US12/737,890 priority patent/US20110157229A1/en
Priority to JP2011525011A priority patent/JP2012501494A/en
Priority to BRPI0916882A priority patent/BRPI0916882A2/en
Priority to EP09806154A priority patent/EP2327224A2/en
Publication of WO2010024938A2 publication Critical patent/WO2010024938A2/en
Publication of WO2010024938A3 publication Critical patent/WO2010024938A3/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/128—Adjusting depth or disparity
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2213/00—Details of stereoscopic systems
    • H04N2213/003—Aspects relating to the "2D+depth" image format
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2213/00—Details of stereoscopic systems
    • H04N2213/005—Aspects relating to the "3D+depth" image format

Abstract

Various implementations are described. Several implementations relate to view synthesis with heuristic view blending for 3D Video (3DV) applications. According to one aspect, at least one reference picture, or a portion thereof, is warped from at least one reference view location to a virtual view location to produce at least one warped reference. A first candidate pixel and a second candidate pixel are identified in the at least one warped reference. The first candidate pixel and the second candidate pixel are candidates for a target pixel location in a virtual picture from the virtual view location. A value for a pixel at the target pixel location is determined based on values of the first and second candidate pixels.
PCT/US2009/004924 2008-08-29 2009-08-28 View synthesis with heuristic view blending WO2010024938A2 (en)

Priority Applications (5)

Application Number Priority Date Filing Date Title
CN200980134021.XA CN102138333B (en) 2008-08-29 2009-08-28 View synthesis with heuristic view blending
US12/737,890 US20110157229A1 (en) 2008-08-29 2009-08-28 View synthesis with heuristic view blending
JP2011525011A JP2012501494A (en) 2008-08-29 2009-08-28 View synthesis with heuristic view blending
BRPI0916882A BRPI0916882A2 (en) 2008-08-29 2009-08-28 visualization synthesis with heuristic visualization mix
EP09806154A EP2327224A2 (en) 2008-08-29 2009-08-28 View synthesis with heuristic view blending

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US9296708P 2008-08-29 2008-08-29
US61/092,967 2008-08-29
US19261208P 2008-09-19 2008-09-19
US61/192,612 2008-09-19

Publications (2)

Publication Number Publication Date
WO2010024938A2 WO2010024938A2 (en) 2010-03-04
WO2010024938A3 true WO2010024938A3 (en) 2010-07-15

Family

ID=41226021

Family Applications (3)

Application Number Title Priority Date Filing Date
PCT/US2009/004924 WO2010024938A2 (en) 2008-08-29 2009-08-28 View synthesis with heuristic view blending
PCT/US2009/004895 WO2010024919A1 (en) 2008-08-29 2009-08-28 View synthesis with boundary-splatting
PCT/US2009/004905 WO2010024925A1 (en) 2008-08-29 2009-08-28 View synthesis with heuristic view merging

Family Applications After (2)

Application Number Title Priority Date Filing Date
PCT/US2009/004895 WO2010024919A1 (en) 2008-08-29 2009-08-28 View synthesis with boundary-splatting
PCT/US2009/004905 WO2010024925A1 (en) 2008-08-29 2009-08-28 View synthesis with heuristic view merging

Country Status (8)

Country Link
US (2) US20110148858A1 (en)
EP (2) EP2321974A1 (en)
JP (2) JP2012501494A (en)
KR (2) KR20110063778A (en)
CN (2) CN102138334A (en)
BR (2) BRPI0916902A2 (en)
TW (2) TWI463864B (en)
WO (3) WO2010024938A2 (en)

Families Citing this family (57)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2008365306B2 (en) * 2008-12-19 2014-02-13 Saab Ab System and method for mixing a scene with a virtual scenario
JP5249114B2 (en) * 2009-04-03 2013-07-31 Ｋｄｄｉ株式会社 Image generating apparatus, method and program
US9124874B2 (en) * 2009-06-05 2015-09-01 Qualcomm Incorporated Encoding of three-dimensional conversion information with two-dimensional video sequence
JP5209121B2 (en) * 2009-09-18 2013-06-12 株式会社東芝 Parallax image generation device
JP2011151773A (en) * 2009-12-21 2011-08-04 Canon Inc Video processing apparatus and control method
TWI434227B (en) * 2009-12-29 2014-04-11 Ind Tech Res Inst Animation generation system and method
CN101895752B (en) * 2010-07-07 2012-12-19 清华大学 Video transmission method, system and device based on visual quality of images
CN101895753B (en) * 2010-07-07 2013-01-16 清华大学 Network congestion degree based video transmission method, system and device
JP5627498B2 (en) * 2010-07-08 2014-11-19 株式会社東芝 Stereo image generating apparatus and method
US8760517B2 (en) * 2010-09-27 2014-06-24 Apple Inc. Polarized images for security
JP5858381B2 (en) * 2010-12-03 2016-02-10 国立大学法人名古屋大学 Multi-viewpoint image composition method and multi-viewpoint image composition system
JP6076260B2 (en) 2010-12-30 2017-02-08 コンパニー ゼネラール デ エタブリッスマン ミシュラン Piezoelectric based system and method for determining tire load
US20120262542A1 (en) * 2011-04-15 2012-10-18 Qualcomm Incorporated Devices and methods for warping and hole filling during view synthesis
US8988558B2 (en) * 2011-04-26 2015-03-24 Omnivision Technologies, Inc. Image overlay in a mobile device
US9536312B2 (en) * 2011-05-16 2017-01-03 Microsoft Corporation Depth reconstruction using plural depth capture units
WO2013012227A2 (en) * 2011-07-15 2013-01-24 엘지전자 주식회사 Method and apparatus for processing a 3d service
US9460551B2 (en) * 2011-08-10 2016-10-04 Telefonaktiebolaget Lm Ericsson (Publ) Method and apparatus for creating a disocclusion map used for coding a three-dimensional video
CN102325259A (en) * 2011-09-09 2012-01-18 青岛海信数字多媒体技术国家重点实验室有限公司 Method and device for synthesizing virtual viewpoints in multi-viewpoint video
CN103828359B (en) * 2011-09-29 2016-06-22 杜比实验室特许公司 For producing the method for the view of scene, coding system and solving code system
FR2982448A1 (en) * 2011-11-07 2013-05-10 Thomson Licensing STEREOSCOPIC IMAGE PROCESSING METHOD COMPRISING AN INCRUSTABLE OBJECT AND CORRESPONDING DEVICE
US9236024B2 (en) 2011-12-06 2016-01-12 Glasses.Com Inc. Systems and methods for obtaining a pupillary distance measurement using a mobile computing device
JP5911166B2 (en) * 2012-01-10 2016-04-27 シャープ株式会社 Image processing apparatus, image processing method, image processing program, imaging apparatus, and image display apparatus
US9313420B2 (en) 2012-01-18 2016-04-12 Intel Corporation Intelligent computational imaging system
TWI478095B (en) 2012-02-07 2015-03-21 Nat Univ Chung Cheng Check the depth of mismatch and compensation depth error of the
US10447990B2 (en) 2012-02-28 2019-10-15 Qualcomm Incorporated Network abstraction layer (NAL) unit header design for three-dimensional video coding
KR101318552B1 (en) * 2012-03-12 2013-10-16 가톨릭대학교 산학협력단 Method for measuring recognition warping about 3d image
CN102663741B (en) * 2012-03-22 2014-09-24 侯克杰 Method for carrying out visual stereo perception enhancement on color digit image and system thereof
US9286715B2 (en) 2012-05-23 2016-03-15 Glasses.Com Inc. Systems and methods for adjusting a virtual try-on
US9311746B2 (en) 2012-05-23 2016-04-12 Glasses.Com Inc. Systems and methods for generating a 3-D model of a virtual try-on product
US9483853B2 (en) 2012-05-23 2016-11-01 Glasses.Com Inc. Systems and methods to display rendered images
CN103716641B (en) * 2012-09-29 2018-11-09 浙江大学 Prognostic chart picture generation method and device
JP6195076B2 (en) * 2012-11-30 2017-09-13 パナソニックＩｐマネジメント株式会社 Different viewpoint image generation apparatus and different viewpoint image generation method
EP2765774A1 (en) 2013-02-06 2014-08-13 Koninklijke Philips N.V. System for generating an intermediate view image
KR102039741B1 (en) * 2013-02-15 2019-11-01 한국전자통신연구원 Method and apparatus for image warping
US9426451B2 (en) * 2013-03-15 2016-08-23 Digimarc Corporation Cooperative photography
CN104065972B (en) * 2013-03-21 2018-09-28 乐金电子(中国)研究开发中心有限公司 A kind of deepness image encoding method, device and encoder
EP2983358A4 (en) * 2013-04-05 2017-01-04 Samsung Electronics Co., Ltd. Interlayer video encoding method and apparatus for using view synthesis prediction, and video decoding method and apparatus for using same
US20140375663A1 (en) * 2013-06-24 2014-12-25 Alexander Pfaffe Interleaved tiled rendering of stereoscopic scenes
US10063851B2 (en) * 2014-04-30 2018-08-28 Intel Corporation System for and method of generating user-selectable novel views on a viewing device
TWI517096B (en) * 2015-01-12 2016-01-11 國立交通大學 Backward depth mapping method for stereoscopic image synthesis
CN104683788B (en) * 2015-03-16 2017-01-04 四川虹微技术有限公司 Gap filling method based on image re-projection
KR20170140187A (en) * 2015-04-23 2017-12-20 오스텐도 테크놀로지스 인코포레이티드 Method for fully parallax compression optical field synthesis using depth information
KR102465969B1 (en) * 2015-06-23 2022-11-10 삼성전자주식회사 Apparatus and method for performing graphics pipeline
US9773302B2 (en) * 2015-10-08 2017-09-26 Hewlett-Packard Development Company, L.P. Three-dimensional object model tagging
CN105488792B (en) * 2015-11-26 2017-11-28 浙江科技学院 Based on dictionary learning and machine learning without referring to stereo image quality evaluation method
EP3496388A1 (en) 2017-12-05 2019-06-12 Thomson Licensing A method and apparatus for encoding a point cloud representing three-dimensional objects
KR102133090B1 (en) * 2018-08-28 2020-07-13 한국과학기술원 Real-Time Reconstruction Method of Spherical 3D 360 Imaging and Apparatus Therefor
US11528461B2 (en) * 2018-11-16 2022-12-13 Electronics And Telecommunications Research Institute Method and apparatus for generating virtual viewpoint image
KR102491674B1 (en) * 2018-11-16 2023-01-26 한국전자통신연구원 Method and apparatus for generating virtual viewpoint image
US11393113B2 (en) 2019-02-28 2022-07-19 Dolby Laboratories Licensing Corporation Hole filling for depth image based rendering
US11670039B2 (en) 2019-03-04 2023-06-06 Dolby Laboratories Licensing Corporation Temporal hole filling for depth image based video rendering
KR102192347B1 (en) * 2019-03-12 2020-12-17 한국과학기술원 Real-Time Reconstruction Method of Polyhedron Based 360 Imaging and Apparatus Therefor
WO2020200235A1 (en) 2019-04-01 2020-10-08 Beijing Bytedance Network Technology Co., Ltd. Half-pel interpolation filter in intra block copy coding mode
US10930054B2 (en) * 2019-06-18 2021-02-23 Intel Corporation Method and system of robust virtual view generation between camera views
CN112291549B (en) * 2020-09-23 2021-07-09 广西壮族自治区地图院 Method for acquiring stereoscopic sequence frame images of raster topographic map based on DEM
US11570418B2 (en) 2021-06-17 2023-01-31 Creal Sa Techniques for generating light field data by combining multiple synthesized viewpoints
KR20230103198A (en) * 2021-12-31 2023-07-07 주식회사 쓰리아이 Texturing method for generating 3D virtual model and computing device therefor

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3826236B2 (en) * 1995-05-08 2006-09-27 松下電器産業株式会社 Intermediate image generation method, intermediate image generation device, parallax estimation method, and image transmission display device
JP3769850B2 (en) * 1996-12-26 2006-04-26 松下電器産業株式会社 Intermediate viewpoint image generation method, parallax estimation method, and image transmission method
WO2001063561A1 (en) * 2000-02-25 2001-08-30 The Research Foundation Of State University Of New York Apparatus and method for volume processing and rendering
US7079157B2 (en) * 2000-03-17 2006-07-18 Sun Microsystems, Inc. Matching the edges of multiple overlapping screen images
US7085409B2 (en) * 2000-10-18 2006-08-01 Sarnoff Corporation Method and apparatus for synthesizing new video and/or still imagery from a collection of real video and/or still imagery
JP2004537082A (en) * 2001-01-26 2004-12-09 ザクセル システムズ インコーポレイテッド Real-time virtual viewpoint in virtual reality environment
US6965379B2 (en) * 2001-05-08 2005-11-15 Koninklijke Philips Electronics N.V. N-view synthesis from monocular video of certain broadcast and stored mass media content
US7003136B1 (en) * 2002-04-26 2006-02-21 Hewlett-Packard Development Company, L.P. Plan-view projections of depth image data for object tracking
US7348963B2 (en) * 2002-05-28 2008-03-25 Reactrix Systems, Inc. Interactive video display system
EP1542167A1 (en) * 2003-12-09 2005-06-15 Koninklijke Philips Electronics N.V. Computer graphics processor and method for rendering 3D scenes on a 3D image display screen
US7292257B2 (en) * 2004-06-28 2007-11-06 Microsoft Corporation Interactive viewpoint video system and process
US7364306B2 (en) * 2005-06-20 2008-04-29 Digital Display Innovations, Llc Field sequential light source modulation for a digital display system
US8384763B2 (en) * 2005-07-26 2013-02-26 Her Majesty the Queen in right of Canada as represented by the Minster of Industry, Through the Communications Research Centre Canada Generating a depth map from a two-dimensional source image for stereoscopic and multiview imaging
US7471292B2 (en) * 2005-11-15 2008-12-30 Sharp Laboratories Of America, Inc. Virtual view specification and synthesis in free viewpoint

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
CHEN S E ET AL: "View interpolation for image synthesis", COMPUTER GRAPHICS PROCEEDINGS, PROCEEDINGS OF SIGGRAPH ANNUALINTERNATIONAL CONFERENCE ON COMPUTER GRAPHICS AND INTERACTIVETECHNIQUES, XX, XX, 1 August 1993 (1993-08-01), pages 279 - 288, XP002269084 *
COOKE E ET AL: "Multi-view synthesis: A novel view creation approach for free viewpoint video", SIGNAL PROCESSING. IMAGE COMMUNICATION, ELSEVIER SCIENCE PUBLISHERS, AMSTERDAM, NL, vol. 21, no. 6, 1 July 2006 (2006-07-01), pages 476 - 492, XP025080518, ISSN: 0923-5965, [retrieved on 20060701] *
MARK W R ET AL: "POST-RENDERING 3D WARPING", PROCEEDINGS OF 1997 SYMPOSIUM ON INTERACTIVE 3 D GRAPHICS 27-30 APRIL 1997 PROVIDENCE, RI, USA; [PROCEEDINGS OF THE SYMPOSIUM ON INTERACTIVE 3D GRAPHICS], PROCEEDINGS 1997 SYMPOSIUM ON INTERACTIVE 3D GRAPHICS ACM NEW YORK, NY, USA, 27 April 1997 (1997-04-27), pages 7 - 16, XP000725355, ISBN: 978-0-89791-884-8 *
VOICU POPESCU: "Forward Rasterization: A reconstruction algorithm for image based rendering", no. TR01-019, 29 May 2001 (2001-05-29), Department of computer science, Chapel Hill, XP002574108, Retrieved from the Internet <URL:http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.75.9117> [retrieved on 20100319] *
Z NI, D TIAN, S BHAGAVATHY, J LLACH, BS MANJUNATH: "Improving the quality of depth image based rendering for 3D video systems", 7 November 2009 (2009-11-07), pages 513 - 516, XP002574109, Retrieved from the Internet <URL:ftp://ftp.elet.polimi.it/users/Stefano.Tubaro/ICIP_USB_Proceedings_v2/pdfs/0000513.pdf> [retrieved on 20100319] *

Also Published As

Publication number Publication date
KR20110063778A (en) 2011-06-14
CN102138333A (en) 2011-07-27
WO2010024938A2 (en) 2010-03-04
US20110157229A1 (en) 2011-06-30
CN102138333B (en) 2014-09-24
TWI463864B (en) 2014-12-01
BRPI0916902A2 (en) 2015-11-24
BRPI0916882A2 (en) 2016-02-10
KR20110073474A (en) 2011-06-29
TW201023618A (en) 2010-06-16
CN102138334A (en) 2011-07-27
WO2010024925A1 (en) 2010-03-04
JP5551166B2 (en) 2014-07-16
US20110148858A1 (en) 2011-06-23
JP2012501494A (en) 2012-01-19
EP2327224A2 (en) 2011-06-01
EP2321974A1 (en) 2011-05-18
WO2010024919A1 (en) 2010-03-04
JP2012501580A (en) 2012-01-19
TW201029442A (en) 2010-08-01

Similar Documents

Publication Publication Date Title
WO2010024938A3 (en) View synthesis with heuristic view blending
WO2008012808A3 (en) Interactive segmentation of images with single scribbles
WO2011108827A3 (en) Pointing device of augmented reality
WO2006105054A3 (en) Method and system for improving video metadata through the use of frame-to-frame correspondences
WO2010141120A3 (en) System and method for generating three dimensional images using lidar and video measurements
DE602008005312D1 (en) NON-PHOTOREALISTIC PLAYBACK OF EXTENDED REALITY
WO2008068707A3 (en) Image processing system for processing combined image data and depth data
GB2552279A (en) Combining a high resolution narrow field display and a mid resolution wide field display
WO2009131703A3 (en) Coding of depth signal
WO2008091207A8 (en) Border region processing in images
WO2007113725A3 (en) Efficient encoding of multiple views
WO2008066699A3 (en) Processing images having color and panchromatic pixels
WO2012019973A3 (en) Composition of a digital image for display on a transparent screen
WO2012010220A3 (en) Filling disocclusions in a virtual view
ATE540732T1 (en) STEREO VIDEO FOR GAMES
WO2008076079A8 (en) Methods and apparatuses for cursor control in image guided surgery
WO2008121272A3 (en) Edge mapping incorporating panchromatic pixels
WO2008147561A3 (en) Rendering, viewing and annotating panoramic images, and applications thereof
WO2012004682A3 (en) Scene background blurring including range measurement
WO2011011353A3 (en) Stereoscopic form reader
WO2009089125A3 (en) Navigating among images of an object in 3d space
WO2010095074A9 (en) Combining 3d image and graphical data
WO2008121280A3 (en) Edge mapping using panchromatic pixels
WO2013006115A3 (en) Depth or disparity map upscaling
EP2051533A3 (en) 3D image rendering apparatus and method

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200980134021.X

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09806154

Country of ref document: EP

Kind code of ref document: A2

WWE Wipo information: entry into national phase

Ref document number: 2009806154

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2011525011

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 20117006916

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: PI0916882

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20110228