Note: This article is an excerpt from a speech by Yu Jingyi, a professor at the Virtual Reality Center of ShanghaiTech University.

A Light Field Journey To Virtual Reality

Speaker: Yu Jingyi (Shanghai University of Science and Technology)

The Fish Eye Approach

Projecting $P[x , y, z]$ to $[a, b]$

a= x/z< span style="display: inline-block; overflow: hidden; height: 1px; width: 0.003em;"> $a=x/z$

Fisheye distortion:

θ d < /span> = θ (1 + k 1 θ 2 + k 2 < /span> θ 4 + k < span style="display: inline-block; width: 0px; height: 2.563em;"> 3 θ < /span> 6 span> + < span style="position: absolute; clip: rect(1.709em 1000em 2.723em -0.424em); top: -2.557em; left: 0.003em;"> k 4 θ< /span> 6) span> < /nobr>

$\theta_d=\theta(1+k_1\theta^2+k_2\ theta^4+k_3\theta^6+k_4\theta^6)$

Unidistort to $[x^{'}, y^{'}]$

< center>
x′~~=(θ d/r)< span class="mi" id="MathJax-Span-29324" style="font-family: MathJax_Math-italic;">x~~ $x^{'}=(\theta_d/r)x$
$y^{'}=(\theta_d/r)y$

Final pixel coordinate $[u, v]$

$u=f(x^{'}+ay^{'})$
$v=fy^{'}$

First 360 Attempt: QuickTime VR

Assumption: All cameras have an identical CoP!
1991 Eric Chen

Image Warping

Probabilistic model for image match verification

Image Quilting/Stitching

Stereo Parallax

Motion Parallax

360 Stereo Panomara

Near Stereo Pairs

Problem Dense Input

JUMP’s solution

Problems: Stereo Inconsistency
Incoherent Stitching

The Origin of Light Field (LF):

5D Plenoptic Function [Adelson ‘92]

Light Field Acquisitions:

Camera Arrays

Light Field Cameras

Days of Future Past

Light Field Rendering[Isakesen ‘00, Ng ‘05]

Problems With Light Field Rendering

Solution: Lumigraph

Question: How to Obtain Geometry?

Capability 1: Refocusing

refocusing via ray blending

Capability 2: Focal Symmetry

Depth-from-Defocus
Occlusion Detection

Capability 3: Dense Angular Sampling

Surface Cameras (SCam) Images

Scam can be used to separate occluding vs. non occluding re gions

Can also used as a consistency metric

How About Using Multiple LFs

Multi-View Light Field 3D Scanning

Now What? A Leap of Magic! Light Field HMD

Lenticular LF Displays
Compressive LF Displays: Magic Leap/Hololens

How Does It Work?
Solution: Tensor Decomposition

Coclusion: A Light Field Journey to VR

注意：本文摘取自上海科技大学虚拟现实中心教授虞晶怡的演讲，仅做摘录。

A Light Field Journey To Virtual Reality

报告人：虞晶怡(上海科技大学)

The Fish Eye Approach

Projecting $P[x, y, z]$ to $[a, b]$

$a=x/z$
$b=y/z$
$r^2=a^2+b^2$
$\theta=atan(r)$

Fisheye distortion:

< span class="mrow" id="MathJax-Span-29249"> θ d = θ (1 + k 1 θ 2 + k 2 θ 4 + k 3 θ 6 + k 4 θ 6)

$\theta_d=\theta(1+k_1\theta^2+k_2\theta^4+k_3\theta^6+k_4\theta^6)$

Unidistort to $[x^{'}, y^{'}]$

x< /span>′=(θd/r)x $x^{'}=(\theta_d/r)x$

y′ =(θd/r)y $y^{'}=(\theta_d/r)y$

Final pixel coordinate $[u, v]$

u=f(x′+ay′) $u=f(x^{'}+ay^{'})$

v=fy′ $v=fy^{'}$

First 360 Attempt: QuickTime VR

Assumption: All cameras have an identical CoP!
1991 Eric Chen

Image Warping

Probabilistic model for image match verification

Image Quilting/Stitching

Stereo Parallax

Motion Parallax

360 Stereo Panomara

Near Stereo Pairs

Problem Dense Input

JUMP’s solution

Problems: Stereo Inconsistency
Incoherent Stitching

The Origin of Light Field (LF):

5D Plenoptic Function [Adelson ‘92]

Light Field Acquisitions:

Camera Arrays

Light Field Cameras

Days of Future Past

Light Field Rendering[Isakesen ‘00, Ng ‘05]

Problems With Light Field Rendering

Solution: Lumigraph

Question: How to Obtain Geometry?

Capability 1: Refocusing

refocusing via ray blending

Capability 2: Focal Symmetry

Depth-from-Defocus
Occlusion Detection

Capability 3: Dense Angular Sampling

Surface Cameras (SCam) Images

Scam can be used to separate occluding vs. non occluding regions
Can also used as a consistency metric

How About Using Multiple LFs

Multi-View Light F ield 3D Scanning

Now What? A Leap of Magic! Light Field HMD

Lenticular LF Displays
Compressive LF Displays: Magic Leap/Hololens

How Does It Work?
Solution: Tensor Decomposition

Coclusion: A Light Field Journey to VR

θd=θ(1+k1θ2+k2θ4+k3θ6+k4θ6)

Virtual reality light field technology

A Light Field Journey To Virtual Reality

Speaker: Yu Jingyi (Shanghai University of Science and Technology)

The Fish Eye Approach

First 360 Attempt: QuickTime VR

Image Warping

360 Stereo Panomara

JUMP’s solution

The Origin of Light Field (LF):

Light Field Acquisitions:

Light Field Cameras

Solution: Lumigraph

Capability 1: Refocusing

Capability 2: Focal Symmetry

Capability 3: Dense Angular Sampling

A Light Field Journey To Virtual Reality

报告人：虞晶怡(上海科技大学)

The Fish Eye Approach

First 360 Attempt: QuickTime VR

Image Warping

360 Stereo Panomara

JUMP’s solution

The Origin of Light Field (LF):

Light Field Acquisitions:

Light Field Cameras

Solution: Lumigraph

Capability 1: Refocusing

Capability 2: Focal Symmetry

Capability 3: Dense Angular Sampling

Leave a Comment Cancel reply