U.S. patent application number 14/505117 was filed with the patent office on 2015-04-02 for method, device and system for packing color frame and original depth frame.
The applicant listed for this patent is NATIONAL CHENG KUNG UNIVERSITY. Invention is credited to Ke-Ying LIAO, Hsi-Chun TSENG, Hung-Ming WANG, Jar-Ferr YANG.
Application Number | 20150092029 14/505117 |
Document ID | / |
Family ID | 51743265 |
Filed Date | 2015-04-02 |
United States Patent
Application |
20150092029 |
Kind Code |
A1 |
YANG; Jar-Ferr ; et
al. |
April 2, 2015 |
METHOD, DEVICE AND SYSTEM FOR PACKING COLOR FRAME AND ORIGINAL
DEPTH FRAME
Abstract
A method, device and system for packing a color frame and an
original depth frame to obtain a packed frame are disclosed. The
color frame is corresponding to the original depth frame, and the
packed frame is to be displayed on a screen. The method for packing
the color frame and the original depth frame includes the steps of
resizing the original depth frame to obtain a resized depth frame;
and combining the color frame and the resized depth frame to obtain
the packed frame. The center of the color frame is displayed in the
central area of the screen.
Inventors: |
YANG; Jar-Ferr; (Tainan
City, TW) ; WANG; Hung-Ming; (Tainan City, TW)
; TSENG; Hsi-Chun; (Zhongli City, TW) ; LIAO;
Ke-Ying; (Taipei City, TW) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
NATIONAL CHENG KUNG UNIVERSITY |
Tainan City |
|
TW |
|
|
Family ID: |
51743265 |
Appl. No.: |
14/505117 |
Filed: |
October 2, 2014 |
Current U.S.
Class: |
348/56 |
Current CPC
Class: |
H04N 13/161 20180501;
H04N 13/139 20180501; H04N 13/194 20180501 |
Class at
Publication: |
348/56 |
International
Class: |
H04N 13/04 20060101
H04N013/04 |
Foreign Application Data
Date |
Code |
Application Number |
Oct 2, 2013 |
TW |
102135714 |
Claims
1. A method for packing a color frame and an original depth frame
to obtain a packed frame, wherein the color frame is corresponding
to the original depth frame, and the packed frame is to be
displayed on a screen, the method comprising steps of: resizing the
original depth frame to obtain a resized depth frame; and combining
the color frame and the resized depth frame to obtain the packed
frame, wherein a center of the color frame is displayed in a
central area of the screen.
2. The method of claim 1, wherein a center point of the color frame
is overlapped with a center point of the screen.
3. The method of claim 1, further comprising: resizing the original
depth frame to obtain an intermediate depth frame; and rearranging
sub-pixels of the intermediate depth frame to obtain the resized
depth frame.
4. The method of claim 3, wherein the size of the resized depth
frame is 1/3 of the size of the intermediate depth frame along a
column direction.
5. The method of claim 3, wherein the resized depth frame has a
plurality of pixels, each of the pixels has three sub-pixels, and
the sub-pixels correspond to sub-pixel values of three pixels of
the intermediate depth frame.
6. The method of claim 1, wherein the step of combining the color
frame and the resized depth frame comprises: dividing the resized
depth frame into two parts; and combining the two parts at a top
side and a bottom side of the color frame, respectively.
7. The method of claim 6, wherein the step of combining the two
parts at the top side and the bottom side of the color frame,
respectively, comprises: flipping the two parts with respect to the
resized depth frame.
8. The method of claim 1, wherein the step of combining the color
frame and the resized depth frame comprises: dividing the resized
depth frame into two parts; and combining the two parts at a left
side and a right side of the color frame, respectively.
9. The method of claim 8, wherein the step of combining the two
parts at the left side and the right side of the color frame,
respectively, comprises: flipping the two parts with respect to the
resized depth frame.
10. The method of claim 3, further comprising: resizing an original
color frame to obtain the color frame.
11. The method of claim 10, wherein the original color frame has a
preset frame size, and the size of the packed frame is the same as
the preset frame size.
12. The method of claim 10, wherein the size of the color frame is
downscaled to A times of the size of the original color frame along
a direction, and the size of the intermediate depth frame is
downscaled to B times of the size of the original depth frame along
the same direction.
13. The method of claim 12, wherein A and B fit the equation of
A+B/3=1.
14. The method of claim 12, wherein A and B are 3/4,
respectively.
15. A device for packing a color frame and an original depth frame
to obtain a packed frame, wherein the color frame is corresponding
to the original depth frame, and the packed frame is to be
displayed on a screen, the device comprising: a size processing
unit for resizing the original depth frame to obtain a resized
depth frame; and a combine processing unit for combining the color
frame and the resized depth frame to obtain the packed frame,
wherein a center of the color frame is displayed in a central area
of the screen.
16. The device of claim 15, wherein a center point of the color
frame is overlapped with a center point of the screen.
17. The device of claim 15, wherein the size processing unit
resizes the original depth frame to obtain an intermediate depth
frame, and rearranges sub-pixels of the intermediate depth frame to
obtain the resized depth frame.
18. The device of claim 17, wherein the size of the resized depth
frame is 1/3 of the size of the intermediate depth frame along a
column direction.
19. The device of claim 17, wherein the resized depth frame has a
plurality of pixels, each of the pixels has three sub-pixels, and
the sub-pixels correspond to sub-pixel values of three pixels of
the intermediate depth frame.
20. The device of claim 15, further comprising: a split processing
unit for dividing the resized depth frame into two parts, wherein
the combine processing unit further combines the two parts at a top
side and a bottom side of the color frame, respectively, to obtain
the packed frame.
21. The device of claim 20, further comprising: a flip processing
unit for flipping the two parts of the resized depth frame before
combining the two parts at the top side and the bottom side of the
color frame, respectively.
22. The device of claim 15, further comprising: a split processing
unit for dividing the resized depth frame into two parts, wherein
the combine processing unit further combines the two parts at a
left side and a right side of the color frame, respectively, to
obtain the packed frame.
23. The device of claim 22, further comprising: a flip processing
unit for flipping the two parts of the resized depth frame before
combining the two parts at the left side and the right side of the
color frame, respectively.
24. The device of claim 17, wherein the size processing unit
further resizes an original color frame to obtain the color
frame.
25. The device of claim 24, wherein the original color frame has a
preset frame size, and the size of the packed frame is the same as
the preset frame size.
26. The device of claim 24, wherein the size of the color frame is
downscaled to A times of the size of the original color frame along
a direction, and the size of the intermediate depth frame is
downscaled to B times of the size of the original depth frame along
the direction.
27. The device of claim 26, wherein A and B fit the equation of
A+B/3=1.
28. The device of claim 26, wherein A and B are 3/4,
respectively.
29. A system for packing a color frame and an original depth frame
to obtain a packed frame, wherein the color frame is corresponding
to the original depth frame, and the packed frame is to be
displayed on a screen, the system comprising: a memory unit for
storing the color frame and the original depth frame; and a
processing unit for resizing the original depth frame to obtain a
resized depth frame, and combining the color frame and the resized
depth frame to obtain the packed frame, wherein a center of the
color frame is displayed in a central area of the screen.
30. The system of claim 29, wherein a center point of the color
frame is overlapped with a center point of the screen.
31. The system of claim 29, wherein the processing unit resizes the
original depth frame to obtain an intermediate depth frame, and
rearranges sub-pixels of the intermediate depth frame to obtain the
resized depth frame.
32. The system of claim 31, wherein the size of the resized depth
frame is 1/3 of the size of the intermediate depth frame along a
column direction.
33. The system of claim 31, wherein the resized depth frame has a
plurality of pixels, each of the pixels has three sub-pixels, and
the sub-pixels correspond to sub-pixel values of three pixels of
the intermediate depth frame.
34. The system of claim 29, wherein the processing unit further
splits the resized depth frame into two parts, and combines the two
parts at a top side and a bottom side of the color frame,
respectively, to obtain the packed frame.
35. The system of claim 34, wherein the processing unit further
flips the two parts of the resized depth frame before combining the
two parts at the top side and the bottom side of the color frame,
respectively.
36. The system of claim 29, wherein the processing unit further
splits the resized depth frame into two parts, and combines the two
parts at a left side and a right side of the color frame,
respectively, to obtain the packed frame.
37. The system of claim 36, wherein the processing unit further
flips the two parts of the resized depth frame before combining the
two parts at the left side and the right side of the color frame,
respectively.
38. The system of claim 31, wherein the processing unit further
resizes an original color frame to obtain the color frame.
39. The system of claim 38, wherein the original color frame has a
preset frame size, and the size of the packed frame is the same as
the preset frame size.
40. The system of claim 38, wherein the size of the color frame is
downscaled to A times of the size of the original color frame along
a direction, and the size of the intermediate depth frame is
downscaled to B times of the size of the original depth frame along
the direction.
41. The system of claim 40, wherein A and B fit the equation of
A+B/3=1.
42. The system of claim 40, wherein A and B are 3/4, respectively.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This Non-provisional application claims priority under 35
U.S.C. .sctn.119(a) on Patent Application No(s). 102135714 filed in
Taiwan, Republic of China on Oct. 2, 2013, the entire contents of
which are hereby incorporated by reference.
BACKGROUND OF THE INVENTION
[0002] 1. Field of Invention
[0003] The present invention relates to a method, device and system
for packing a color frame and an original depth frame to obtain a
packed frame.
[0004] 2. Related Art
[0005] As the progress of technologies, the performance and quality
of electronic products have become the prior consideration of
electronic product development. For example, the basic requirement
for display devices is to provide a high-definition and more
comfortable images. Except for the basic requirement, the
manufacturer further develops the 3D display device for providing
multi-view angle 3D images as well as a new visual experience to
the users.
[0006] In general, a 3D image generating system can analyze the
original 2D image data (e.g. an original color frame) so as to
produce an original depth frame corresponding to the original color
frame, and then transmit the original color frame and the original
depth frame to a 3D display device of the user through the
conventional transmission apparatus. After receiving the original
color frame and the original depth frame, the 3D display device can
process the frames by depth-image-based rendering (DIBR) to produce
two or more multi-view angle images for any naked-eye 3D display
device or to produce a left-eye image and a right-eye image for a
glasses-type 3D display device.
[0007] However, the general televisions for family use are majorly
the 2D display devices, which are not equipped with the function of
producing the multi-view angle images or the left-eye and right-eye
images. As a result, after receiving the 3D image signals, the 2D
display device will display the combined left-eye and right-eye
images or the single-view angle and single depth images side by
side. In fact, there are more and more available 3D images, so it
will be very common to execute the 3D image signals by the 2D
display devices. Unfortunately, the 2D display devices can only
display 2D images (color frame), which are not smooth and will make
the viewer very uncomfortable.
SUMMARY OF THE INVENTION
[0008] An objective of the present invention is to provide a
method, a device and a system for packing a color frame and an
original depth frame to obtain a packed frame, which can be
directly displayed by a 2D display device, while the displayed 2D
images are smoothly shown in the central area of a screen and will
not make the viewer uncomfortable.
[0009] To achieve the above objective, the present invention
discloses a method for packing a color frame and an original depth
frame to obtain a packed frame. The color frame is corresponding to
the original depth frame, and the packed frame is to be displayed
on a screen. The method includes the following steps of: resizing
the original depth frame to obtain a resized depth frame; and
combining the color frame and the resized depth frame to obtain the
packed frame. A center of the color frame is displayed in a central
area of the screen.
[0010] To achieve the above objective, the present invention also
discloses a device for packing a color frame and an original depth
frame to obtain a packed frame, which includes a size processing
unit and a combine processing unit. The color frame is
corresponding to the original depth frame, and the packed frame is
to be displayed on a screen. The size processing unit resizes the
original depth frame to obtain a resized depth frame. The combine
processing unit combines the color frame and the resized depth
frame to obtain the packed frame. A center of the color frame is
displayed in a central area of the screen.
[0011] To achieve the above objective, the present invention
further discloses a system for packing a color frame and an
original depth frame to obtain a packed frame, which includes a
memory unit and a processing unit. The color frame is corresponding
to the original depth frame, and the packed frame is to be
displayed on a screen. The memory unit stores the color frame and
the original depth frame. The processing unit resizes the original
depth frame to obtain a resized depth frame, and combines the color
frame and the resized depth frame to obtain the packed frame. A
center of the color frame is displayed in a central area of the
screen.
[0012] As mentioned above, in the method, device and system of the
invention for packing a color frame and an original depth frame to
obtain a packed frame, the color frame and resized depth frame are
combined to obtain a packed frame with a center displayed at the
central area of the screen. Afterward, the packed frame can be
restored to obtain the original color frame or the original depth
frame, which can be further processed by the 3D display device
according to the depth image-based rendering (DIBR) so as to
produce correct naked-eye multi-view angle 3D images or glasses
type dual-view angle 3D images. Most important, since the center of
the color frame is displayed at the central area of the screen, and
the center point of the color frame and the center point of the
screen are preferably overlapped and most preferably totally
overlapped, the packed frame can be directly displayed on the
screen of a 2D display device. Moreover, the displayed 2D images
(color frames) are smoothly shown in the central of a screen and
will not make the viewer uncomfortable.
[0013] In addition, the method, device and system of the invention
also resize the original depth frame to obtain the resized depth
frame and combine the color frame and the resized depth frame to
obtain a simplified packed frame. Accordingly, the technology of
the invention can be cooperated with other video compression system
to enhance the compression ratio, thereby decreasing the bandwidth
requirement for 3D video. This feature is further benefit to the
promotion of 3D channels.
BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The invention will become more fully understood from the
detailed description and accompanying drawings, which are given for
illustration only, and thus are not limitative of the present
invention, and wherein:
[0015] FIG. 1A is a flow chart of a method for packing a color
frame and an original depth frame according to a preferred
embodiment of the invention;
[0016] FIG. 1B is a functional block diagram of a device for
packing a color frame and an original depth frame according to a
preferred embodiment of the invention;
[0017] FIGS. 2A to 2D are schematic diagrams showing the procedure
for processing an original depth frame of a first embodiment;
[0018] FIGS. 3A and 3B are schematic diagrams showing the procedure
for processing an original color frame of the first embodiment;
[0019] FIG. 4A is a schematic diagram showing a packed frame of the
first embodiment;
[0020] FIG. 4B is a schematic diagram showing the color frame and a
screen as a 2D display device displays the packed frame of the
first embodiment;
[0021] FIG. 4C is a schematic diagram showing an image of another
packed frame obtained by the method and device for packing a color
frame and an original depth frame;
[0022] FIG. 4D is a schematic diagram showing the central area of
the screen according to the embodiment of the invention;
[0023] FIGS. 5A to 5D are schematic diagrams showing the procedure
for processing an original depth frame of a second embodiment;
[0024] FIGS. 6A and 6B are schematic diagrams showing the procedure
for processing an original color frame of the second
embodiment;
[0025] FIG. 7A is a schematic diagram showing a packed frame of the
second embodiment;
[0026] FIG. 7B is a schematic diagram showing the color frame and a
screen as a 2D display device displays the packed frame of the
second embodiment;
[0027] FIG. 7C is a schematic diagram showing an image of another
packed frame obtained by the method and device for packing a color
frame and an original depth frame;
[0028] FIG. 8 is a functional block diagram of a device for packing
a color frame and an original depth frame of another aspect
according to a preferred embodiment of the invention;
[0029] FIGS. 9A to 9D are schematic diagrams showing the procedure
for processing an original depth frame of a third embodiment;
[0030] FIGS. 10A and 10B are schematic diagrams showing the
procedure for processing an original color frame of the third
embodiment;
[0031] FIG. 11A is a schematic diagram showing a packed frame of
the third embodiment;
[0032] FIG. 11B is a schematic diagram showing the color frame and
a screen as a 2D display device displays the packed frame of the
third embodiment;
[0033] FIG. 11C is a schematic diagram showing an image of another
packed frame obtained by the method and device for packing a color
frame and an original depth frame; and
[0034] FIG. 12 is a functional block diagram of a system for
packing a color frame and an original depth frame according to a
preferred embodiment of the invention.
DETAILED DESCRIPTION OF THE INVENTION
[0035] The present invention will be apparent from the following
detailed description, which proceeds with reference to the
accompanying drawings, wherein the same references relate to the
same elements.
[0036] FIG. 1A is a flow chart of a method for packing a color
frame and an original depth frame according to a preferred
embodiment of the invention, and FIG. 1B is a functional block
diagram of a device 1 for packing a color frame and an original
depth frame according to a preferred embodiment of the invention.
The obtained packed frame according to the invention can be
directly displayed in a screen, and especially a screen of a 2D
display device. The resolution of 2D display device is, for example
but not limited to, 1920.times.1080 pixels.
[0037] As shown in FIG. 1A, the method for packing a color frame
and an original depth frame to obtain a packed frame includes steps
S01 and S02. As shown in FIG. 1B, the device 1 includes a size
processing unit 11 and a combine processing unit 12. Moreover, the
device 1 further includes a split processing unit 13 and a flip
processing unit 14. The method and device 1 for packing a color
frame and an original depth frame to obtain a packed frame will be
described hereinafter with reference to FIGS. 2A to 4B.
[0038] FIGS. 2A to 2D are schematic diagrams showing the procedure
for processing an original depth frame of a first embodiment. FIG.
2A shows an original depth frame, FIG. 2B shows an intermediate
depth frame, FIG. 2C shows a resized depth frame, and FIG. 2D shows
two parts of the resized depth frame after being split and flipped.
In addition, FIGS. 3A and 3B are schematic diagrams showing the
procedure for processing an original color frame of the first
embodiment. FIG. 3A shows an original color frame, and FIG. 3B
shows a color frame produced by resizing the original color frame
of FIG. 3A.
[0039] In this embodiment, the color frame is obtained by resizing
the original color frame. The original color frame has a preset
frame size such as, for example but not limited to, 1920.times.1080
pixels, which is the same as the resolution of the HD 2D display
device. In this case, the sizes of the original depth frame and the
original color frame are the same and equal to the preset frame
size. Of course, in other aspects, the preset frame size may have
different values. In addition, the figures show a row direction D1
and a column direction D2. The row direction D1 is a horizontal
direction of the displayed frame image, and the column direction D2
is a vertical direction of the display frame image. To be noted,
although the shown images in FIGS. 3A and 3B are black-and-white
images, they in fact can be color images in a color display
device.
[0040] The original depth frame is corresponding to the original
color frame. That is, the original depth frame and the original
color frame have the same size or resolution. Besides, the original
depth frame is produced according to the depth values of all
objects in the original color frame. The original depth frame and
the original color frame can synthesize according to DIBR so as to
produce a 3D image to be displayed by a 3D display device. Since
the original depth frame is corresponding to the original color
frame, the original depth frame is also corresponding to the color
frame.
[0041] As shown in FIG. 1A, the step S01 of this method is to
resize the original depth frame to obtain a resized depth frame
(from FIG. 2A to FIG. 2B). Herein, the original depth frame of FIG.
2A is a gray-level frame, so all sub-pixels in one pixel have the
same sub-pixel value (the same gray-level value). Each pixel may
have 2 or more sub-pixels. In this embodiment, each pixel has 3
sub-pixels R, G, B for example.
[0042] In step S01, the original depth frame (FIG. 2A) is resized
to obtain an intermediate depth frame (FIG. 2B), and then the
sub-pixels of the intermediate depth frame is rearranged to obtain
the resized depth frame (FIG. 2C). The step of resizing the
original depth frame to obtain the intermediate depth frame is
achieved by downscaling the size of the original depth frame or to
reducing the resolution of the original depth frame. In this
embodiment, the size processing unit 11 can downscale the original
depth frame of FIG. 2A so as to obtain the intermediate depth frame
of FIG. 2B. Herein, the size of the intermediate depth frame along
the column direction D2 is downscaled to 3/4 of the size of the
original depth frame. As a result, the size of the intermediate
depth frame is 1920.times.810 (810=1080.times.3/4).
[0043] Then, the sub-pixels are rearranged. The sub-pixel
rearrangement is to retrieve two sub-pixel values from three pixels
of the intermediate depth frame, and then to correspondingly store
the sub-pixel values in three sub-pixels of one pixel of the
resized depth frame. Herein, since the original depth frame is a
gray-level frame, all three sub-pixel values of one pixel are the
same, any sub-pixel value in one pixel can be a representative
value.
[0044] The sub-pixel rearrangement is to retrieve the sub-pixel
values of three pixels of the intermediate depth frame and then
store the sub-pixel values into three sub-pixels of one pixel of
the resized depth frame. In more detailed, the sub-pixel value of a
first pixel of the intermediate depth frame is stored in a first
sub-pixel of a first pixel of the resized depth frame, the
sub-pixel value of a second pixel of the intermediate depth frame
is stored in a second sub-pixel of the first pixel of the resized
depth frame, and the sub-pixel value of a third pixel of the
intermediate depth frame is stored in a third sub-pixel of the
first pixel of the resized depth frame. This rule can be applied to
other pixels and sub-pixels. Accordingly, the intermediate depth
frame of FIG. 2B is downscaled to 1/3 of the original one along the
column direction D2, and the resized depth frame of FIG. 2C has a
resolution of 1920.times.270 (270=810.times.1/3).
[0045] Accordingly, the size of the resized depth frame of FIG. 2C
is 1/4 (3/4.times.1/3) of the original depth frame along the column
direction D2. Since the sub-pixel rearrangement technology is used,
the gray-level values of three columns of pixels in the
intermediate depth frame can be stored in three sub-pixels of one
pixel in the resized depth frame. Compared to the original depth
frame, the resized depth frame has higher coding efficiency and
less data amount, so that it is possible to improve the
transmission efficiency during data transmission.
[0046] The sub-pixel rearrangement can make the frame be
downscaled, so this technology is also called a sub-pixel packing
rearrangement.
[0047] The step of resizing the original color frame to obtain the
color frame is to downscale the size of the original color frame or
to reduce the resolution of the original color frame. In this
embodiment, the size processing unit 11 downscales the original
color frame of FIG. 3A to obtain the color frame of FIG. 3B.
Herein, the size of the color frame along the column direction D2
is downscaled to 3/4 of the size of the original color frame. In
other words, the size of the original color frame of FIG. 3A along
the column direction D2 is downscaled by 3/4 to obtain the color
frame of FIG. 3B. As a result, the resolution of the color frame is
1920.times.810 (810=1080.times.3/4). Of course, in other
embodiments, the size of the original color frame can be downscaled
by other proportions or it can be downscaled along the row
direction D1 or along the row direction D1 and the column direction
D2. This invention is not limited.
[0048] After the color frame and the resized depth frame are
obtained, the step S02 is performed to combine the color frame and
the resized depth frame to obtain a packed frame. Before the step
S02 of combining the color frame and the resized depth frame by the
combine processing unit 12, a split processing unit 13 splits the
resized depth frame into two parts (splitting along the row
direction D1), and a flip processing unit 14 flips the two parts so
as to obtain two parts of FIG. 2D (the size of each part is
1920.times.135). Afterwards, the combine processing unit 12
combines the two flipped parts at a top side and a bottom side of
the color frame, respectively, to obtain the packed frame of FIG.
4A. Herein, "flipping" is to turn over by 180 degrees and make the
part upside down. The sizes of the two parts split from the resized
depth frame are the same.
[0049] FIG. 4B is a schematic diagram showing the color frame and a
screen as a 2D display device displays the packed frame of the
first embodiment.
[0050] The packed frame is restored to obtain the original color
frame or the original depth frame, which can be further processed
by the 3D display device according to the depth image-based
rendering (DIBR) so as to produce correct naked-eye multi-view
angle 3D images or glasses type dual-view angle 3D images. As shown
in FIG. 4B, the packed frame can be directly displayed on the
screen of the 2D display device. Herein, the center point of the
color frame (the cross point of the diagonal lines of the color
frame) and the center point of the screen (the cross point of the
diagonal lines of the screen), which are all the center point O,
are overlapped. Accordingly, the packed frame can be directly
displayed on the screen of the 2D display device. As shown in FIG.
4A, the color frame (2D image) viewed by eyes is very smooth and
will not make the viewer feel uncomfortable. Of course, in order to
avoid the uncomfortable feeling, it is discovered that to display
the center of the color frame at the central area of the screen can
provide satisfied effect. In this case, the central area includes
the center point and several pixels surrounding the center point.
The center area can be, for example, a circular or square area. For
example, FIG. 4C shows a central area of the screen as the 2D
display device displays the packed frame. In FIG. 4D, the central
area CA of the screen includes the center point O of the screen and
the area surrounding the center point O (about 1/3 of the screen).
When the center of the color frame is located within this area, the
viewer will not feel uncomfortable.
[0051] To be noted, in the first embodiment, the size of the packed
frame of FIG. 4A and the preset frame size of the original color
frame of FIG. 3A are the same and are both 1920.times.1080
(1080=135+810+135). In the conventional art, the original color
frame and the original depth frame are arranged by single-view
angle and single depth arrangement from left to right, so that the
resolution of the arranged image become twice of the original data,
which can cause additional load in transmission bandwidth.
According to the method and device 1 for packing the color frame
and the original depth frame to obtain the packed frame of this
invention, the sizes of the packed frame and the original color
frame (or the original depth frame) are the same (all are
1920.times.1080). In addition, the method and device 1 for packing
the color frame and the original depth frame to obtain the packed
frame can be cooperated with any video compression method to
achieve a better compression efficiency. Accordingly, the packed
frame will not cause additional load for the transmission bandwidth
during data transmission, and it will not cause the overloading of
the 3D display device in the user end due to the exceeded
decompression calculations.
[0052] FIGS. 5A to 5D are schematic diagrams showing the procedure
for processing an original depth frame of a second embodiment. FIG.
5A shows an original depth frame, FIG. 5B shows an intermediate
depth frame, FIG. 5C shows a resized depth frame, and FIG. 5D shows
two parts of the resized depth frame after being split and flipped.
In addition, FIGS. 6A and 6B are schematic diagrams showing the
procedure for processing an original color frame of the second
embodiment. FIG. 6A shows an original color frame, and FIG. 6B
shows a color frame produced by resizing the original color frame
of FIG. 6A.
[0053] Different from the procedure of FIGS. 2A to 2D, the
procedure of FIGS. 5A to 5D remains the height (1080) of the frame
and resizes the frame along the row direction D1. In this case, the
size (480.times.1080) of the resized depth frame is 1/4 of the
original depth frame of FIG. 5A along the row direction D1. Then,
the resized depth frame is split into two parts (240.times.1080,
FIG. 5D), which are flipped later. The combine processing unit 12
combines the two parts to the left side and right side of the
downscaled color frame of FIG. 6B (1440.times.1080), thereby
obtaining the packed frame of FIG. 7A. In addition, FIG. 7B shows
the packed frame displayed in the screen.
[0054] In other embodiments, it is also possible to downscale the
frame along both of the row direction D1 and the column direction
D2. In addition, the depth frame and the color frame can also be
downscaled to different ratios. The size of the color frame is
downscaled to A times of the size of the original color frame along
a direction, and the size of the intermediate depth frame is
downscaled to B times of the size of the original depth frame along
the same direction. Wherein, A and B fit the equality of A+B/3=1.
This invention is not to limit the values of A and B. In the first
embodiment, the downscaled ratio of the intermediate depth frame
along the column direction D2 is the same as the downscaled ratio
of the size of the color frame along the column direction D2, which
means A=B=3/4. In the second embodiment, the downscaled ratio of
the size of the intermediate depth frame along the row direction D1
is the same as the downscaled ratio of the size of the color frame
along the row direction D1, which means A=B=3/4.
[0055] To be noted, the color frame can be obtained by resizing the
original color frame. Otherwise, in other embodiments, it is
possible to combine the color frame and the two parts of the
resized depth frame so as to obtain the packed frame of different
sizes. In addition, the size processing unit 11, the combine
processing unit 12, the split processing unit 13 and the flip
processing unit 14 can be carried out by software executed by a
processor (e.g. MCU).
[0056] The above mentioned software is an example only, and the
skilled person in the art can code different programs base on the
above descriptions. Or, it is also possible to carry out the
functions of the size processing unit 11, the combine processing
unit 12, the split processing unit 13 and the flip processing unit
14 by hardware or firmware. This invention is not limited.
[0057] FIGS. 4C and 7C are schematic diagrams showing an image of
another packed frame obtained by the method and device 1 for
packing a color frame and an original depth frame. The size of the
frames is 1920.times.1080.
[0058] As shown in FIG. 4C or 7C, the person's image in the color
frame viewed by eyes is very nature and will not make the viewer
feel uncomfortable.
[0059] FIG. 8 is a functional block diagram of a device 1a for
packing a color frame and an original depth frame of another aspect
according to a preferred embodiment of the invention.
[0060] Different from the device 1 of FIG. 1B, the device 1a
includes a rotate processing unit 14a instead of the flip
processing unit 14.
[0061] The method and device for packing a color frame and an
original depth frame to obtain a packed frame according to a third
embodiment will be described hereinafter with reference to FIGS. 1A
and 8 in view of FIGS. 9A to 11A. FIGS. 9A to 9D are schematic
diagrams showing the procedure for processing an original depth
frame of a third embodiment. FIG. 9A shows an original depth frame,
FIG. 9B shows an intermediate depth frame, FIG. 9C shows a resized
depth frame, and FIG. 9D shows two parts of the resized depth frame
after being split and flipped. FIGS. 10A and 10B are schematic
diagrams showing the procedure for processing an original color
frame of the third embodiment. FIG. 10A shows an original color
frame, and FIG. 10B shows a color frame produced by resizing the
original color frame of FIG. 10A. Third embodiment will be
described with the original depth frame and the original color
frame of the first embodiment, which means FIG. 9A is the same as
FIG. 2A, and FIG. 10A is the same as FIG. 3A.
[0062] The step S01 is to resize the original depth frame to obtain
a resized depth frame (from FIG. 9A to FIG. 9C). In step S01, the
original depth frame (FIG. 9A) is resized to obtain an intermediate
depth frame (FIG. 9B), and then the sub-pixels of the intermediate
depth frame is rearranged to obtain the resized depth frame (FIG.
9C). The size processing unit 11 can downscale the size of the
original depth frame of FIG. 9A so as to obtain the intermediate
depth frame of FIG. 9B. Herein, the size of the original depth
frame of FIG. 9A along the column direction D2 is downscaled to 2/3
of the original one, and the size of the original depth frame of
FIG. 9A along the row direction D1 is downscaled to 9/16 of the
size of the original one, thereby obtaining the intermediate depth
frame of FIG. 9B. As a result, the resolution of the intermediate
depth frame is 1080.times.720 (1080=1920.times.9/16,
720=1080.times.2/3). The sub-pixel rearrangement has been described
hereinabove, so the detailed description thereof will be omitted.
After the sub-pixel rearrangement, the size of the intermediate
depth frame of FIG. 9B along the column direction D2 is downscaled
to 1/3 of the size of the original one, so that the resolution of
the resized depth frame of FIG. 9C is 1080x240 (240=720.times.1/3).
Besides, the size of the resized depth frame of FIG. 9C along the
column direction D2 is 2/9 of the original depth frame, and the
size of the resized depth frame (1/3.times.1/3) of FIG. 9C along
the row direction D1 is 9/16 of the size of the original depth
frame.
[0063] In addition, the size processing unit 11 downscales the
original color frame of FIG. 10A to obtain the color frame of FIG.
10B. Herein, the size of the color frame of FIG. 10B along the row
direction D1 is downscaled to 7/8 of the original color frame of
FIG. 10A, so that the resolution of the color frame is
1680.times.1080 (1680=1920.times.7/8).
[0064] After the color frame and the resized depth frame are
obtained, the step S02 is performed to combine the color frame and
the resized depth frame to obtain a packed frame. Before the step
S02 of combining the color frame and the resized depth frame by the
combine processing unit 12, a split processing unit 13 splits the
resized depth frame of FIG. 9C into two parts (splitting along the
row direction D1), and a rotate processing unit 14a rotates the two
parts so as to obtain two parts of FIG. 9D (the size of each part
is 120.times.1080). Afterwards, the combine processing unit 12
combines the two rotated parts at a left side and a right side of
the color frame, respectively, to obtain the packed frame of FIG.
11A. Herein, "rotating" is to rotate clockwise by 90 degrees.
Referring to the packed frame as shown in FIG. 11A, the sizes of
the packed frame and the preset frame size of the original color
frame are the same and are 1920.times.1080 (1920=120+1680+120).
[0065] FIG. 11B is a schematic diagram showing the color frame and
a screen as a 2D display device displays the packed frame of the
third embodiment.
[0066] The packed frame is restored to obtain the original color
frame or the original depth frame, which can be further processed
by the 3D display device according to the depth image-based
rendering (DIBR) so as to produce correct naked-eye multi-view
angle 3D images or glasses type dual-view angle 3D images. As shown
in FIG. 11B, the packed frame can be directly displayed on the
screen of the 2D display device. Herein, the center point of the
color frame (the cross point of the diagonal lines of the color
frame) and the center point of the screen (the cross point of the
diagonal lines of the screen), which are all the center point O,
are overlapped. Accordingly, the packed frame can be directly
displayed on the screen of the 2D display device. As shown in FIG.
11A, the color frame (2D image) viewed by eyes is very smooth and
will not make the viewer feel uncomfortable. Of course, it is
discovered that to display the center of the color frame at the
central area of the screen can provide the same and satisfied
effects, so it is unnecessary to totally overlap the center
points.
[0067] To be noted, in the third embodiment, the size of the packed
frame of FIG. 11A and the preset frame size of the original color
frame of FIG. 10A are the same and are both 1920.times.1080
(1080=135+810+135). Similar to the first embodiment, the method and
device 1a for packing the color frame and the original depth frame
to obtain the packed frame can be cooperated with any video
compression method to achieve a better compression efficiency.
Accordingly, the packed frame will not cause additional load for
the transmission bandwidth during data transmission, and it will
not cause the overloading of the 3D display device in the user end
due to the exceeded decompression calculations.
[0068] In the third embodiment, the original depth frame of FIG. 9A
is resized in the row direction D1, so that the width and height
thereof are the same (the width thereof is adjusted from 1920 to
1080). If the size of the original color frame is downscaled by C
times along the row direction D1, the height of the original color
frame is not adjusted. If the size of the original depth frame is
downscaled by D times along the column direction D2, C and D must
fit the equality, C+(original height/original width).times.(D/3)=1.
This invention is not to limit the values of C and D. In the third
embodiment, C=7/8 and D=2/3.
[0069] FIG. 11C is a schematic diagram showing an image of another
packed frame obtained by the method and device 1a for packing a
color frame and an original depth frame. The size of the frames is
1920.times.1080.
[0070] The size of FIG. 11C is still 1920.times.1080. As shown in
FIG. 11C, the person's image in the color frame viewed by eyes is
very nature and will not make the viewer feel uncomfortable.
[0071] FIG. 12 is a functional block diagram of a system 2 for
packing a color frame and an original depth frame of another aspect
according to a preferred embodiment of the invention.
[0072] The system 2 includes a memory unit 21 and a processing unit
22, which are electrically connected to each other. The memory unit
21 stores the color frame and the original depth frame. Of course,
the memory unit 21 also stores the resized depth frame and the two
parts thereof, the intermediate depth frame, the packed frame or
the original color frame. The memory unit 21 can be a
non-transitory computer readable storage medium, such as a memory,
memory card, CD, tape, or any of their combinations, for storing
the needed information. In this embodiment, the memory can be a
ROM, RAM, flash memory, FPGA, and any usable memory, and this
invention is not limited.
[0073] The processing unit 22 includes the core control assembly of
the system 2, such as a CPU and a memory including any control
hardware, software or firmware. In this embodiment, the processing
unit 22 resizes the original depth frame to obtain an intermediate
depth frame, and then rearranges the sub-pixels of the intermediate
depth frame to obtain a resized depth frame. In one embodiment, the
processing unit 22 further split the resized depth frame into two
parts, and combines the two parts of the resized depth frame in the
top side and bottom side of the color frame so as to obtain the
packed frame. Before combining the two parts of the resized depth
frame in the top side and bottom side of the color frame, the
processing unit 22 further flips the two parts of the resized depth
frame.
[0074] In another embodiment, the processing unit 22 further split
the resized depth frame into two parts, and combines the two parts
of the resized depth frame in the left side and right side of the
color frame so as to obtain the packed frame. Before combining the
two parts of the resized depth frame in the left side and right
side of the color frame, the processing unit 22 further rotates the
two parts of the resized depth frame. The processing unit 22
further resizes the original color frame so as to obtain the color
frame.
[0075] In addition, since the center of the color frame is
displayed at the central area of the screen, and the center point
of the color frame and the center point of the screen are
preferably overlapped. The screen is a 2D screen. The other
technical features of the original depth frame, the intermediate
depth frame, the resized depth frame, the color frame, the original
color frame and the packed frame have been described in the
previous embodiments, so they will be omitted.
[0076] To sum up, in the method, device and system of the invention
for packing a color frame and an original depth frame to obtain a
packed frame, the color frame and resized depth frame are combined
to obtain a packed frame with a center displayed at the central
area of the screen. Afterward, the packed frame can be restored to
obtain the original color frame or the original depth frame, which
can be further processed by the 3D display device according to the
depth image-based rendering (DIBR) so as to produce correct
naked-eye multi-view angle 3D images or glasses type dual-view
angle 3D images. Most important, since the center of the color
frame is displayed at the central area of the screen, and the
center point of the color frame and the center point of the screen
are preferably overlapped and most preferably totally overlapped,
the packed frame can be directly displayed on the screen of a 2D
display device. Moreover, the displayed 2D images (color frames)
are smoothly shown in the central area of a screen and will not
make the viewer uncomfortable.
[0077] In addition, the method, device and system of the invention
also resize the original depth frame to obtain the resized depth
frame and combine the color frame and the resized depth frame to
obtain a simplified packed frame. Accordingly, the technology of
the invention can be cooperated with other video compression system
to enhance the compression ratio, thereby decreasing the bandwidth
requirement for 3D video. This feature is further benefit to the
promotion of 3D channels.
[0078] Although the invention has been described with reference to
specific embodiments, this description is not meant to be construed
in a limiting sense. Various modifications of the disclosed
embodiments, as well as alternative embodiments, will be apparent
to persons skilled in the art. It is, therefore, contemplated that
the appended claims will cover all modifications that fall within
the true scope of the invention.
* * * * *