U.S. patent application number 13/965169 was filed with the patent office on 2014-05-29 for data processing apparatus for configuring camera interface based on compression characteristic of compressed multimedia data and related data processing method.
This patent application is currently assigned to MEDIATEK INC.. The applicant listed for this patent is MEDIATEK INC.. Invention is credited to Chi-Cheng Ju, Tsu-Ming Liu.
Application Number | 20140146187 13/965169 |
Document ID | / |
Family ID | 50772955 |
Filed Date | 2014-05-29 |
United States Patent
Application |
20140146187 |
Kind Code |
A1 |
Ju; Chi-Cheng ; et
al. |
May 29, 2014 |
DATA PROCESSING APPARATUS FOR CONFIGURING CAMERA INTERFACE BASED ON
COMPRESSION CHARACTERISTIC OF COMPRESSED MULTIMEDIA DATA AND
RELATED DATA PROCESSING METHOD
Abstract
A data processing apparatus at a transmitter end has an output
interface and a camera controller. The output interface packs a
compressed multimedia data into an output bitstream transmitted via
a camera interface. The camera controller refers to a compression
characteristic of the compressed multimedia data to configure a
transmission setting of the output interface over the camera
interface. A data processing apparatus at a receiver end has an
input interface and a controller. The input interface un-packs an
input bitstream received via the camera interface into a compressed
multimedia data. The controller configures a reception setting of
the input interface over the camera interface in response to a
compression characteristic of the compressed multimedia data. In
addition, the data processing apparatus at the transmitter end may
selectively enable a compression mode by checking the
de-compression capability of the data processing apparatus at the
receiver end.
Inventors: |
Ju; Chi-Cheng; (Hsinchu
City, TW) ; Liu; Tsu-Ming; (Hsinchu City,
TW) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
MEDIATEK INC. |
Hsin-Chu |
|
TW |
|
|
Assignee: |
MEDIATEK INC.
Hsin-Chu
TW
|
Family ID: |
50772955 |
Appl. No.: |
13/965169 |
Filed: |
August 12, 2013 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61729426 |
Nov 23, 2012 |
|
|
|
Current U.S.
Class: |
348/207.1 |
Current CPC
Class: |
H04N 19/179 20141101;
G06F 1/325 20130101; H04N 5/232411 20180801; Y02D 10/00 20180101;
H04N 5/23241 20130101; H04N 5/23293 20130101; H04N 19/184 20141101;
G09G 2340/04 20130101; H04N 19/146 20141101; G09G 2340/02 20130101;
H04N 5/23229 20130101; H04N 19/10 20141101; H04N 5/23235 20130101;
G06F 3/14 20130101; G06F 1/3278 20130101; H04N 19/154 20141101;
H04N 19/12 20141101 |
Class at
Publication: |
348/207.1 |
International
Class: |
G06F 1/32 20060101
G06F001/32 |
Claims
1. A data processing apparatus comprising: an output interface,
arranged for packing a compressed multimedia data into an output
bitstream and outputting the output bitstream via a camera
interface; and a camera controller, arranged for referring to at
least a compression characteristic of the compressed multimedia
data to configure a transmission setting of the output interface
over the camera interface.
2. The data processing apparatus of claim 1, wherein the camera
interface is a camera serial interface (CSI) standardized by a
Mobile Industry Processor Interface (MIPI).
3. The data processing apparatus of claim 1, wherein the
compression characteristic is a compression ratio corresponding to
the compressed multimedia data.
4. The data processing apparatus of claim 1, wherein the
transmission setting of the output interface comprises a number of
data lines enabled over the camera interface.
5. The data processing apparatus of claim 4, wherein an operating
frequency of each data line remains unchanged regardless of the
configured number of data lines.
6. The data processing apparatus of claim 1, wherein the
transmission setting of the output interface comprises an operating
frequency of each data line.
7. The data processing apparatus of claim 6, wherein a number of
data lines enabled over the camera interface remains unchanged
regardless of the configured operating frequency.
8. The data processing apparatus of claim 1, wherein the
transmission setting of the output interface comprises a behavior
of the output interface during a blanking period between adjacent
data transmissions.
9. The data processing apparatus of claim 8, wherein a number of
data lines enabled over the camera interface and an operating
frequency of each data line remain unchanged regardless of the
configured behavior of the output interface during the blanking
period.
10. The data processing apparatus of claim 1, wherein the data
processing apparatus is coupled to another data processing
apparatus via the camera interface; and the camera controller is
further arranged for checking a de-compression capability of the
another data processing apparatus, where the data processing
apparatus selectively enables a compression mode according to a
checking result.
11. A data processing apparatus comprising: an input interface,
arranged for receiving an input bitstream via a camera interface,
and un-packing the input bitstream into a compressed multimedia
data that is transmitted over the camera interface; and an image
signal processor (ISP) controller, arranged for configuring a
reception setting of the input interface over the camera interface
in response to at least a compression characteristic of the
compressed multimedia data.
12. The data processing apparatus of claim 11, wherein the camera
interface is a camera serial interface (CSI) standardized by a
Mobile Industry Processor Interface (MIPI).
13. The data processing apparatus of claim 11, wherein the
compression characteristic is a compression ratio corresponding to
the compressed multimedia data.
14. The data processing apparatus of claim 11, wherein the
reception setting of the input interface comprises a number of data
lines enabled over the camera interface.
15. The data processing apparatus of claim 14, wherein an operating
frequency of each data line remains unchanged regardless of the
configured number of data lines.
16. The data processing apparatus of claim 11, wherein the
reception setting of the input interface comprises an operating
frequency of each data line.
17. The data processing apparatus of claim 16, wherein a number of
data lines enabled over the camera interface remains unchanged
regardless of the configured operating frequency.
18. The data processing apparatus of claim 11, wherein the
reception setting of the input interface comprises a behavior of
the input interface during a blanking period between adjacent data
transmissions.
19. The data processing apparatus of claim 18, wherein a number of
data lines enabled over the camera interface and an operating
frequency of each data line remain unchanged regardless of the
configured behavior of the input interface during the blanking
period.
20. The data processing apparatus of claim 11, wherein the data
processing apparatus is coupled to another data processing
apparatus via the camera interface, and the ISP controller is
further arranged for informing the another data processing
apparatus of a de-compression capability of the data processing
apparatus.
21. A data processing method comprising: referring to at least a
compression characteristic of a compressed multimedia data to
configure a transmission setting of an output interface over the
camera interface; and utilizing an output interface for packing the
compressed multimedia data into an output bitstream and outputting
the output bitstream via the camera interface.
22. The data processing method of claim 21, wherein the camera
interface is a camera serial interface (CSI) standardized by a
Mobile Industry Processor Interface (MIPI).
23. The data processing method of claim 21, wherein the compression
characteristic is a compression ratio corresponding to the
compressed multimedia data.
24. The data processing method of claim 21, wherein the
transmission setting of the output interface comprises a number of
data lines enabled over the camera interface.
25. The data processing method of claim 21, wherein the
transmission setting of the output interface comprises an operating
frequency of each data line.
26. The data processing method of claim 21, wherein the
transmission setting of the output interface comprises a behavior
of the output interface during a blanking period between adjacent
data transmissions.
27. The data processing method of claim 21, wherein the output
bitstream is transmitted to a data processing apparatus via the
camera interface; and the data processing method further comprises:
checking a de-compression capability of the data processing
apparatus, and selectively enabling a compression mode according to
a checking result.
28. A data processing method comprising: configuring a reception
setting of an input interface over a camera interface in response
to at least a compression characteristic of a compressed multimedia
data; and utilizing an input interface for receiving an input
bitstream via the camera interface, and un-packing the input
bitstream into the compressed multimedia data that is transmitted
over the camera interface.
29. The data processing method of claim 28, wherein the camera
interface is a camera serial interface (CSI) standardized by a
Mobile Industry Processor Interface (MIPI).
30. The data processing method of claim 28, wherein the compression
characteristic is a compression ratio corresponding to the
compressed multimedia data.
31. The data processing method of claim 28, wherein the reception
setting of the input interface comprises a number of data lines
enabled over the camera interface.
32. The data processing method of claim 28, wherein the reception
setting of the input interface comprises an operating frequency of
each data line.
33. The data processing method of claim 28, wherein the reception
setting of the input interface comprises a behavior of the input
interface during a blanking period between adjacent data
transmissions.
34. The data processing method of claim 28, wherein the input
bitstream is received from a data processing apparatus via the
camera interface; and the data processing method further comprises:
informing the data processing apparatus of a de-compression
capability.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of U.S. provisional
application No. 61/729,426, filed on Nov. 23, 2012 and incorporated
herein by reference.
BACKGROUND
[0002] The disclosed embodiments of the present invention relate to
transmitting and receiving multimedia data over a camera interface,
and more particularly, to a data processing apparatus for
configuring a camera interface based on a compression
characteristic of a compressed multimedia data and related data
processing method.
[0003] A camera interface is disposed between a first chip and a
second chip to transmit multimedia data from the first chip to the
second chip for further processing. For example, the first chip may
include a camera module, and the second chip may include an image
signal processor (ISP). The multimedia data may include image data
(i.e., a single still image) or video data (i.e., a video sequence
composed of images). When a camera sensor with a higher resolution
is employed in the camera module, the multimedia data transmitted
over the camera interface would have a larger data size/data rate,
which increases the power consumption of the camera interface
inevitably. If the camera module and the ISP are both located at a
portable device (e.g., a smartphone) powered by a battery device,
the battery life is shortened due to the increased power
consumption of the camera interface. Thus, there is a need for an
innovative design which can effectively reduce the power
consumption of the camera interface.
SUMMARY
[0004] In accordance with exemplary embodiments of the present
invention, a data processing apparatus for configuring a camera
interface based on a compression characteristic of a compressed
multimedia data and related data processing method are
proposed.
[0005] According to a first aspect of the present invention, an
exemplary data processing apparatus is disclosed. The exemplary
data processing apparatus includes an output interface and a camera
controller. The output interface is arranged for packing a
compressed multimedia data into an output bitstream and outputting
the output bitstream via a camera interface. The camera controller
is arranged for referring to at least a compression characteristic
of the compressed multimedia data to configure a transmission
setting of the output interface over the camera interface.
[0006] According to a second aspect of the present invention, an
exemplary data processing apparatus is disclosed. The exemplary
data processing apparatus includes an input interface and an image
signal processor (ISP) controller. The input interface is arranged
for receiving an input bitstream via a camera interface, and
un-packing the input bitstream into a compressed multimedia data
that is transmitted over the camera interface. The controller is
arranged for configuring a reception setting of the input interface
over the camera interface in response to at least a compression
characteristic of the compressed multimedia data.
[0007] According to a third aspect of the present invention, an
exemplary data processing method is disclosed. The exemplary data
processing method includes following steps: referring to at least a
compression characteristic of a compressed multimedia data to
configure a transmission setting of an output interface over the
camera interface; and utilizing an output interface for packing the
compressed multimedia data into an output bitstream and outputting
the output bitstream via the camera interface.
[0008] According to a fourth aspect of the present invention, an
exemplary data processing method is disclosed. The exemplary data
processing method includes the following steps: configuring a
reception setting of an input interface over a camera interface in
response to at least a compression characteristic of a compressed
multimedia data; and utilizing an input interface for receiving an
input bitstream via the camera interface, and un-packing the input
bitstream into the compressed multimedia data that is transmitted
over the camera interface.
[0009] These and other objectives of the present invention will no
doubt become obvious to those of ordinary skill in the art after
reading the following detailed description of the preferred
embodiment that is illustrated in the various figures and
drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
[0010] FIG. 1 is a block diagram illustrating a data processing
system according to an embodiment of the present invention.
[0011] FIG. 2 is a flowchart illustrating a first control and data
flow of the data processing system shown in FIG. 1.
[0012] FIG. 3 is a flowchart illustrating a second control and data
flow of the data processing system shown in FIG. 1.
[0013] FIG. 4 is a diagram illustrating the change of the blanking
period after the interface compression is applied.
[0014] FIG. 5 is a flowchart illustrating a third control and data
flow of the data processing system shown in FIG. 1.
[0015] FIG. 6 is a flowchart illustrating a method of setting the
camera module by referring to a result of checking a de-compression
capability of the ISP according to an embodiment of the present
invention.
DETAILED DESCRIPTION
[0016] Certain terms are used throughout the description and
following claims to refer to particular components. As one skilled
in the art will appreciate, manufacturers may refer to a component
by different names. This document does not intend to distinguish
between components that differ in name but not function. In the
following description and in the claims, the terms "include" and
"comprise" are used in an open-ended fashion, and thus should be
interpreted to mean "include, but not limited to . . . ". Also, the
term "couple" is intended to mean either an indirect or direct
electrical connection. Accordingly, if one device is coupled to
another device, that connection may be through a direct electrical
connection, or through an indirect electrical connection via other
devices and connections.
[0017] The present invention proposes applying data compression to
a multimedia data and then transmitting a compressed multimedia
data over a camera interface. As the data size/data rate of the
compressed multimedia data is smaller than that of the original
un-compressed multimedia data, the power consumption of the camera
interface is reduced correspondingly. Besides, as the compressed
multimedia data requires a smaller occupied bandwidth, the present
invention further proposes configuring the camera interface based
on the compression characteristic of the compressed multimedia
data. Further details will be described as below.
[0018] FIG. 1 is a block diagram illustrating a data processing
system according to an embodiment of the present invention. The
data processing system 100 includes a plurality of data processing
apparatuses such as a camera module 102 and an image signal
processor (ISP) 104. The camera module 102 and the ISP 104 may be
different chips, and the camera module 102 communicates with the
ISP 104 via a camera interface 103 having a plurality of data lines
(e.g., DL0, DL1, DL2, and DL3) (for example, differential pair, a
pin or group of pins) and one clock line CLK. It should be noted
that the number of data lines shown in FIG. 1 is for illustrative
purposes only, and is not meant to be a limitation of the present
invention. In this embodiment, the camera interface 103 may be a
camera serial interface (CSI) standardized by a Mobile Industry
Processor Interface (MIPI).
[0019] The camera module 102 supports un-compressed data
transmission and compressed data transmission. When the camera
module 102 is used to transmit un-compressed data to the ISP 104,
the camera module 102 generates the un-compressed multimedia data
D1 according to an input multimedia data derived from capturing the
incident light, and transmits the un-compressed multimedia data D1
over the camera interface 103. When the camera module 102 is used
to transmit compressed data to the ISP 104, the camera module 102
generates a compressed multimedia data D1' according to the input
multimedia data derived from capturing the incident light, and
transmits the compressed multimedia data D1' over the camera
interface 103. By way of example, but not limitation, the input
multimedia data may be an image data or a video data.
[0020] As shown in FIG. 1, the camera module 102 includes a camera
controller 112, an output interface 114 and other circuitry 116.
The other circuitry 116 includes circuit elements required for
capturing the incident light to obtain the input multimedia data
which is used for generating the un-compressed data D1 or the
compressed data D1'. For example, the other circuitry 116 may have
a lens module, a camera sensor, a camera buffer, a compressor,
multiplexer(s), etc. The lens module guides the incident light to
the camera sensor, and the camera sensor obtains the input
multimedia data correspondingly. The input multimedia data obtained
by the camera sensor may be a single captured image or a video
sequence composed of a plurality of captured images. Besides, the
input multimedia data obtained by the camera sensor may be single
view data for 2D display or multiple view data for 3D display. The
camera buffer is arranged to buffer the input multimedia data
obtained by the camera sensor. The compressor performs data
compression. The multiplexer receives the un-compressed multimedia
data D1 and the compressed multimedia data D1', and selectively
outputs the un-compressed multimedia data D1 or the compressed
multimedia data D1' according to the operation mode of the camera
module 102. For example, when the camera module 102 is operated
under a compression mode, the multiplexer outputs the compressed
multimedia data D1'; and when the camera module 102 is operated
under a non-compression mode, the multiplexer outputs the
un-compressed multimedia data D1. As the present invention focuses
on the control of the output interface 114, further description of
the other circuitry 116 is omitted here for brevity.
[0021] The output interface 114 includes a packing unit 117 and a
plurality of switches (e.g., 118_1, 118_2, 118_3 and 118_4). It
should be noted that the number of the switches included in the
output interface 114 is equal to the number of data lines included
in the camera interface 103. Hence, each of the switches
118_1-118_4 controls whether a data line is used for data
transmission. In this embodiment, the switches 118_1-118_4 are
controlled by a plurality of enable signals EN0-EN3 generated from
the camera controller 112, respectively. By way of example, the
enable signals EN0-EN3 may be transmitted via an I.sup.2C
(Inter-Integrated Circuit) interface, and/or programmed in an
e-fuse device or an EPPROM (erasable programmable read only memory)
by the camera controller 112. When an enable signal has a first
logic value (e.g., `1`), a corresponding switch is enabled (i.e.,
switched on) to enable the data transmission over the corresponding
data line; and when the enable signal has a second logic value
(e.g., `0`), the corresponding switch is disabled (i.e., switched
off) to disable the data transmission over the corresponding data
line.
[0022] Regarding the ISP 104, it communicates with the camera
module 102 via the camera interface 103. In this embodiment, the
ISP 104 supports un-compressed data reception and compressed data
reception. When the camera module 102 transmits the un-compressed
data D1 to the ISP 104, the ISP 104 is operated under a
non-decompression mode to receive an un-compressed data D2 from the
camera interface 103, generate a processed multimedia data
according to the un-compressed multimedia data D2, and send the
processed multimedia data to one or multiple processors, such as a
display processor 106_1, a video processor 106_2 and/or a graphic
processor 106_3, for further processing. When the camera module 102
transmits the compressed data D1' to the ISP 104, the ISP 104 is
operated under a de-compression mode to receive a compressed
multimedia data D2' from the camera interface 103, generate a
processed multimedia data according to a de-compressed multimedia
data derived from de-compressing the compressed multimedia data
D2', and send the processed multimedia data to one or multiple
processors, such as the display processor 106_1, the video
processor 106_2 and/or the graphic processor 106_3, for further
processing. If there is no error introduced during the data
transmission, the un-compressed data D1 transmitted under the
non-compression mode should be identical to the un-compressed data
D2 received under the non-decompression mode, and the compressed
data D1' transmitted under the compression mode should be identical
to the compressed data D2' received under the de-compression
mode.
[0023] As shown in FIG. 1, the ISP 104 includes an ISP controller
122, an input interface 124 and other circuitry 126. The other
circuitry 126 may include circuit elements required for generating
the processed multimedia data according to an internal transmission
mode (e.g., an on-the-fly mode or an off-line mode). For example,
the other circuitry 126 may have a de-compressor, a write direct
memory access (DMA) controller, a read DMA controller, an image
processor, multiplexers, etc. The de-compressor is used for
performing data de-compression to obtain a de-compressed multimedia
data. The write DMA controller is arranged for storing a multimedia
data into an external image buffer of the ISP 105, where the
multimedia data may be an un-compressed multimedia data, a
compressed multimedia data or a de-compressed multimedia data. The
read write DMA controller is arranged for reading a buffered
multimedia data from the external image buffer of the ISP 104. The
image processor is arranged to support several image processing
operations, including resizing/scaling, rotation, quality
enhancement, etc., where the processed multimedia data is generated
from the image processor. The multiplexers control interconnections
of the de-compressor, the image processor, the write DMA
controller, and the read DMA controller. As the present invention
focuses on the control of the input interface 124, further
description of the other circuitry 126 is omitted here for
brevity.
[0024] The input interface 124 includes an un-packing unit 127 and
a plurality of switches (e.g., 128_1, 1 28_2, 128_3 and 128_4). It
should be noted that the number of the switches included in the
input interface 124 is equal to the number of data lines included
in the camera interface 103. Hence, each of the switches
128_1-128_4 controls whether a data line is used for data
reception. In this embodiment, the switches 128_1-128_4 are
controlled by a plurality of enable signals EN0'-EN3' generated
from the ISP controller 122, respectively. By way of example, the
enable signals EN0'-EN3' may be transmitted via an I.sup.2C
interface, and/or programmed in an e-fuse device or an EPPROM by
the ISP controller 122. When an enable signal has a first logic
value (e.g., `1`), a corresponding switch is enabled (i.e.,
switched on) to enable the data reception over the corresponding
data line; and when the enable signal has a second logic value
(e.g., `0`), the corresponding switch is disabled (i.e., switched
off) to disable the data reception over the corresponding data
line. It should be noted that, to make the data successfully
transmitted from the camera module 102 and received by the ISP 104,
switches located at different ends of one data line of the camera
interface 103 should be both enabled. Further details of
controlling the output interface 114 of the camera module 102 and
the input interface 124 of the ISP 104 are described as below.
[0025] When the non-compression mode of the camera module 102 is
enabled, the camera controller 112 controls the other circuitry 116
to generate the un-compressed multimedia data D1 to the output
interface 114, and controls the output interface 114 to use all of
the data lines DL0-DL3, each having a predetermined operating
frequency, for data transmission. The predetermined operating
frequency (i.e., a bit clock rate per line) may be set by using the
following equations.
Pixel Rate ( Mhz ) = Frame Resolution .times. Frame Rate .times. (
1 + % of Blanking ) 10 6 ( 1 ) Bit Clock Rate per Line = Pixel Rate
.times. Bits per Pixel Number of Data Lines ( 2 ) ##EQU00001##
[0026] For example, considering a 20M (20-Mega pixels) sensor
operating at 30 fps (frames per second) capture rate with 10% of
blanking overhead, the pixel rate equals 660 MHz. When Bayer 10-bit
color per pixel is transmitted over 4 data lines, the bit clock
rate per line reaches 1.65 Gbps.
[0027] Besides, when the non-decompression mode of the ISP 104 is
enabled, the ISP controller 122 controls the input interface 124 to
use all of the data lines DL0-DL3, each having the predetermined
operating frequency, for data reception, and controls the other
circuitry 126 to generate the processed multimedia data based on
the un-compressed multimedia data D2.
[0028] Regarding the output interface 114, the packing unit 117 is
arranged for packing/packetizing the un-compressed multimedia data
D1 based on the transmission protocol of the camera interface 103
and accordingly generating an output bitstream to the camera
interface 103, wherein all of the switches 118_1-118_4 are enabled
(i.e., switched on), and each data line is operated under the
predetermined operating frequency (e.g., 1.65 Gbps) determined
according to above equations. In other words, the output interface
114 has a transmission setting for packing/packetizing the
un-compressed multimedia data D1 into an output bitstream and
outputting the output bitstream via the camera interface 103.
[0029] Regarding the input interface 124, the un-packing unit 117
is arranged for un-packing/un-packetizing an input bitstream based
on the transmission protocol of the camera interface 103 and
accordingly generating the un-compressed multimedia data D2 to the
other circuitry 126, wherein all of the switches 128_1-128_4 are
enabled (i.e., switched on), and each data line is operated under
the predetermined operating frequency (e.g., 1.65 Gbps) determined
according to above equations. In other words, the input interface
124 has a reception setting for receiving an input bitstream via
the camera interface 103 and un-packing/un-packetizing the input
bitstream into the un-compressed multimedia data D1 that is
transmitted over the camera interface 103.
[0030] In general, the aforementioned un-compressed data
transmission is exploited and standardized by MIPI's CSI. However,
it may not afford the high data rate, and may have potential
problems in high power dissipation and low design yield. To
alleviate the aforementioned problems, the other circuitry 116 in
the camera module 102 is configured to have a compressor
implemented therein, and the other circuitry 126 in the ISP 104 is
configured to have a de-compressor implemented therein. Hence, the
compressed data transmission over the camera interface 103 is
realized through the compressor and the de-compressor.
[0031] When the compression mode of the camera module 102 is
enabled, the camera controller 112 controls the other circuitry 116
to generate the compressed multimedia data D1' to the output
interface 114, wherein the compressor implemented in the other
circuitry 116 may employ a lossy or lossless compression algorithm,
depending upon actual design consideration/requirement. Regarding
the output interface 114, the packing unit 117 packs/packetizes the
compressed multimedia data D1' based on the transmission protocol
of the camera interface 103 and accordingly generates an output
bitstream to the camera interface 103. In other words, when the
camera module 102 is operated under the compression mode, the
output interface 114 is arranged for packing/packetizing the
compressed multimedia data D1' into an output bitstream and
outputting the output bitstream via the camera interface 103.
[0032] When the de-compression mode of the ISP 104 is enabled, the
ISP controller 122 controls the other circuitry 126 to generate the
processed multimedia data based on the compressed multimedia data
D2', wherein the de-compressor implemented in the other circuitry
126 may employ a lossy or lossless de-compression algorithm,
depending upon actual design consideration/requirement. Regarding
the input interface 124, the un-packing unit 127
un-packs/un-packetizes the input bitstream into the compressed
multimedia data D2' based on the transmission protocol of the
camera interface 103. In other words, when the ISP 104 is operated
under the de-compression mode, the input interface 124 is arranged
for receiving an input bitstream via the camera interface 103, and
un-packing/un-packetizing the input bitstream into the compressed
multimedia data D2' that is transmitted over the camera interface
103.
[0033] As the data size of the compressed multimedia data is
smaller than that of the original un-compressed multimedia data,
the output interface 114 is controlled by the camera controller 112
to have a different transmission setting, and the input interface
124 of the ISP 104 is controlled by the ISP controller 122 to have
a different reception setting. In this embodiment of the present
invention, the camera interface 103 provides a data line management
layer and is scalable to the number of data lines according to the
bandwidth and the compression requirement. The camera controller
112 is therefore arranged for referring to at least a compression
characteristic of the compressed multimedia data D1' to configure a
transmission setting of the output interface 114 over the camera
interface 103, and the ISP controller 122 is arranged for
configuring a reception setting of the input interface 124 over the
camera interface 103 in response to at least the compression
characteristic of the compressed multimedia data D1'.
[0034] By way of example, but not limitation, the aforementioned
compression characteristic may be a compression ratio M
corresponding to the compressed multimedia data D1', where
M.ltoreq.1.0. The compression ratio M may be expressed using the
following equation.
M = amount of compressed data amount of un - compressed data ( 3 )
##EQU00002##
[0035] Take 32.times.1 Bayer raw 10-bit pixels as an example, the
amount of un-compressed data reaches 320 bits (i.e.,
32.times.1.times.10 bits). When the compression ratio M is equal
0.5, the compressed data would have 160 bits. As the data amount to
be transmitted is reduced by data compression with the compression
ratio M smaller than 1, the number of data lines, the operating
frequency of each data line, and/or the behavior in the blanking
period may be adjusted to improve the overall system
performance.
[0036] In a first exemplary design, each of the camera controller
112 and the ISP controller 122 refers to the compression ratio M to
configure the number of data lines enabled over the camera
interface 103 for data transmission and reception when the camera
module 102 is used to transmit the compressed multimedia data D1'
to the ISP 104, wherein an operating frequency of each data line
(i.e., a bit clock rate of each data line) remains unchanged
regardless of the configured number of data lines. As the data
compression is performed on the camera module 102, the camera
module 102 may inform the ISP 104 of the compression ratio M
corresponding to the compressed multimedia data D1'. Suppose that
all of the data lines DL0-DL3, each having the predetermined
operating frequency, will be used for transmitting the
un-compressed multimedia data D1. Thus, when the non-compression
mode of the camera module 102 is enabled and the non-decompression
mode of the ISP 104 is enabled, the camera controller 112 sets each
of the enable signals EN0-EN3 by the first logic value (e.g., `1`)
such that all of the switches 118_1-118_4 are enabled (i.e.,
switched on), and the ISP controller 122 sets each of the enable
signals EN0'-EN3' by the first logic value (e.g., `1`) such that
all of the switches 128_1-128_4 are enabled (i.e., switched
on).
[0037] However, when the compression mode of the camera module 102
is enabled and the de-compression mode of the ISP 104 is enabled,
the number of data lines enabled over the camera interface 103 is
set to a value equal to a product of the number of data lines
DL0-DL3 (i.e., the number of data lines without compression) and
the compression ratio M. For example, if M=0.5 and the number of
data lines without compression is 4, the number of data lines with
compression is equal to 2. As the number of data lines is reduced
to 2, the camera controller 112 may set two of four enable signals
(e.g., EN0 and EN1) by the first logic value (e.g., `1`) and set
the remaining enable signals (e.g., EN2 and EN3) by the second
logic value (e.g., `0`), and the ISP controller 122 may set two of
four enable signals (e.g., EN0' and EN1') by the first logic value
(e.g., `1`) and set the remaining enable signals (e.g., EN2' and
EN3') by the second logic value (e.g., `0`). Thus, only a portion
of the switches 118_1-118_4 are enabled (i.e., switched on) by the
camera controller 112, and only a portion of the switches
128_1-128_4 are enabled (i.e., switched on) by the ISP controller
122. To put it simply, the bandwidth can be reduced by the
interface compression, and the reduced number of data lines is
related to the compression ratio M. Besides, due to the reduced
number of data lines enabled over the camera interface 103, the
transmission power dissipation and electromagnetic interference
(EMI) can be alleviated.
[0038] Please refer to FIG. 2, which is a flowchart illustrating a
first control and data flow of the data processing system 100 shown
in FIG. 1. In this embodiment, the compression ratio M is
referenced to configure the number of data lines enabled over the
camera interface 103, and the operating frequency of each data line
remains unchanged. Provided that the result is substantially the
same, the steps are not required to be executed in the exact order
shown in FIG. 2. The exemplary first control and data flow may be
briefly summarized by following steps.
[0039] Step 200: Start.
[0040] Step 202: Check if a compression mode is enabled. If yes, go
to step 210; otherwise, go to step 204.
[0041] Step 204: The other circuitry 116 generates the
un-compressed multimedia data D1 according to an input multimedia
data obtained by a camera sensor.
[0042] Step 206: The camera controller 112 controls the output
interface 114 to switch on all of the switches 118_1-118_4. In this
way, all of the data lines DL0-DL3 of the camera interface 103
would be used by the camera module 102 for data transmission.
[0043] Step 208: The packing unit 117 directly packs/packetizes the
un-compressed multimedia data D1 into an output bitstream. Go to
step 216.
[0044] Step 210: The other circuitry 116 generates the compressed
multimedia data D1' according to the input multimedia data obtained
by the camera sensor.
[0045] Step 212: The camera controller 112 refers to the
compression ratio M corresponding to the compressed multimedia data
D1' to switch on a portion of the switches 118_1-118_4. In this
way, only part of the data lines DL0-DL3 of the camera interface
103 would be used by the camera module 102 for data
transmission.
[0046] Step 214: The packing unit 117 packs/packetizes the
compressed multimedia data D1' into an output bitstream.
[0047] Step 216: Transmit the output bitstream over the camera
interface 103.
[0048] Step 218: Check if a de-compression mode is enabled. If yes,
go to step 226; otherwise, go to step 220.
[0049] Step 220: The ISP controller 122 controls the input
interface 124 to switch on all of the switches 128_1-128_4. In this
way, all of the data lines DL0-DL3 of the camera interface 103
would be used by the ISP 104 for data reception.
[0050] Step 222: The un-packing unit 127 un-packs/un-packetizes an
input bitstream into the un-compressed multimedia data D2.
[0051] Step 224: The other circuitry 126 generate a processed
multimedia data according to the un-compressed multimedia data D2.
Go to step 232.
[0052] Step 226: The ISP controller 122 refers to the compression
ratio M corresponding to the compressed multimedia data D1' to
switch on a portion of the switches 128_1-128_4. In this way, only
part of the data lines DL0-DL3 of the camera interface 103 would be
used by the ISP 104 for data reception.
[0053] Step 228: The un-packing unit 127 un-packs/un-packetizes the
input bitstream into the compressed multimedia data D2'.
[0054] Step 229: The other circuitry 126 derives a de-compressed
multimedia data from de-compressing the compressed multimedia data
D2'.
[0055] Step 230: The other circuitry 126 generates the processed
multimedia data according to the de-compressed multimedia data.
[0056] Step 232: End.
[0057] It should be noted that steps 202-216 are performed by the
camera module 102, and steps 218-230 are performed by the ISP 104.
As a person skilled in the art can readily understand details of
each step shown in FIG. 2 after reading above paragraphs, further
description is omitted here for brevity.
[0058] In a second exemplary design, each of the camera controller
112 and the ISP controller 122 refers to the compression ratio M to
configure an operating frequency of each data line (i.e., a bit
clock rate of each data line) when the camera module 102 is used to
transmit the compressed multimedia data D1' to the ISP 104, wherein
the number of data lines enabled over the camera interface 103
remains unchanged regardless of the configured operating frequency.
As the data compression is performed on the camera module 102, the
camera module 102 may inform the ISP 104 of the compression ratio M
corresponding to the compressed multimedia data D1'. Suppose that
all of the data lines DL0-DL3, each having the predetermined
operating frequency, will be used for transmitting the
un-compressed multimedia data D1. Thus, when the non-compression
mode of the camera module 102 is enabled and the non-decompression
mode of the ISP 104 is enabled, each of the data lines DL0-DL3 is
controlled to operate under the predetermined operating frequency,
wherein all of the switches 118_1-118_4 are enabled (i.e., switched
on) by the camera controller 112, and all of the switches
128_1-128_4 are enabled (i.e., switched on) by the ISP controller
122.
[0059] However, when the compression mode of the camera module 102
is enabled and the de-compression mode of the ISP 104 is enabled,
the operating frequency of each data line is set by a value equal
to a product of the predetermined operating frequency (i.e., a
non-compression/non-decompression mode bit clock rate) and the
compression ratio M. Thus, the compressed multimedia data D1' is
transmitted at a lower clock rate under the condition where the
number of data lines enabled over the camera interface 103 remains
unchanged. For example, if M=0.5, the compression/de-compression
mode bit clock rate is half of the
non-compression/non-decompression mode bit clock rate. In one
exemplary implementation, the required clock frequency selection
may be implemented using different clock generators which supply
clocks with different clock rates. However, this is for
illustrative purposes only, and is not meant to be a limitation of
the present invention. To put it simply, the bandwidth can be
reduced by the interface compression, and the reduced operating
frequency is related to the compression ratio M. Besides, due to
the reduced operating frequency of each data line enabled over the
camera interface 103, the transmission power dissipation and EMI
can be alleviated.
[0060] Please refer to FIG. 3, which is a flowchart illustrating a
second control and data flow of the data processing system 100
shown in FIG. 1. In this embodiment, the compression ratio M is
referenced to configure the operating frequency of each data line,
and the number of data lines enabled over the camera interface 103
remains unchanged. Provided that the result is substantially the
same, the steps are not required to be executed in the exact order
shown in FIG. 3. The exemplary second control and data flow may be
briefly summarized by following steps.
[0061] Step 300: Start.
[0062] Step 302: Check if a compression mode is enabled. If yes, go
to step 310; otherwise, go to step 304.
[0063] Step 304: The other circuitry 116 generates the
un-compressed multimedia data D1 according to an input multimedia
data obtained by a camera sensor.
[0064] Step 306: The camera controller 112 controls the output
interface 114 to set a predetermined operating frequency of data
transmission on each data line.
[0065] Step 308: The packing unit 117 directly packs/packetizes the
un-compressed multimedia data D1 into an output bitstream. Go to
step 316.
[0066] Step 310: The other circuitry 116 generates the compressed
multimedia data D1' according to the input multimedia data obtained
by the camera sensor.
[0067] Step 312: The camera controller 112 refers to the
compression ratio M corresponding to the compressed multimedia data
D1' to set a reduced operating frequency of data transmission on
each data line.
[0068] Step 314: The packing unit 117 packs/packetizes the
compressed multimedia data D1' into an output bitstream.
[0069] Step 316: Transmit the output bitstream over the camera
interface 103.
[0070] Step 318: Check if a de-compression mode is enabled. If yes,
go to step 326; otherwise, go to step 320.
[0071] Step 320: The ISP controller 122 controls the input
interface 124 to set a predetermined operating frequency of data
reception on each data line.
[0072] Step 322: The un-packing unit 127 un-packs/un-packetizes an
input bitstream into the un-compressed multimedia data D2.
[0073] Step 324: The other circuitry 126 generates a processed
multimedia data according to the un-compressed multimedia data D2.
Go to step 332.
[0074] Step 326: The ISP controller 122 refers to the compression
ratio M corresponding to the compressed multimedia data D1' to set
a reduced operating frequency for data reception on each data
line.
[0075] Step 328: The un-packing unit 127 un-packs/un-packetizes the
input bitstream into the compressed multimedia data D2'.
[0076] Step 329: The other circuitry 126 derives a de-compressed
multimedia data from de-compressing the compressed multimedia data
D2'.
[0077] Step 330: The other circuitry 126 generates the processed
multimedia data according to the de-compressed multimedia data.
[0078] Step 332: End.
[0079] It should be noted that steps 302-316 are performed by the
camera module 102, and steps 318-330 are performed by the ISP 104.
As a person skilled in the art can readily understand details of
each step shown in FIG. 3 after reading above paragraphs, further
description is omitted here for brevity.
[0080] In a third exemplary design, when a compression mode of the
camera module 102 is enabled and a de-compression mode of the ISP
104 is enabled, the camera controller 112 refers to the compression
ratio M to configure a behavior of the output interface 114 during
a blanking period between adjacent data transmissions, and the ISP
controller 122 refers to the compression ratio M to configure a
behavior of the input interface 124 during a blanking period
between adjacent data transmissions, wherein the number of data
lines enabled over the camera interface 103 and the operating
frequency of each data line remain unchanged regardless of the
configured behavior of the output interface 114 and the configured
behavior of the input interface 124.
[0081] Please refer to FIG. 4, which is a diagram illustrating the
change of the blanking period after the interface compression is
applied. When no interface compression is applied (i.e., the
un-compressed multimedia data D1 is transmitted over the camera
interface 103 and the un-compressed multimedia data D2 is received
from the camera interface 103), the banking period is generally
shorter than the data transmission period. However, when interface
compression is applied (i.e., the compressed multimedia data D1' is
transmitted over the camera interface 103 and the compressed
multimedia data D2' is received from the camera interface 103), the
banking period is extended due to a shorter data transmission
period. In other words, as the compression ratio M is smaller than
1 (e.g., M=0.5), the interface compression is capable of creating
an extra blanking period. In one exemplary design, the extra
blanking period may allow negotiation between the camera module 102
and the ISP 104 so as to alleviate electrostatic discharge (ESD),
EMI and/or power consumption. For example, during the blanking
period extended by the interface compression, the data transmission
may be turned off and/or additional commands may be sent from the
camera module 102 to the ISP 104. To put it simply, the bandwidth
can be reduced by the interface compression, and the extended
blanking period is related to the compression ratio M.
[0082] Please refer to FIG. 5, which is a flowchart illustrating a
third control and data flow of the data processing system 100 shown
in FIG. 1. In this embodiment, the compression ratio M is
referenced to configure the behavior of the output interface 114
and the input interface 124 during the blanking period, and the
number of data lines enabled over the camera interface 103 and the
operating frequency of each data line remain unchanged. Provided
that the result is substantially the same, the steps are not
required to be executed in the exact order shown in FIG. 5. The
exemplary third control and data flow may be briefly summarized by
following steps.
[0083] Step 500: Start.
[0084] Step 502: Check if a compression mode is enabled. If yes, go
to step 508; otherwise, go to step 504.
[0085] Step 504: The other circuitry 116 generates the
un-compressed multimedia data D1 according to an input multimedia
data obtained by a camera sensor.
[0086] Step 506: The packing unit 117 directly packs/packetizes the
un-compressed multimedia data D1 into an output bitstream. Go to
step 514.
[0087] Step 508: The other circuitry 116 generates the compressed
multimedia data D1' according to the input multimedia data obtained
by the camera sensor.
[0088] Step 510: The camera controller 112 refers to the
compression ratio M corresponding to the compressed multimedia data
D1' to configure the behavior of the output interface 114 during
the blanking period between adjacent data transmissions.
[0089] Step 512: The packing unit 117 packs/packetizes the
compressed multimedia data D1' into an output bitstream.
[0090] Step 514: Transmit the output bitstream over the camera
interface 103.
[0091] Step 516: Check if a de-compression mode is enabled. If yes,
go to step 522; otherwise, go to step 518.
[0092] Step 518: The un-packing unit 127 un-packs/un-packetizes an
input bitstream into the un-compressed multimedia data D2.
[0093] Step 520: The other circuitry 126 generates a processed
multimedia data according to the un-compressed multimedia data D2.
Go to step 528.
[0094] Step 522: The ISP controller 122 refers to the compression
ratio M corresponding to the compressed multimedia data D1' to
configure the behavior of the input interface 214 during the
blanking period between adjacent data transmissions.
[0095] Step 524: The un-packing unit 127 un-packs/un-packetizes the
input bitstream into the compressed multimedia data D2'.
[0096] Step 525: The other circuitry 126 derives a de-compressed
multimedia data from de-compressing the compressed multimedia data
D2'.
[0097] Step 526: The other circuitry 126 generates the processed
multimedia data according to the de-compressed multimedia data.
[0098] Step 528: End.
[0099] It should be noted that steps 502-514 are performed by the
camera module 102, and steps 516-526 are performed by the ISP 104.
As a person skilled in the art can readily understand details of
each step shown in FIG. 5 after reading above paragraphs, further
description is omitted here for brevity.
[0100] The camera interface 103 may have at least one data line
which is a bidirectional line. For example, the data line DL0 as
shown in FIG. 1 is bidirectional. Hence, information transaction
between the camera module 102 and the ISP 104 can be realized by
using the bidirectional data line DL0. In this embodiment of the
present invention, the camera controller 112 may further check a
de-compression capability of the ISP 104 by sending a request to
the ISP 104, and the ISP controller 122 may further inform the
camera module 102 of the de-compression capability of the ISP 104
by sending a response to the camera module 102. In this way, the
camera module 102 can detect whether the ISP 104 has the ability of
performing data de-compression, and further detect what kinds of
de-compression algorithms the ISP 104 supports if the ISP 104 is
equipped with de-compression capability. In a case where the ISP
104 supports the interface compression, the ISP 104 may enable the
de-compression mode after transmitting the de-compression
capability information in response to the request issued by the
camera module 102. Next, the camera module 102 enables the
compression mode according to a checking result of the
de-compression capability of the ISP 104, and configures the
compressor in the other circuitry 116 to employ one of compression
algorithms corresponding to de-compression algorithms supported by
the ISP 104. However, when the ISP 104 does not support the
interface compression, the ISP 104 simply operates under the
non-decompression mode, and the camera module 102 would enable the
non-compression mode according to the checking result of the
de-compression capability of the ISP 104.
[0101] Please refer to FIG. 6, which is a flowchart illustrating a
method of setting the camera module 102 by referring to a result of
checking a de-compression capability of the ISP 104 according to an
embodiment of the present invention. Provided that the result is
substantially the same, the steps are not required to be executed
in the exact order shown in FIG. 6. The exemplary method may be
briefly summarized by following steps.
[0102] Step 600: Start.
[0103] Step 602: Check a de-compression capability of the ISP 104
to see if the ISP 104 supports the proposed interface compression,
and accordingly obtain a checking result.
[0104] Step 604: Refer to the checking result to determine if the
de-compression mode in the ISP 104 is set? If yes, go to step 606;
otherwise, go to step 608.
[0105] Step 606: Enable the compression mode in the camera module
102. Go to step 610.
[0106] Step 608: Enable the non-compression mode in the camera
module 102.
[0107] Step 610: End.
[0108] As a person skilled in the art can readily understand
details of each step shown in FIG. 6 after reading above
paragraphs, further description is omitted here for brevity.
[0109] Those skilled in the art will readily observe that numerous
modifications and alterations of the device and method may be made
while retaining the teachings of the invention. Accordingly, the
above disclosure should be construed as limited only by the metes
and bounds of the appended claims.
* * * * *