U.S. patent application number 16/542411 was filed with the patent office on 2020-12-17 for method and system for improving efficiency of optical camera communication.
The applicant listed for this patent is Wipro Limited. Invention is credited to Subhas Chandra Mondal, Shailesh Prabhu.
Application Number | 20200396377 16/542411 |
Document ID | / |
Family ID | 1000004378781 |
Filed Date | 2020-12-17 |
![](/patent/app/20200396377/US20200396377A1-20201217-D00000.png)
![](/patent/app/20200396377/US20200396377A1-20201217-D00001.png)
![](/patent/app/20200396377/US20200396377A1-20201217-D00002.png)
![](/patent/app/20200396377/US20200396377A1-20201217-D00003.png)
![](/patent/app/20200396377/US20200396377A1-20201217-D00004.png)
![](/patent/app/20200396377/US20200396377A1-20201217-D00005.png)
![](/patent/app/20200396377/US20200396377A1-20201217-D00006.png)
![](/patent/app/20200396377/US20200396377A1-20201217-D00007.png)
![](/patent/app/20200396377/US20200396377A1-20201217-D00008.png)
![](/patent/app/20200396377/US20200396377A1-20201217-D00009.png)
![](/patent/app/20200396377/US20200396377A1-20201217-D00010.png)
View All Diagrams
United States Patent
Application |
20200396377 |
Kind Code |
A1 |
Mondal; Subhas Chandra ; et
al. |
December 17, 2020 |
METHOD AND SYSTEM FOR IMPROVING EFFICIENCY OF OPTICAL CAMERA
COMMUNICATION
Abstract
A method and a system for improving efficiency of a
bi-directional optical camera communication between a first OCC
device and a second OCC device are disclosed. In an embodiment, the
method may include estimating, by a first OCC device comprising a
first display device and a first camera device, a first
transmission parameter with respect to the first display device and
a first reception parameter with respect to the first camera
device. The first reception parameter is based on a second
transmission parameter of a second OCC device or a channel
condition parameter, and the second transmission parameter is
derived from an analysis of information captured by the second OCC
device from the first OCC device. The method may further include
dynamically modifying, by the first OCC device, the first
transmission parameter based on the first reception parameter in
order to maximize throughput of OCC.
Inventors: |
Mondal; Subhas Chandra;
(Bangalore, IN) ; Prabhu; Shailesh; (Manipal,
IN) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Wipro Limited |
Bangalore |
|
IN |
|
|
Family ID: |
1000004378781 |
Appl. No.: |
16/542411 |
Filed: |
August 16, 2019 |
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
H04N 5/235 20130101;
H04N 5/23227 20180801; H04N 5/23232 20130101 |
International
Class: |
H04N 5/232 20060101
H04N005/232; H04N 5/235 20060101 H04N005/235 |
Foreign Application Data
Date |
Code |
Application Number |
Jun 17, 2019 |
IN |
201941024035 |
Claims
1. A method of improving efficiency of a bi-directional optical
camera communication (OCC) between a first OCC device and a second
OCC device, the method comprising: estimating, by a first OCC
device comprising a first display device and a first camera device,
at least one first transmission parameter with respect to the first
display device and at least one first reception parameter with
respect to the first camera device, wherein the at least one first
reception parameter is based on at least one second transmission
parameter of a second OCC device or at least one channel condition
parameter, and wherein the at least one second transmission
parameter is derived from an analysis of information captured by
the second OCC device from the first OCC device, wherein the at
least one first transmission parameter comprises a frame rate
parameter of the first display device and a block size of a
transmission packet to be displayed by the first display device,
wherein the frame rate parameter and the block size are dynamically
varied to obtain a dynamic rate adaptation; and dynamically
modifying, by the first OCC device, the at least one first
transmission parameter based on the at least one first reception
parameter in order to maximize throughput of OCC.
2. The method of claim 1, further comprising estimating and
modifying frame rate parameter of the first display device based on
the at least one first reception parameter, wherein the at least
one first reception parameter is determined upon successful
reception, by the first camera device, of an OCC packet transmitted
from the second OCC device.
3. The method of claim 1, further comprising estimating and
modifying the block size of a transmission packet to be displayed
by the first display device based on a pre-determined specification
of an application transmitting data.
4. The method of claim 3, wherein modifying the block size further
comprises creating contiguous memory blocks having block size of a
predetermined denomination.
5. The method of claim 1, further comprising performing version
controlled re-transmission (VCR) from the first display device upon
receiving negative acknowledgement (NACK) signal from the first
camera device based on a comparison of a NACK score with a
threshold NACK score.
6. The method of claim 5, further comprising at least one of:
optimizing a number of retransmission frames in the transmit buffer
based on a packet loss error rate (PLER); and evaluating a validity
of the re-transmission based on the PLER.
7. The method of claim 1, further comprising at least one of:
modifying brightness of the first display device; modifying color
space of the first display device; modifying a number of symbols to
be transmitted per frame by the first display device; and modifying
a resolution of a receive frame captured by the first camera
device.
8. The method of claim 1, wherein the at least one channel
condition parameter comprises ambient light of an environment, and
wherein dynamically modifying the at least one first transmission
parameter is further based on the ambient light.
9. The method of claim 1, wherein the first display device
comprises at least one of: a screen based display device configured
to display an image or a pattern; and a set of spatially arranged
Light Emitting Diodes (LEDs) configured to display a pattern.
10. The method of claim 9, wherein the image or the pattern is
placed inside a colored boundary.
11. The method of claim 1, further comprising creating a knowledge
building model to predict the at least one first transmission
parameter based on historical knowledge.
12. A system for improving efficiency of a bi-directional optical
camera communication (OCC) between a first OCC device and a second
OCC device, the system comprising: a first OCC device comprising a
first display device, a first camera device, a processor, and a
computer-readable medium storing instructions that, when executed
by the processor, cause the at least one processor to perform
operations comprising: estimating at least one first transmission
parameter with respect to the first display device and at least one
first reception parameter with respect to the first camera device,
wherein the at least one first reception parameter is based on at
least one second transmission parameter of a second OCC device or
at least one channel condition parameter, and wherein the at least
one second transmission parameter is derived from an analysis of
information captured by the second OCC device from the first OCC
device, wherein the at least one first transmission parameter
comprises a frame rate parameter of the first display device and a
block size of a transmission packet to be displayed by the first
display device, wherein the frame rate parameter and the block size
are dynamically varied to obtain a dynamic rate adaptation; and
dynamically modifying, by the first OCC device, the at least one
first transmission parameter based on the at least one first
reception parameter in order to maximize throughput of OCC.
13. The system of claim 12, wherein the at least one first
transmission parameter comprises the frame rate parameter of the
first display device, and wherein the at least one first reception
parameter is determined upon successful reception, by the first
camera device, of an OCC packet transmitted from the second OCC
device.
14. The system of claim 12, wherein the at least one first
transmission parameter comprises the block size of a transmission
packet to be displayed by the first display device, and wherein the
block size of the transmission packet is dynamically modified based
on a pre-determined specification of an application transmitting
data.
15. The system of claim 12, wherein the operations further comprise
performing version controlled re-transmission (VCR) from the first
display device upon receiving negative acknowledgement (NACK)
signal from the first camera device based on a comparison of a NACK
score with a threshold NACK score.
16. The system of claim 12, wherein the operations further comprise
at least one of: modifying brightness of the first display device;
modifying color space of the first display device; modifying a
number of symbols to be transmitted per frame by the first display
device; and modifying a resolution of a receive frame captured by
the first camera device.
17. The system of claim 12, wherein the at least one channel
condition parameter comprises ambient light of an environment, and
wherein dynamically modifying the at least one first transmission
parameter is further based on the ambient light.
18. The system of claim 12, wherein the first display device
comprises at least one of: a screen based display device configured
to display an image or a pattern; and a set of spatially arranged
Light Emitting Diodes (LEDs) configured to display a pattern.
19. The system of claim 18, wherein the image or the pattern is
placed inside a colored boundary.
20. A non-transitory computer-readable medium storing
computer-executable instructions for improving efficiency of a
bi-directional optical camera communication (OCC) between a first
OCC device and a second OCC device, the non-transitory
computer-readable medium configured for: estimating at least one
first transmission parameter with respect to a first display device
and at least one first reception parameter with respect to a first
camera device, wherein the first camera device and the first
display device are a part of the first OCC device, wherein the at
least one first reception parameter is based on at least one second
transmission parameter of a second OCC device or at least one
channel condition parameter, and wherein the at least one second
transmission parameter is derived from an analysis of information
captured by the second OCC device from the first OCC device,
wherein the at least one first transmission parameter comprises a
frame rate parameter of the first display device and a block size
of a transmission packet to be displayed by the first display
device, wherein the frame rate parameter and the block size are
dynamically varied to obtain a dynamic rate adaptation; and
dynamically modifying the at least one first transmission parameter
based on the at least one first reception parameter in order to
maximize throughput of OCC.
Description
TECHNICAL FIELD
[0001] This disclosure relates generally to optical camera
communication (OCC), and more particularly to a method and a system
for improving efficiency of a bi-directional OCC.
BACKGROUND
[0002] There is a growing need for an alternate wireless
communication medium to augment the network capacity provided by
the Radio Frequency (RF) spectrum. Visible light is one such medium
that can double up as a data communication medium. Thus, visible
light communication (VLC) or optical wireless communication (OWC)
involve communication over a light channel by transmit-receive
pair. The transceiver pair can transmit information using a Light
Emitting Diode (LED) and receive using a light sensor such as a
photo diode or a camera. For example, in some cases, the VLC/OWC
may take the form of light fidelity (Li-Fi) technology that employs
LED as the transmitter and a photodetector as the receiver. The
adoption of Li-Fi technology is slow or limited due to the fact
that the standard devices need to have a built in Li-Fi
transceiver. Alternatively, in some cases, the VLC/OWC may take the
form of optical camera communication (OCC) technology that employs
screen or spatially arranged LEDs as a transmitter and camera as a
receiver. Thus, OCC may be enabled with a software application
running on a standard device, without any need for hardware
changes.
[0003] In OCC, a camera based receiver is involved in receiving the
transmitted information and decoding the received information,
while a screen based transmitter or a spatially arranged LEDs based
transmitter is involved in encoding information and transmitting
the encoded information. The information may be encoded in form of
an image or a spatially arranged color codes that may then be
transmitted in frames. The received frames (i.e., captured image or
color codes) may then processed to extract the encoded information.
For example, in a screen based transmission scheme, a set of color
patterns may be used to encode the information. Quick Response (QR)
code based communication that displays QR codes in a screen is an
example of spatially arranged coding scheme used in OCC. In such
scheme, the information to be transmitted may be broken up in
smaller blocks to generate a sequence of QR code. Each QR code is
displayed for a small time-interval before the next QR code is
displayed. Camera capture rate determines the display interval for
a QR code. Similarly, for example, in LED based transmission
scheme, the information to be transmitted may be encoded by color
codes emitted by a set of spatially arranged LEDs. In such case,
the color emitted by each LED may carry a number of bits from the
information block depending on the number of colors that may be
decoded by the receiving camera. If more number of colors are used
to encode the information, the overall throughput may increase.
However, the receiving camera sensitivity, distance between the
transceivers, background illumination and media condition play an
important role in determining how many colors may be used to encode
the transmitted information.
[0004] In both the forms of OCC mechanisms explained above (i.e.,
screen based transmission and LED based transmission), the
receiving camera may take a picture of the frame as seen through
the free space between the transmit source and the camera.
Depending on the distance between the transmit and receive pair and
the optical resolution of the camera, the size of the received
image of the transmit frame may vary. A transmit frame is a spatial
arrangement of light sources of distinct colors. The size of the
received image of the transmit frame may determine the ability to
distinguish the colors in the spatial arrangement which in turn may
determine the ability to decode the information correctly. If there
is information loss in the received image resulting in the
incorrect decoding of the received information, the overall system
throughput will decrease. Information loss may be due to long
distance, low camera resolution, background illumination, and
opaqueness of the media between the transmitter and the
receiver.
[0005] In other words, in absence of rate adaptation, the system
throughput may be sub-optimal under different conditions. Thus,
conventional techniques are limited with respect to employing a
mechanism of rate adaptation in the context of OCC to increase the
system throughput. More generally, current techniques fail to
adaptively adjust transmit-receive parameters to maximize the
system throughput.
SUMMARY
[0006] In one embodiment, a method of improving efficiency of a
bi-directional optical camera communication (OCC) is disclosed. In
one example, the method may include estimating, by a first OCC
device comprising a first display device and a first camera device,
at least one first transmission parameter with respect to the first
display device and at least one first reception parameter with
respect to the first camera device, The at least one first
reception parameter is based on at least one second transmission
parameter of a second OCC device or at least one channel condition
parameter, while the at least one second transmission parameter is
derived from an analysis of information captured by the second OCC
device from the first OCC device. The method may further include
dynamically modifying, by the first OCC device, the at least one
first transmission parameter based on the at least one first
reception parameter in order to maximize throughput of OCC,
[0007] In another embodiment, a system for improving efficiency of
a bi-directional OCC is disclosed. In one example, the system may
include a first OCC device comprising a first display device, a
first camera device, a processor, and a computer-readable medium
communicatively coupled to the processor. The computer-readable
medium may store processor-executable instructions, which, on
execution, may cause the processor to estimate at least one first
transmission parameter with respect to the first display device and
at least one first reception parameter with respect to the first
camera device. The at least one first reception parameter is based
on at least one second transmission parameter of a second OCC
device or at least one channel condition parameter, while the at
least one second transmission parameter is derived from an analysis
of information captured by the second OCC device from the first OCC
device. The processor-executable instructions, on execution, may
further cause the processor to dynamically modify the at least one
first transmission parameter based on the at least one first
reception parameter in order to maximize throughput of OCC.
[0008] In yet another embodiment, a non-transitory
computer-readable medium storing computer-executable instructions
for improving efficiency of a bi-directional OCC is disclosed. In
one example, the stored instructions, when executed by a processor,
may cause the processor to perform operations including estimating
at least one first transmission parameter with respect to a first
display device and at least one first reception parameter with
respect to a first camera device. The first camera device and the
first display device are a part of the first OCC device.
Additionally, the at least one first reception parameter is based
on at least one second transmission parameter of a second OCC
device or at least one channel condition parameter, while the at
least one second transmission parameter is derived from an analysis
of information captured by the second OCC device from the first OCC
device. The operations may further include dynamically modifying
the at least one first transmission parameter based on the at least
one first reception parameter in order to maximize throughput of
OCC.
[0009] It is to be understood that both the foregoing general
description and the following detailed description are exemplary
and explanatory only and are not restrictive of the invention, as
claimed.
BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The accompanying drawings, which are incorporated in and
constitute a part of this disclosure, illustrate exemplary
embodiments and, together with the description, serve to explain
the disclosed principles.
[0011] FIG. 1 illustrates a typical system for implementing optical
camera communication (OCC) between a first OCC device and a second
OCC device.
[0012] FIG. 2 illustrates an exemplary system for implementing OCC
between a first OCC device and a second OCC device, in accordance
with some embodiments of the present disclosure,
[0013] FIG. 3 is a functional block diagram of a rate adaptation
control module, in accordance with some embodiments of the present
disclosure.
[0014] FIG. 4 illustrates a multi-symbol transmission per Quick
Response (QR) code frame and a single-symbol transmission per QR
code frame, in accordance with some embodiments of the present
disclosure.
[0015] FIG. 5 is an exemplary graphical representation showing
variation of throughput with respect to error rate, in accordance
with some embodiments of the present disclosure.
[0016] FIG. 6 is an exemplary tabular result of packet error rate
estimation for multi-symbol transmission and single-symbol
transmission per frame, in accordance with some embodiments of the
present disclosure.
[0017] FIG. 7 is a flow diagram of an exemplary method of Signal to
Noise Ratio (SNR) control mechanism, in accordance with some
embodiments of the present disclosure.
[0018] FIG. 8 illustrates a color matrix based transmit frame, in
accordance with some embodiments of the present disclosure.
[0019] FIG. 9 illustrates a single-symbol transmission per frame
and a multi-symbol transmission per frame, in accordance with some
embodiments of the present disclosure.
[0020] FIG. 10 is an exemplary information about Central Processing
Unit (CPU) architecture from Iscpu command, in accordance with some
embodiments of the present disclosure.
[0021] FIG. 11 illustrates an exemplary htop Graphical User
Interface (GUI), in accordance with some embodiments of the present
disclosure.
[0022] FIG. 12 illustrates an exemplary htop GUI for assigning of
cores, in accordance with some embodiments of the present
disclosure.
[0023] FIG. 13 illustrates an exemplary transmission mechanism with
respect to modification in the block size, in accordance with some
embodiments of the present disclosure.
[0024] FIG. 14 illustrates an exemplary transmission mechanism with
respect to modification in the frame rate, in accordance with some
embodiments of the present disclosure.
[0025] FIG. 15 is a flow diagram of an exemplary method for fixing
the block size, in accordance with some embodiments of the present
disclosure.
[0026] FIG. 16 is a flow diagram of an exemplary method for fixing
a frame rate, in accordance with some embodiments of the present
disclosure.
[0027] FIG. 17 illustrates an exemplary transmit area of 8*8 LED
matrix, in accordance with some embodiments of the present
disclosure.
[0028] FIG. 18 illustrates an exemplary transmit area of 8*8 LED
matrix (re-oriented LEDs), with the block size reduced to 4*4
active LED matrix, in accordance with some embodiments of the
present disclosure.
[0029] FIG. 19 illustrates an exemplary transmission mechanism for
fixing the block size, in accordance with some embodiments of the
present disclosure.
[0030] FIG. 20 illustrates an exemplary process of reducing
information rate, in accordance with some embodiments of the
present disclosure.
[0031] FIG. 21 is a flow diagram of a version-controlled
re-transmission, in accordance with some embodiments of the present
disclosure.
[0032] FIG. 22 is a block diagram of an exemplary receiver, in
accordance with some embodiments of the present disclosure.
[0033] FIG. 23 illustrates a sequence of ambient illumination
detection, in accordance with some embodiments of the present
disclosure.
[0034] FIG. 24 illustrates an exemplary transmit frame with
surrounding "RED" border, in accordance with some embodiments of
the present disclosure.
[0035] FIG. 25 is a flow diagram of an exemplary method of
improving SNR by transmit frame skew correction, in accordance with
some embodiments of the present disclosure.
[0036] FIG. 26 is a flow diagram of an exemplary method of transmit
frame magnification, in accordance with some embodiments of the
present disclosure.
[0037] FIG. 27 illustrates an arrangement of contiguous memory
blocks with pointers, in accordance with some embodiments of the
present disclosure.
[0038] FIG. 28 is a flow diagram of a method of improving
efficiency of a bi-directional OCC between a first OCC device and a
second OCC device, in accordance with some embodiments of the
present disclosure.
DETAILED DESCRIPTION
[0039] Exemplary embodiments are described with reference to the
accompanying drawings. Wherever convenient, the same reference
numbers are used throughout the drawings to refer to the same or
like parts. While examples and features of disclosed principles are
described herein, modifications, adaptations, and other
implementations are possible without departing from the spirit and
scope of the disclosed embodiments. It is intended that the
following detailed description be considered as exemplary only,
with the true scope and spirit being indicated by the following
claims. Additional illustrative embodiments are listed below.
[0040] FIG. 1 illustrates a typical optical camera communication
(OCC) system 100 for implementing a bi-directional OCC between a
first OCC device 102 and a second OCC device 104. The communication
system 100 may include a first OCC device 102 and a second OCC
device 104. The first OCC device 102 may include a first camera 106
and a first display device 108. The second OCC device 104 may
include a second camera 118 and a second display device 120. Each
of the display devices 108 and 120 may be a screen based display or
spatially arranged LED Matrix, capable of transmitting visual
content. The first camera 106 and the second display device 120 may
together form a first transmission-reception system. Similarly, the
second camera 118 and the first display device 108 together form a
second transmission-reception system. The first OCC device 102 may
further include a first decoder 110 and a first encoder 112.
Similarly, the second OCC device 104 may include a second decoder
122 and a second encoder 124. Application receiving data 114 and
126 and application transmitting data 116 and 128 may interface
with the transmitter-receiver systems. An application may interact
with the transmission system for sending the data, while the
application mat interact with the receiver system for receiving the
data.
[0041] The first OCC device 102 may receive application
transmitting data 116 which may further be encoded by the first
encoder 112. This encoded information may be then transmitted to a
receiver via a visual content in the form of a transmit frame
displayed by the first display device 108. Each of the first
display device 108 and the second display device 120 may be a
screen based display or spatially arranged LED Matrix. The second
camera 118 (receiver camera) may capture the image of this transmit
frame and decode the encoded information in the decoder 122. The
decoded information may be post processed in an application
receiving data 126.
[0042] Referring now to FIG. 2, an exemplary OCC system 200 for
implementing OCC between a first OCC device 202 and a second OCC
device 204 is illustrated, in accordance with an embodiment of the
present disclosure. The first OCC device 202 may include a camera
206, a display device 208, a decoder 210, a closed loop feedback
module 212, an encoder 214, a transmit control module 216, and a
rate adaptation control module 218. Corresponding to the first OCC
device 202, the second OCC device 204 may include a camera 224, a
display device 226, a decoder 228, a closed loop feedback module
230, an encoder 232, a transmit control module 234, and a rate
adaptation control module 236. The rate adaptation control modules
218 and 236 are further explained in conjunction with FIG. 3.
[0043] Referring now to FIG. 3, a rate adaptation control module
300 (corresponding to rate adaptation control modules 218 and 236)
is illustrated in accordance with an embodiment. The rate
adaptation control module 300 may include a frame-rate estimating
module 302, a frame-rate adaptation control module 304, a rate
adaptation control module 306, a display brightness adaptation
control module 308, a block size estimating module 310, a buffer
312, and a block size adaptation control module 314.
[0044] The frame-rate estimating module 302 may estimate frame rate
parameter of the transmitter. Depending on the successful reception
of the packets at a receiver, a transmitter application may require
the frame-rate estimating module 302 to increase or decrease the
frame rate of the transmitter, through the frame-rate adaptation
control module 304. The frame-rate adaptation control module 304
may be assisted by the frame-rate estimating module 302 to
adaptively control the frame rate of the transmitter by
communicating with the transmitter application. The frame-rate
adaptation control module 304 may carry out Version Controlled
Re-Transmission (VCR) whenever there is a Negative Acknowledgment
(NACK) from the receiver, when a NACK score (i.e., channel quality
indication) is more than a threshold value [NT]. The frame-rate
adaptation control module 304 may, thereby, monitor Packet Loss
Error Rate (PLER) and optimize a number of retransmission frames in
the transmit buffer.
[0045] The display brightness adaptation control module 308 may
address power concerns in the screen-based communication. The
display brightness adaptation control module 308 may control
display brightness of the screen-based transmitter and may set
optimal brightness level required by the transmitter. The block
size estimating module 310 may estimate size of a required block
size. Buffer 312 may provide the application with a set of data
fragments with the block size specified by the application. The
buffer 312 may include data fragments of variable length. Once the
block size estimating module 310 sends the block size length
required, the buffer 312 may return a set of fragments with the
size specified in the block size length, which may then be
forwarded to the transmitter application, through the block size
adaptation control module 314. The buffer 312 may store transmit
information. The buffer 312 may further provide the same to the
block size estimating module 310, whenever required. The block size
adaptation control module 314 may assist the application in
adaptively controlling the block size by communicating with the
transmitter application.
[0046] It should be noted that all such aforementioned modules may
be implemented in programmable hardware devices such as
programmable gate arrays, programmable array logic, programmable
logic devices, or the like. Alternatively, all such aforementioned
modules may be implemented in software for execution by various
types of processors. An identified module of executable code may,
for instance, include one or more physical or logical blocks of
computer instructions which may, for instance, be organized as an
object, procedure, function, or other construct. Nevertheless, the
executables of an identified module need not be physically located
together but may include disparate instructions stored in different
locations which, when joined logically together, comprise the
module and achieve the stated purpose of the module. Indeed, a
module of executable code could be a single instruction, or many
instructions, and may even be distributed over several different
code segments, among different applications, and across several
memory devices.
[0047] Further, it may be understood that the display devices 208
and 226 may act as transmitters. These transmitters may be screen
based, or a set of spatially arranged LEDs. In a screen based
transmission scheme, a set of color patterns may be used to encode
the information. In LED based transmission scheme, the information
to be transmitted may be encoded by color codes emitted by a set of
spatially arranged LEDs. It may be further understood that the
cameras 206 and 224 may act as receivers. The cameras 206 and 224
may take a snap of free space between the transmit source and the
camera, and may preserve the spatial information. This spatial
information may be used to reconstruct what has been transmitted.
The receiver system may also include a photodetector for ambient
light measurement.
[0048] In some embodiments, the transmitter-receiver pairs may
include a Quick Response (QR) code based rate adaptation, such as a
screen-based QR code at the transmitter and camera at the receiver.
Further, the screen-based QR code may either be based on dynamic
transmit area, or fix transmit area. Further, the QR code based
rate adaptation may include a dot matrix LED based color QR code at
the transmitter, and a camera based receiver. In alternate
embodiments, the transmitter-receiver pairs may include a color
matrix based rate adaptation, based on spatial arrangement of LEDs
at the transmitter and camera based receiver.
Signal to Noise Ratio (SNP) Control
[0049] Case 1: QR Code Based Transmitter and Camera Based
Receiver
[0050] Referring now to FIG. 4, a multi-symbol transmission per QR
code frame 402 and a single-symbol transmission per QR code frame
404 is illustrated, in accordance with an embodiment. As will be
appreciated by those skilled in the art, under unfavorable
conditions, carrying out a multi-symbol transmission per frame may
decrease error propensity of the transmission. As shown in FIG. 4,
a single QR code per frame containing a block of information 402 is
divided into 4 individual QR codes (402A, 402B, 402C, 402D) per
frame, containing the same amount of information as a whole. This
may decrease the error propensity by at least by 25%, depending on
the media conditions. It may be noted that another factor to be
considered is the processing time of the transmit frame at the
receiver. The processing time taken to decode four individual QR
codes in a single frame is more compared to a single QR code per
frame. Hence, the system throughput is limited by the computational
power at the receiver.
[0051] According to observation in an experimentation study,
multi-symbol transmission per frame produces higher throughput
compared to the single-symbol transmission per frame, till a
particular frame rate [F.sub.T]. Once the transmit frame rate goes
beyond this threshold, due to the processing time constraint at the
receiver side (in decoding the information in the transmit frame),
the single-symbol transmission per frame performs better than
multi-symbol transmission per frame. The results of this
experimentation study is depicted in FIG. 5. FIG. 5 is a graphical
representation 500 showing variation of throughput with respect to
error rate.
[0052] The experiment further estimated system constraint in terms
of the processing capability. Accordingly, during transmission, the
transmit frame rate may be continuously monitored. For lower frame
rates (<F.sub.T), each block of information may be encoded into
multiple QR codes in a single frame, to achieve higher throughput.
Once the transmit frame rate goes beyond F.sub.T, single-symbol
transmission may be carried out, by encoding a block of information
to a single QR code in a frame. The experiment of the performance
of single-symbol transmission and multi-symbol transmission under
ambient light scenario was carried out in following the
conditions:
TABLE-US-00001 Screen Resolution 1400*1050 pixels Block size 500
Bytes Distance between 60 cm screen based transmitter and camera
based receiver Testbed Processor used is Intel .RTM. Core .TM.
i5-3320M CPU @ 2.60 GHz with 4 cores, belonging to the Family 6
(Pentium Pro); Camera A laptop webcam with 1.3 MP resolution, at
the receiver end)
[0053] The results of the packet error rate estimation for
multi-symbol transmission and single-symbol transmission per frame
are illustrated via Table 1 shown in FIG. 6, Referring now to FIG.
6, the Table 1 shows that for a given testbed, till Frame Rate=7
fps (frames per second), multi-symbol transmission per frame
performs better than single-symbol transmission. After 7 fps, under
compute resource constraint scenario at the receiver side,
single-symbol transmission per frame performs better that
multi-symbol transmission per frame.
[0054] Referring now to FIG. 7, a flow diagram of a method 700 of
SNR control mechanism is illustrated, in accordance with an
embodiment. At step 702, a frame rate [F.sub.R] may be obtained
from the frame-rate estimating module 302. At step 704, a check may
be performed to determine if the obtained frame rate [F.sub.R] is
less than a threshold frame rate [F.sub.T]. If the obtained frame
rate [F.sub.R] is not less than the threshold frame rate [F.sub.T],
the method may proceed to the step 706 ("No" path). At step 706,
the information may be encoded to single symbol per frame. At step
710, transmission may be performed. Returning back to step 704, if
the obtained frame rate [F.sub.R] is determined to be less than the
threshold frame rate [F.sub.T], the method 700 may proceed to the
step 708 ("Yes" path). At step 708, the information may be encoded
into multiple symbols per frame. At step 710, once again, the
encoded information may be transmitted. The method 700 may then
once again proceed to step 702, and the process may be
repeated.
[0055] Case 2: Color Matrix based transmitter and Camera based
receiver.
[0056] Referring now to FIG. 8, a transmit frame 800 based on color
matrix based transmitter and camera based receiver is illustrated,
in accordance with an embodiment. The transmit frame 800 may
include a number of spatially arranged LEDs. The transmit frame 800
may be captured by a receiver camera and further processed to
decode the information. Each LED in the matrix may represent a
certain number of bits, depending on how many colors are present in
the encoding color set. For example, if the color set includes 16
colors, then each LED may represent 4 bits of information. In
general, a single LED may represent `n` bits of information, if the
color set consists of 2.sup.n colors. By way of example, as
illustrated, the transmit frame 800 includes 16 LEDs of 16
different colors including Red (RD), Light Green (LG), Blue (BL),
Pink (PK), Yellow (YL), Sky Blue (SB), brown (BR), Orange (OR),
Magenta (MG), maroon (MR), Dark Green (DG), Olive (OL), Grey (GY),
Purple (PR), Navy Blue (NB), and White (WH).
[0057] Referring now to FIG. 9, a single-symbol transmission color
matrix per frame 902 and a multi-symbol transmission color matrix
per frame 904 is illustrated. It may be noted that similar to Case
1, under unfavorable conditions, a single transmit frame including
one symbol may be divided into 4 symbols (904A, 904B, 904C, 904D).
At the receiver, the received frame may be divided into four
frames, and each individual frame may be decoded separately. Under
compute resource constraint scenario at the receiver side, for
higher frame rates, above a threshold Fr, single-symbol
transmission may be preferred, as explained in the Case 1.
Core Affinity
[0058] As will be appreciated by those skilled in that art, in
decoding a transmit frame including four symbols of 125 characters
each at the receiver, the individual symbols may be cropped from
the frame. The decoding may further include multi-threading, where
4 threads may be created, followed by decoding of each symbol. It
may be understood that this may cause load on the CPU. In such
cases, as per some embodiments, more than one cores may be assigned
to the process where the threads are running. This process may be
called CPU pinning. As per CPU pinning, a command Iscpu may give
information about the CPU architecture. An exemplary Iscpu
information 1000 is illustrated in FIG. 10.
[0059] The CPU architecture information may be gathered from sysfs
and /proc/cpuinfo. The information may further include a number of
CPUs, threads, cores, sockets, Non-Uniform Memory Access (NUMA)
nodes, etc. The information may further include information about
the CPU caches and cache sharing, family, model, bogoMIPS, byte
order, and stepping.
[0060] Once the number of cores present is determined, the cores
may be assigned to the process using command taskset. The taskset
command may be used to set or retrieve the CPU affinity of a
running process given its pid. Alternatively, the taskset command
may be used to launch a new command with a given CPU affinity. The
CPU affinity is a scheduler property that "bonds" a process to a
given set of CPUs on the system. An exemplary usage of the taskset
command is given below: [0061] taskset -cp 0,4 <pid>
[pid-process id] [0062] taskset -p 0x11 <pid> [11-00010001,
Hence enabling 0.sup.th and 4.sup.th core]
[0063] Referring now to FIG. 11 an exemplary htop GUI (view) is
shown. As shown in the FIG. 11, two threads may run per core.
Accordingly, for 4 threads operating at a time, it may be optimal
to assign 2 cores. As it will be appreciated, htop is an
interactive process viewer. It is similar to top, but may
additionally allow scrolling vertically and horizontally, thereby
allowing viewing of all the processes running on the system, along
with their full command lines. The htop may further enable viewing
of the processes as a process tree, selecting multiple processes,
and acting on them all at once. FIG. 12 shows an exemplary htop GUI
(view) for assigning of cores to the process. As shown in FIG. 3,
core 3 and core 4 may be assigned to the transmit frame decoding
process. It may be noted that using the above method, the unused
CPU cores are involved in the decoding process. As a result, the
CPU resources are utilized effectively, thereby improving the
efficiency of the system.
Transmission Rate Adaptation Mechanism
[0064] Case 1: OR Code Based Transmitter and Camera Based
Receiver
[0065] Referring now to FIG. 13 and FIG. 14, complete transmission
mechanisms are illustrated, in accordance with some embodiments.
FIG. 13 shows a complete transmission mechanism 1300 according to
fixing the block size. The frame rate is set to 1 fps (<F.sub.T)
by the frame rate estimating module 302, and the display brightness
is set to 50%. By way of an example, assuming the buffer may have 5
fragments to transmit, the block size estimating module 310 may fix
a block size to 100 characters. To decrease the error propensity,
at lower frame rates (<F.sub.T), multi-symbol transmission may
be carried out, as already explained. The transmit buffer may
return 5 fragments, setting the size of each fragment to 100 Bytes,
which may be then transmitted.
[0066] Upon reception, assuming the receiver is able to decode all
the fragments successfully, the receiver may send an ACK.
Thereupon, the transmitter may request the block size estimating
module 310 to increase the block size of the fragments. The buffer
may return fragments of size 200 Bytes, which may then be
transmitted. As long as the receiver sends ACK, the block size may
continuously increased.
[0067] By way of an example, assuming the current block size to be
600 Bytes, the receiver may send a NACK. Along with this NACK, the
receiver may further send a "NACK Score", indicating the Channel
Quality. If the NACK score is below a threshold (<N.sub.T), the
transmitter may request the display brightness adaptation control
module 308 to increase the brightness to 100%. This may increase
the optical resolution of the transmit frame. Even after this
action, if the NACK score doesn't improve and stays below the
threshold (<N.sub.T), then the previously recorded block size
for which an ACK was received may be fixed. On the other hand, if
the NACK score becomes greater than the threshold (>N.sub.T),
then the transmitter may request the rate adaptation control module
306 to carry out the VCR, which is explained in detail in the
following paragraphs.
[0068] After VCR, the rate adaptation control module 306 may send a
response to the transmitter indicating the Maximum Error Limit
(MEL) status [>MEL, <MEL]. If the response is >MEL, the
retransmission may be considered inefficient and the previously
recorded block size for which an ACK was received may be fixed. If
the response is <MEL, the retransmission may be considered
valid, and the transmitter may request for further increasing of
the block size of the data fragments.
[0069] Referring to FIG. 14, a transmission mechanism 1400
according to fixing the frame rate is illustrated, in accordance
with an embodiment. Once the block size is fixed (for example, at
500 Bytes), the transmitter may request for a gradual increase in
the frame-rate, in a similar fashion and may fix the frame-rate. It
may be noted that for lower frame-rates, multi-symbol transmission
may be carried out to decrease the error propensity, and for higher
frame-rate, single-symbol transmission may be carried out, as
already explained. It may be further noted that with this
mechanism, a dynamic rate adaptation can be achieved, by
dynamically varying the transmit parameters like block size and
frame-rate. Also, power consumption is taken care of by display
brightness adaptation control.
[0070] Referring now to FIG. 15, a flow diagram 1500 of a method
for fixing the block size is illustrated, in accordance with an
embodiment. At step 1502, transmit parameters may be set to
minimum. For example, block size may be set at 100 Bytes, and frame
rate may be set at 1 frame per second (FPS). At step 1504,
transmission may be carried out. At step 1506, a check may be
performed to determine if an acknowledgement (ACK) is received or
not. If the acknowledgement is received, the method may proceed to
step 1508 ("yes" path). At step 1508, the block size may be
incremented. The method may once again proceed to step 1502.
However, if at step 1506, the acknowledgement is not received, the
method may proceed to step 1510 ("No" path). At step 1510, a NACK
score may be obtained.
[0071] At step 1512, a check may be performed to determine if the
NACK score is less than threshold score NT or not. If the NACK
score is less than the threshold score, the method may proceed to
step 1514 ("Yes" path). At step 1514, a check may be performed to
determine if display brightness (DB) is 100% or not. If the display
brightness is 100%, the method may proceed to step 1522 ("Yes"
path). At step 1522, the current packet may be dropped, and a
previous block size may be fixed. However, if at step 1514, the
display brightness is not 100%, the method may proceed to step 1516
("No" path). At step 1516, the display brightness may be
incremented to 100%. The method may then proceed once again to step
1504, to carry out transmission.
[0072] Returning back to step 1512, if the NACK score is not less
than the threshold score, the method may proceed to step 1518. At
step 1518, version controlled re-transmission may be performed. At
step 1520, a check may be performed to determine whether packet
error rate is greater than MEL. If the packet error rate is greater
than the MEL, the method may proceed to step 1522, where the
current packet may be dropped, and a previous block size may be
fixed. However, if at step 1520, the packet error rate is not
greater than the MEL, the method may proceed to step 1508. At step
1508, the block size may be incremented. Thereafter, the method may
once again proceed to step 1502.
[0073] Referring now to FIG. 16, a flow diagram 1600 of a method
for fixing a frame rate is illustrated, in accordance with an
embodiment. At step 1602, after the bock size is fixed the frame
rate may be set at minimum. For example, the frame rate may be set
at 1 frame per second (FPS). At step 1604, transmission may be
carried out. At step 1606, a check may be performed to determine if
an acknowledgement (ACK) is received or not. If the acknowledgement
is received, the method may proceed to step 1608 ("yes" path). At
step 1608, the frame rate may be incremented. The method may once
again proceed to step 1602. However, if at step 1606, the
acknowledgement is not received, the method may proceed to step
1610 ("No" path). At step 1610, a NACK score may be obtained.
[0074] At step 1612, a check may be performed to determine if the
NACK score is less than threshold score NT or not. If the NACK
score is less than the threshold score, the method may proceed to
step 1614 ("Yes" path). At step 1614, a check may be performed to
determine if display brightness is 100% or not. If the display
brightness is 100%, the method may proceed to step 1622 ("Yes"
path). At step 1622, the current packet may be dropped, and a
previous frame rate may be fixed. However, if at step 1614, the
display brightness is not 100%, the method may proceed to step 1616
("No" path). At step 1616, the display brightness may be
incremented to 100%. The method may then proceed once again to step
1604, to carry out transmission.
[0075] Returning back to step 1612, if the NACK score is not less
than the threshold score, the method may proceed to step 1618. At
step 1618, version controlled re-transmission may be performed. At
step 1620, a check may be performed to determine whether packet
error rate is greater than MEL. If the packet error rate is greater
than the MEL, the method may proceed to step 1622, where the
current packet may be dropped, and a previous frame rate may be
fixed. However, if at step 1620, the packet error rate is not
greater than the MEL, the method may proceed to step 1608. At step
1608, the frame rate may be incremented. Thereafter, the method may
once again proceed to step 1602.
[0076] Case 2: Color Matrix Based Transmitter and Camera Based
Receiver
[0077] It may be noted that unlike in Case 1, in color matrix based
transmitter, the transmit area may be fixed. Accordingly, it may be
important to fix the orientation of the active LEDs, depending on
the media conditions. At the beginning of the transmission, the
block size estimating module 310 may set the block size to maximum.
Further, the frame rate may be set to minimum, thereby transmitting
a "TEST-FRAME". Depending on the successful reception and decoding
of information at the receiver, the placement of active LEDs in the
fixed transmit area may be decided.
[0078] Referring now to FIG. 17, a transmit area of 8*8 LED matrix
1700 is illustrated, in accordance with an embodiment. The
outermost layer of the LED matrix may be used as a boundary, by
assigning a color which the receiver camera may use to distinguish
the LEDs containing the data from the background. The remaining 6*6
LED matrix may be used for transmitting data [Active LEDs].
[0079] By way of an example, upon decoding the information, it may
be found that the lower part of the frame was not clearly captured.
In such example, the block size of the transmit frame may be
reduced, and the boundary may be shrunk in such a way that the
active LEDs containing data are present in the upper part of the
transmit frame. As shown in FIG. 18 (Active LEDs re-oriented), the
block size may be reduced to 4*4 active LED matrix 1800, with an
outer boundary, making the rest of the LEDs inactive. It may be
noted that with the orientation of the active LEDs known, the
transmission mechanism may be carried out.
[0080] Referring now to FIG. 19, an exemplary transmission
mechanism 1900 for fixing the block size is illustrated, in
accordance with an embodiment. In the Case 2 (using LED matrix),
since the transmit area is fixed, the block size may be kept
maximum, using all the LEDs in the matrix surrounded by the
boundary. This frame may be called "Test Frame", since the response
from the receiver may help the transmitter to determine the active
LEDs' further effective data transmission.
[0081] If the receiver sends an ACK, then the block size may be set
to maximum, in this example (6*6 active LEDs). If the receiver
sends a NACK, the NACK score may be first analyzed, If the NACK
Score is below the threshold (<N.sub.T), the transmitter may
request the display brightness adaptation control module 308 to
increase the brightness to 100% which increases the clarity of the
transmit frame and improves the decoding ability of the receiver.
Even after this action, if the NACK score doesn't improve and stays
below the threshold (<N.sub.T), then the previously recorded
block size for which an ACK was received may be fixed. On the other
hand, if the NACK score is greater than the threshold
(>N.sub.T), then the transmitter may request the rate adaptation
control module 306 to carry out the VCR,
[0082] If the VCR response is >MEL, the transmitter application
may request the block size estimating module 310 to decrease the
block size, making the active LEDs to 4*4. This procedure may be
repeated until the VCR sends a positive response [<MEL]. This
way, the block size is fixed in case of LED based transmitter.
Information Rate Control
[0083] Referring now to FIG. 20, a process 2000 of reducing
information rate is illustrated, in accordance with an embodiment.
The information rate may be reduced by reducing the color space and
transmitting the same color into multiple blocks of LEDs. If the
receiver camera is unable to distinguish the special separation
between the adjacent LEDs in the transmit frame 2002 even after the
camera magnification, the information rate may be decreased by
encoding the same color into blocks of LEDs, as shown in 2004. This
may enable the camera to distinguish the colors in the frame
easily.
[0084] By way of example, as illustrated, the transmit frame 2002
frame includes sixteen LEDs of different colors including Purple
(PR), Orange (OR), Yellow (YL), Green (GR), Sky Blue (SB), Blue
(BL), Navy Blue (NB), Purple (PR), Brown (BR), Cyan (CY), White
(WH), Black (BK), Light Blue (LB), Red (RD), Light Purple (LP), and
Dark Blue (DB). Further, the transmit frame 2004 with the
information rate decreased by encoding same color into blocks of
LEDs include sixteen LEDs including four Purple (PR), four Orange
(OR), four yellow (YL), and four Green (GR).
Version Controlled Retransmission
[0085] Referring now to FIG. 21, a flow diagram of a version
controlled re-transmission 2100 is illustrated, in accordance with
an embodiment. Once NACK is received from the receiver (Rx) along
with the Channel Quality Indication [CQI] or NACK score, the score
may be first evaluated. If the score is too less (below the
threshold [N.sub.T]), then the transmitter may request the block
size estimating module 310 to fix the previously recorded block
size, for which there was an ACK, and may further increase the
frame rate with constant block size. On the other hand, if the NACK
score is above the threshold, the transmitter may request the rate
adaptation control module 306 for a VCR.
[0086] To initiate the transmission, the transmitter may send first
version [Version 1] of the data fragments, for example, in the
order [1-2-3-4-5]. At the receiver, there may be a loss of
fragments and only [1-2] fragments may be received. Analysis of the
order of reception of the fragments and creation of a newer version
may be performed dynamically. In the previous version, as only the
first few fragments are received, in the next version, the order of
the fragments may be changed such that missing fragments may be
placed in the beginning indices, which may result in the order
[3-4-5-{}-{}]. As [1-2] are already received successfully in the
previous transmission, [1-2] may not be pushed into re-transmission
buffer in the newer version, so as to avoid redundancy. Instead,
the duty cycle of the missing fragments may be increased, which may
further increase the probability of successful retransmission.
[0087] The missing fragments that can be placed in the remaining
two indices may be either of [3, 4, 5]. Since [3,4] are in the
initial indices and are more probable to be successfully
transmitted (according to the version transmitted in the previous
transmission), the duty cycle of the fragment [5] may be increased,
to complete the version 2 of the transmit buffer [3-4-5-5-5]. This
version may not only promise successful transmission of [3-4], but
may also increase probability of successful transmission of [5]. If
the fragment [5] fails to be transmitted successfully, in the next
version, the lost fragment [5] may be placed in the initial index,
as well as in the remaining indices, which may look like
[5-5-5-5-5].
[0088] If there is a failure in further transmitting any of the
fragments, in the next version of the transmit buffer, the duty
cycle of the fragment may be increased and transmitted. Once this
version is sent, there may be a high chance of receiving missing
fragments successfully. Now, an ACK may be received from the
receiver, indicating that all the fragments have been successfully
transmitted and received. Once the ACK is received from the
receiver, a set of state parameters may be recorded. These set of
state parameters may include a number of retransmissions, buffer
size and frame rate at which it was transmitted. By way of an
example, the state parameters and their associated values may be as
follows:
TABLE-US-00002 Number of re-transmissions 3 Buffer size 5
[0089] It may be noted that for an efficient retransmission, packet
error rate should be minimum. To conclude on the validity of the
retransmission, first the packet error rate of the VCR may be
calculated. This packet error rate may be compared with MEL. If the
packet error rate is greater than MEL, the rate adaptation control
module 306 may send the ">MEL" signal to the transmitter, which
may indicate that the retransmission is not efficient for the
corresponding block size. The transmitter may then request the
block size estimating module 310 to fix the previously recorded
block size for which an ACK was received. On the other hand, if the
number of retransmissions is less than MEL, the rate adaptation
control module 306 may send the "<MEL" signal to the
transmitter, indicating that retransmission could be considered
valid. The transmitter may then request the block size estimating
module 310 to further increase the block size.
Maximum Error Limit (MEL)
[0090] The MEL may be set based on the channel quality indication
present in the NACK score. This score may indicate what packet
error rate is currently being faced, and may apply the same as the
MEL. The packet error rate of the overall VCR may be compared with
this MEL and then the retransmission validity may be evaluated.
[0091] For example, the transmission buffer may have five
fragments, 20 Bytes each. Accordingly, the transmission buffer may
now hold 100 Bytes, ready to be transmitted. After the
transmission, it is found that 20% of the packets is lost, giving a
NACK score of 80. thereby setting MEL to 20. This may trigger a VCR
with the lost fragments transferred to the re-transmission buffer.
If the packet error rate of the overall retransmission is within
20%, the re-transmission may be considered valid and the rate
adaptation control module 306 may send the "<MEL" signal to the
transmitter indicating that retransmission could be considered
valid. Further, the transmitter may then request the block size
estimating module 310 to further increase the block size. On the
other hand, if the packet error rate exceeds 20%, the rate
adaptation control module 306 may send the ">MEL" signal to the
transmitter, which indicates that the retransmission is not
efficient for the corresponding block size. The transmitter may
then request the block size estimating module 310 to fix the
previously recorded block size for which an ACK was received.
Ambient Illumination Detection
[0092] Referring now to FIG. 22, a block diagram of a receiver is
illustrated, in accordance with an embodiment. The receiver may
include a camera 2202, a receiver application 2204, and
photodetector 2206. It may be noted that a rate limiting mechanism
may estimate an illumination level in the media and control
transmit parameters. The channel conditions may be continuously
monitored and the gradient of the variation in the background
illumination may be determined. This gradient may be communicated
back to the transmitter to adapt the rate to dynamic channel
conditions.
[0093] FIG. 23 illustrates a sequence of ambient illumination
detection 2300, in accordance with an embodiment. The photodetector
2206 may periodically return the background illumination value to
the receiver application 2204. If this value exceeds beyond a
threshold, indicating the channel condition has worsened, the
receiver application 2204 may communicate this value back to the
transmitter. The transmitter may then slow down the rate, thereby
adapting to the media conditions.
Transmit Frame Identification
[0094] It may be noted that due to the varying distance between the
transmitter and the camera, the size of the transmit frame in the
receiver image may vary. When the distance increases, the size of
the transmit frame in the receiver image may decrease, thereby
deteriorating the ability of the camera to identify the object.
[0095] In accordance with some embodiments, the transmitted frame
may be correctly extracted by introducing a concept of guard band
in the form of a colored boundary. As a result the noise around the
transmit frame may be eliminated with minimum processing time. This
method may be implemented and tested with results for a
screen-based transmitter and camera-based receiver.
[0096] Case 1: QR Code Based Transmitter and Camera Based
Receiver
[0097] It may be noted that transmit frame identification of the QR
code based transmitter may be performed with finder patterns
present as a part of the QR code. The finder patterns may be
present at three corners of the QR code, of size 7.times.7 modules
each. Additionally, there may be a number of alignment patterns
depending on the version of the QR code, and timing patterns, which
may help the receiver camera to understand orientation and version
of the QR code transmitted.
[0098] Case 2: Color Matrix Based Transmitter and Camera Based
Receiver
[0099] In order to identify the transmit frame in the receiver
image, at the transmitter side, the transmit frame may be
surrounded with a border of "RED" color, which may serve as a guard
band, as shown in FIG. 24.
[0100] Referring now to FIG. 24, an exemplary transmit frame 2400
with surrounding "RED" border 2402 is illustrated. The border
containing red color may have HSV values [255, 0, 0]. In order to
extract the frame, the following operations may be performed on the
border:
[0101] (i) As any image is basically a 3D matrix, containing the
matrix for Red, Green and Blue, from the received image, the 1D
matrix of Red may be extracted and the other 2 matrices may be made
0.
[0102] (ii) The 1D matrix may include the red component value of
all the pixels in the image. In this 1D matrix, the red component
value corresponding to the border may be close to 255.
[0103] (iii) The image threshold operation may be performed,
limiting the value of the red component in this 1D matrix to
`220`.
[0104] (iv) The pixels with red component less than 220' may be
eliminated, thereby reducing most of the noise in the image.
[0105] (v) Once the noise around the transmit frame is eliminated,
edge detection may be performed on the resultant image, to extract
the transmit frame from the received image.
Improving SNR by Transmit Frame Skew Correction
[0106] Once the image of the transmit frame is captured by the
receiver camera, and the noise is eliminated using the guard band,
there may be signal impairment due to tilted camera capture. The
SNR may be improved by correcting the signal impairment caused due
to the deviated capture by the receiver camera with respect to the
transmitter.
[0107] Referring now to FIG. 25, a flow diagram 2500 of a method of
improving SNR by transmit frame skew correction is illustrated, in
accordance with an embodiment. It may be noted that the transmit
frame in the captured image may be skewed due to the tilted angle
of the capture by the receiver camera. Once the contours of the
transmit frame are extracted from shape detection, the contours may
be sent for skew correction.
[0108] At step 2502, contours of the object may be obtained using
shape detection. In some embodiments, 2D perspective transformation
may be performed on the object extracted. At step 2504, maximum
width may be determined using the 2 horizontal sides. The maximum
width of the object may be used to obtain the scaling factor. At
step 2506, maximum height may be determined using the 2 vertical
sides. At step 2508, the image may be scaled using the maximum
width and the maximum height.
Transmit Frame Magnification:
[0109] It may be noted that even by detecting transmit frame
boundary using image processing algorithms, information in the
transmit frame might not be decoded, This may be due to the
camera's inability to distinguish different colors in smaller size
of the transmit frame, or due to the reduced spatial separation of
the color patterns in the transmit frame. Therefore, before the
transmit frame boundary is arrested, the camera zoom-in feature may
be used to magnify the transmit frame, and increase optical
resolution.
[0110] Referring now to FIG. 26, a flow diagram 2600 of a method of
transmitting frame magnification is illustrated, in accordance with
an embodiment. At step 2602, height and width of the captured image
may be obtained. At step 2604, scaling factor for magnification
(scale) may be obtained. The scaling factor may decide the amount
of zooming of the receiver camera. At step 2606, radius of the area
may be scaled using this scaling factor. The following relation may
be applied:
Radius_x=Scale*height/100
Radius_y=Scale*width/100
[0111] At step 2608, the image be cropped using the estimated
scaled coordinates centre, radius_x, and radius_y.
Effective Utilization of Transmission Buffer
[0112] As will be appreciated, the efficiency of the transmission
process may be dependent on number of WRITE operations. The WRITE
operations may be expensive operations on the CPU time. A buffering
technique may be used to minimize the number of WRITE operations
for improving efficiency in the transmission process. In some
embodiments, contiguous memory blocks may be created with each
block having size of least denomination. One or more pointers may
be allocated to each block. As the transmission opportunity changes
due to change in the channel conditions, required block size may be
effectively delivered, that may improve performance of the
transmission process.
[0113] Referring now to FIG. 27, an arrangement of the contiguous
memory blocks with pointers 2700 is illustrated, in accordance with
an embodiment. As shown in the FIG. 27, one or more blocks of 100
Bytes may be created, assuming 100 Bytes is the lowest denomination
to the transmitted. Each block of 100B may be assigned a pointer
[P1, P2, P3, and so on]. If block size requirement is 100 Bytes,
then initial and final pointers will be P1 and P2, thereby
allocating 100 Bytes of information to the data encoder. If the
requirement changes to 200 Bytes, the initial and final pointers P1
and P3 may be returned.
Knowledge Building Model
[0114] As it will be appreciated, as the communication channel
conditions change dynamically, it may be necessary to decide on the
data rate intelligently, by execution of the adaptation algorithm.
Accordingly, as and when an ACK/NACK is received, a configuration
data may be noted and stored in a dataset. For example, the
database may include the parameters of frame rate, block size,
display brightness, ACK/NACK, NACK SCORE, and VCR Response
[<MEL/>MEL].
[0115] A model may be built and trained using this dataset. The
model may learn the configuration after a set of executions of the
algorithm. If the error rate increases and the NACK score
decreases, the algorithm may be executed all over again for
efficiency. In order to save time, the block size and frame rate
may be fixed based on the prediction by the model, using NACK Score
and VCR Response. The model may be used to predict distance between
the transmitter and the receiver.
[0116] It may be noted that the current model may be built at the
receiver, and may be applied in the context of historical knowledge
building model for rate adaptation at the access point, where
diverse type for users may be coming in the vicinity of
communication. By creating a knowledge bank based on the historical
adaptation rate (rather than a blind attempt), an initialization
vector can be created, based on the knowledge database.
[0117] As will be appreciated by one skilled in the art, a variety
of processes may be employed for improving efficiency of optical
camera communication (OCC) between OCC devices. For example, the
exemplary OCC system 200 and the associated rate adaptation control
module 300 may improve efficiency of a bi-directional OCC between a
first OCC device and a second OCC device by the processes discussed
herein. In particular, as will be appreciated by those of ordinary
skill in the art, control logic and/or automated routines for
performing the techniques and steps described herein may be
implemented by the OCC system 200 and the associated rate
adaptation control module 300, either by hardware, software, or
combinations of hardware and software. For example, suitable code
may be accessed and executed by the one or more processors on the
OCC system 200 to perform some or all of the techniques described
herein. Similarly, application specific integrated circuits (ASICs)
configured to perform some or all of the processes described herein
may be included in the one or more processors on the OCC system
200.
[0118] For example, referring now to FIG. 28, an exemplary process
2800 for improving efficiency of a bi-directional OCC between a
first OCC device and a second OCC device is illustrated, in
accordance with some embodiments of the present disclosure. As
illustrated, the process 2800 may include the step of estimating,
by the first OCC device comprising a first display device and a
first camera device, at least one first transmission parameter with
respect to the first display device and at least one first
reception parameter with respect to the first camera device at step
2802. It should be noted that the at least one first reception
parameter may be based on at least one second transmission
parameter of a second OCC device or at least one channel condition
parameter. Additionally, it should be noted that the at least one
second transmission parameter may be derived from an analysis of
information captured by the second OCC device from the first OCC
device. The process 2800 may further include the step of
dynamically modifying, by the first OCC device, the at least one
first transmission parameter based on the at least one first
reception parameter in order to maximize throughput of OCC at step
2804. In some embodiments. the process 2800 may further include the
step of creating a knowledge building model to predict the at least
one first transmission parameter based on historical knowledge.
[0119] In some embodiments, the at least one first transmission
parameter may include a frame rate parameter of the first display
device. In such embodiments, the process 2800 may include the steps
of estimating and modifying the frame rate parameter of the first
display device based on the at least one first reception parameter.
It should be noted that, in some embodiments, the at least one
first reception parameter may be determined upon successful
reception, by the first camera device, of an OCC packet transmitted
from the second OCC device.
[0120] Similarly, in some embodiments, the at least one first
transmission parameter may include a block size of a transmission
packet to be displayed by the first display device. In such
embodiments, the process 2800 may include the steps of estimating
and modifying the block size of the transmission packet to be
displayed by the first display device based on a predetermined
specification of an application transmitting data. Additionally, in
such embodiments, modifying the block size may include the step of
creating contiguous memory blocks having block size of a
predetermined denomination. For example, the process 2800 may
implement a buffering technique to minimize the number of write
operations by creating contiguous memory blocks having block size
of the least denomination.
[0121] In some embodiments, the at least one channel condition
parameter may include an ambient light of an environment. In such
embodiments, dynamically modifying the at least one first
transmission parameter at step 2804 may be based on the ambient
light. For example, in some embodiments, the process 2800 may
include the step of modifying brightness of the first display
device based on the ambient light.
[0122] Similarly, in some embodiments, the process 2800 may further
include the step of modifying color space of the first display
device. For example, the process 2800 may reduce the information
rate by reducing the color space and transmitting the same color
into multiple blocks of LEDs. Additionally, in some embodiments,
the process 2800 may further include the step of modifying a
resolution of a receive frame captured by the first camera device.
For example, the process 2800 may magnify the receiver camera to
increase the resolution of the transmit frame in order to decode
the information present in the frame effectively.
[0123] Further, in some embodiments, the process 2800 may further
include the step of modifying a number of symbols to be transmitted
per frame by the first display device. For example, the process
2800 may control (e.g., improve) the SNR under channel fading
condition by switching between single-symbol transmission per frame
and multi-symbol transmission per frame under compute resource
scenario. Additionally, for example, the process 2800 may improve
the SNR under signal impairment condition caused due to
deviated/tilted angle of camera capture by the second OCC device
with respect to the first OCC device. In particular, if the
transmit frame is captured at a titled angle by the camera, the
spatial arrangement of the colors may be affected, causing
erroneous decoding at the receiver end. Hence, skew correction may
be applied in order to proportionately reshape the transmit frame
and maintain the spatial arrangement of the colors. Thus, the
signal impairment may be corrected and SNR may be improved.
[0124] In some embodiments, the process 2800 may further include
the step of performing VCR from the first display device upon
receiving NACK signal from the first camera device based on a
comparison of a NACK score with a threshold NACK score.
Additionally, in such embodiments, the process 2800 may include the
step of optimizing a number of retransmission frames in the
transmit buffer based on a PLER, Further, in such embodiments, the
process 2800 may include the step of evaluating a validity of the
re-transmission based on the PLER.
[0125] In some embodiments, the first display device may include a
screen based display device configured to display an image or a
pattern. Additionally or alternatively, in some embodiments, the
first display device may include a set of spatially arranged Light
Emitting Diodes (LEDs) configured to display a pattern. Further, in
some embodiments, the image or the pattern may be placed inside a
colored boundary. For example, the process 2800 may employ guard
band in the form of a colored boundary around the transmit frame in
order to separate the transmit frame from the noise with minimum
processing time.
[0126] As will be also appreciated, the above described techniques
may take the form of computer or controller implemented processes
and apparatuses for practicing those processes. The disclosure can
also be embodied in the form of computer program code containing
instructions embodied in tangible media, such as floppy diskettes,
solid state drives, CD-ROMs, hard drives, or any other
computer-readable storage medium, wherein, when the computer
program code is loaded into and executed by a computer or
controller, the computer becomes an apparatus for practicing the
invention. The disclosure may also be embodied in the form of
computer program code or signal, for example, whether stored in a
storage medium, loaded into and/or executed by a computer or
controller, or transmitted over some transmission medium, such as
over electrical wiring or cabling, through fiber optics, or via
electromagnetic radiation, wherein, when the computer program code
is loaded into and executed by a computer, the computer becomes an
apparatus for practicing the invention. When implemented on a
general-purpose microprocessor, the computer program code segments
configure the microprocessor to create specific logic circuits.
[0127] As will be appreciated by those skilled in the art, the
techniques described in the various embodiments discussed above
pertain to improving efficiency of a bi-directional optical camera
communication (OCC). In particular, the techniques provide for:
[0128] a) SNR control in order to decrease error propensity. For
example, as discussed above, there will be information loss under
changing conditions of the signal quality if information rate of
transmission is kept constant. In order to avoid the information
loss, the information rate may be reduced by decreasing the block
size per optical symbol in the form of a color code. Additionally,
the information rate may be reduced by reducing the color space and
transmitting the same color into multiple blocks of LEDs. [0129] b)
PLER monitoring and optimization of the number of retransmission
frames in the transmit buffer. For example, as discussed above, a
dynamic version of the transmit buffer, based on the NACK from the
receiver, may be created. The probability of successful
retransmission of packets after transmitting the next version of
the transmit buffer is increased by varying the duty cycle of the
fragments which were failed to be transmitted in the previous
transmissions. This method may significantly reduce the number of
retransmissions. [0130] c) Evaluation of the validity of
re-transmission based on the MEL with respect to the PLER. The
retransmission may either be considered valid, or the packets may
be dropped depending on the MEL. The MEL is set based on the
channel quality indication present in the NACK score. [0131] d)
Display brightness control in order to optimize the power
consumption at the screen-based transmitter and to ensure
successful transmission. The brightness level may be optimized
based on what is required for the successful transmission, thereby
addressing the power concerns of the transmitter. [0132] e)
Effective utilization of the transmission buffers by implementing
dynamic information rate per symbol, which minimizes the number of
write operations for efficiency gain in the transmission process.
[0133] f) Transmit frame identification by introducing a concept of
guard band in the form of a colored boundary in order to extract
the transmitted frame correctly and to eliminate the noise around
the transmit frame with minimum processing time. It should be noted
that may also be implemented for a screen-based transmitter and
camera-based receiver. [0134] g) Transmit frame magnification to
increase the resolution of the transmit frame captured by the
receiver camera in order to decode the information present in the
frame effectively. [0135] h) Limiting the transmitter frame rate
periodically based on the dynamic processing condition and channel
media condition as factors to determine the reception rate of the
receiver camera. For example, as discussed above, the ambient
illumination at the receiver may be detected using photodetectors
in order to modify the transmit parameters accordingly. [0136] i)
Knowledge building model to predict the data transmission rate for
the future iterations, thereby saving the processing time. Such
knowledge model may also be employed to estimate the distance
between the transmitter and the receiver. As discussed above, after
every iteration, a set of state parameters may be noted in order to
build and train the knowledge model
[0137] Thus, the system and method described in various embodiments
discussed above may adaptively and dynamically detect/estimate and
modify various transmission parameters (e.g., frame rate) to
minimize the bit error rate and, consequently, to maximize the
system throughput under various situations (e.g., under various
detected/estimated reception parameters). Further, as described in
various embodiments discussed above, various reception parameters
are themselves dependent on various transmission parameter of
paired DCC device or various channel condition parameters.
[0138] Furthermore, one or more computer-readable storage media may
be utilized in implementing embodiments consistent with the present
disclosure. A computer-readable storage medium refers to any type
of physical memory on which information or data readable by a
processor may be stored. Thus, a computer-readable storage medium
may store instructions for execution by one or more processors,
including instructions for causing the processor(s) to perform
steps or stages consistent with the embodiments described herein.
The term "computer-readable medium" should be understood to include
tangible items and exclude carrier waves and transient signals,
i.e., be non-transitory. Examples include random access memory
(RAM), read-only memory (ROM), volatile memory, nonvolatile memory,
hard drives, CD ROMs, DVDs, flash drives, disks, and any other
known physical storage media.
[0139] It is intended that the disclosure and examples be
considered as exemplary only, with a true scope and spirit of
disclosed embodiments being indicated by the following claims.
* * * * *