U.S. patent application number 16/917673 was filed with the patent office on 2021-12-30 for image reading system, image forming system, and image reading method that perform image processing for each area.
This patent application is currently assigned to KYOCERA Document Solutions Inc.. The applicant listed for this patent is KYOCERA Document Solutions Inc.. Invention is credited to Naohiko KOSAKA.
Application Number | 20210409569 16/917673 |
Document ID | / |
Family ID | 1000006024475 |
Filed Date | 2021-12-30 |
United States Patent
Application |
20210409569 |
Kind Code |
A1 |
KOSAKA; Naohiko |
December 30, 2021 |
IMAGE READING SYSTEM, IMAGE FORMING SYSTEM, AND IMAGE READING
METHOD THAT PERFORM IMAGE PROCESSING FOR EACH AREA
Abstract
Provided is an image reading system capable of improving the
image quality of a document in which text and images are mixed in
the same page. A camera captures a document in page units to obtain
captured image data. A document reading unit is an image scanner
having a higher resolution than the camera. An area classifying
unit classifies the captured image data captured by the camera into
areas for each content. An area selecting unit selects, for each
classified area, whether to output captured image data based on the
area classification and the state of the captured image data. An
image processing unit processes the captured image data or the
scanned image data for each area, and outputs or deletes the data
as area data. The document output unit collects the processed area
data, reconstructs the data into document data, and outputs the
document data.
Inventors: |
KOSAKA; Naohiko; (Dublin,
CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
KYOCERA Document Solutions Inc. |
Osaka |
|
JP |
|
|
Assignee: |
KYOCERA Document Solutions
Inc.
Osaka
JP
|
Family ID: |
1000006024475 |
Appl. No.: |
16/917673 |
Filed: |
June 30, 2020 |
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
H04N 1/387 20130101;
G06K 9/00456 20130101; H04N 1/04 20130101 |
International
Class: |
H04N 1/387 20060101
H04N001/387; G06K 9/00 20060101 G06K009/00; H04N 1/04 20060101
H04N001/04 |
Claims
1. An image reading system having an image reading apparatus and an
information processing apparatus, the image reading apparatus
including: a camera configured to capture a document in page units
and acquire captured image data, an image scanner configured to
have a higher resolution than the captured image data; and a
document feeder configured to feed the document to the image
scanner; and the information processing apparatus including: a
controller and a non-temporary recording medium; wherein a control
program is stored on the non-temporary recording medium; the
controller is configured to execute the control program to operate
as an area classifying unit, an area selecting unit, and a document
output unit; the area classifying unit is configured to classify
the captured image data captured by the camera into respective
areas for each content and store corresponding rectangular
coordinate data of the respective areas, the area selecting unit is
configured to select, for each area classified by the area
classifying unit, whether to output a corresponding portion of the
captured image data or to scan the corresponding portion of the
captured image data with the image scanner, based on classification
of the area and state of the corresponding portion of the captured
image data, thereby selecting to output a portion of the captured
image data corresponding to one or more areas and selecting not to
output a portion of the captured image data corresponding to one or
more areas, the image scanner is configured to 1) scan, with the
higher resolution, a portion of the document corresponding to the
rectangular coordinate data of the one or more areas of the
document for which the area selecting unit selected not to output
the corresponding portion of the captured image data, thereby
providing scanned image data; and 2) not scan a portion of the
document corresponding to the rectangular coordinate data of the
one or more areas of the document for which the area selecting unit
selected to output the corresponding portion of the captured image
data, the image processing unit is configured to process, for each
area, the corresponding portion of the captured image data selected
for output or the scanned image data scanned by the image scanner,
and output the area as an area data or delete the area, the
document output unit is configured to collect the area data
processed by the image processing unit, reconstruct the area data
for each of the areas into document data, and output the document
data; and the camera is arranged such that the capture of the
document is obtained from the document when the document is on the
document feeder.
2. The image reading system according to claim 1, wherein:
classification of the area classified by the area classifying unit
includes a text, a table, and an image; and the image processing
unit processes the captured image data or the scanned image data of
the area classified into the text or the table, and outputs the
area data including character data and/or table data.
3. The image reading system according to claim 1, wherein: the area
selecting unit does not output the captured image data and scans
with the image scanner when the area is classified as a text or a
table and the state of the captured image data is not suitable for
optical character recognition.
4. The image reading system according to claim 2, wherein: the
image processing unit recognizes and deletes the area that is not
related to the content of other text.
5. The image reading system according to claim 4, wherein: an image
not related to the content of the other text includes an
advertisement, an illustration, and a logo.
6. An image forming system having an image forming apparatus and an
information processing apparatus: the image reading apparatus
including: a camera configured to capture a document in page units
and acquire captured image data, an image scanner configured to
have a higher resolution than the captured image data, a document
feeder configured to feed the document to the image scanner, and an
image forming unit configured to form an image of the document
data; and the information processing apparatus including: a
controller and a non-temporary recording medium; wherein a control
program is stored on the non-temporary recording medium; the
controller is configured to execute the control program to operate
as an area classifying unit, an area selecting unit, and a document
output unit; the area classifying unit is configured to classify
the captured image data captured by the camera into respective
areas for each content and store corresponding rectangular
coordinate data of the respective areas, the area selecting unit is
configured to select, for each area classified by the area
classifying unit, whether to output a corresponding portion of the
captured image data or to scan the corresponding portion of the
captured image data with the image scanner, based on classification
of the area and state of the corresponding portion of the captured
image data, thereby selecting to output a portion of the captured
image data corresponding to one or more areas and selecting not to
output a portion of the captured image data corresponding to one or
more areas, the image scanner is configured to 1) scan, with the
higher resolution, a portion of the document corresponding to the
rectangular coordinate data of the one or more areas of the
document for which the area selecting unit selected not to output
the corresponding portion of the captured image data, thereby
providing scanned image data; and 2) not scan a portion of the
document corresponding to the rectangular coordinate data of the
one or more areas of the document for which the area selecting unit
selected to output the corresponding portion of the captured image
data, the image processing unit is configured to process, for each
area, the corresponding portion of the captured image data selected
for output or the scanned image data scanned by the image scanner,
and output the area as an area data or delete the area, the
document output unit is configured to collect the area data
processed by the image processing unit, reconstruct the area data
for each of the areas into document data, and output the document
data; and the camera is arranged such that the capture of the
document is obtained from the document when the document is on the
document feeder.
7. The image forming system according to claim 6, wherein:
classification of the area classified by the area classifying unit
includes a text, a table, and an image; and the image processing
unit processes the captured image data or the scanned image data of
the area classified into the text or the table, and outputs the
area data including character data and/or table data.
8. The image forming system according to claim 6, wherein: the area
selecting unit does not output the captured image data and scans
with the image scanner when the area is classified as a text or a
table and the state of the captured image data is not suitable for
optical character recognition.
9. The image forming system according to claim 7, wherein: the
image processing unit recognizes and deletes the area that is not
related to the content of other text.
10. The image forming system according to claim 9, wherein: an
image not related to the content of the other text includes an
advertisement, an illustration, and a logo.
11. An image reading method executed by an image reading system
including an image reading apparatus and an information processing
apparatus, comprising the steps of: with a camera in the image
reading apparatus, capturing a document in page units and acquiring
captured image data, wherein the capturing is performed when the
document is on a document feeder that feeds the document to an
image scanner; in the information processing apparatus, classifying
the captured image data that is captured into respective areas for
each content and storing corresponding rectangular coordinate data
of the respective areas; in the information processing apparatus,
selecting, for each classified area, whether or not to output a
corresponding portion of the captured image data based on
classification of the area and state of the portion of the captured
image data, thereby selecting to output a portion of the captured
image data corresponding to one or more areas and selecting not to
output a portion of the captured image data corresponding to of one
or more areas; with an image scanner 1) scanning, with a higher
resolution than the captured image data in the image reading
apparatus, a portion of the document corresponding to the
rectangular coordinate data of the one more areas of the document
for which the corresponding portion of the captured image data was
selected not to output, thereby providing scanned image data, and
2) not scanning a portion of the document corresponding to the
rectangular coordinate data of the one or more areas of the
document for which the area the corresponding portion of the
captured image data was selected to output; in the information
processing apparatus, processing, for each area, the corresponding
portion of the captured image data selected for output or the
scanned image data scanned by the image scanner, and outputting the
area as an area data or deletes the area; and in the information
processing apparatus, collecting processed area data,
reconstructing the area data for each of the areas into document
data, and outputting the document data.
12. The image reading method according to claim 11, wherein:
classification includes a text, a table, and an image; and the
method further includes processing the captured image data or the
scanned image data of the area classified into the text or the
table, and outputting the area data including character data and/or
table data.
13. The image reading method according to claim 11, further
comprising: not outputting the captured image data and scanning
with the image scanner when the area is classified as a text or a
table and the state of the captured image data is not suitable for
optical character recognition.
14. The image reading method according to claim 12, further
comprising: recognizing and deleting the area that is not related
to the content of other text.
15. The image reading method according to claim 14, wherein: an
image not related to the content of the other text includes an
advertisement, an illustration, and a logo.
Description
BACKGROUND
[0001] The present disclosure relates to an image reading system,
an image forming system, and an image reading method, and
particularly to an image reading system, an image forming system,
and an image reading method that perform image processing for each
area.
[0002] Typically, there is an image forming apparatus such as a
multifunctional peripheral (MFP) capable of printing a document or
an image. The image forming apparatus is also used as an image
reading apparatus, which is mainly used for scanning. Moreover,
there are also a dedicated reading apparatus such as a document
scanner and a network scanner.
[0003] A typical image reading apparatus is disclosed. The typical
apparatus is an image scanner system and an image scanner apparatus
capable of correcting the brightness of a captured image without
using an initial setting sheet. In this apparatus, a display
section is provided on a mat section, a rectangular section of the
display section is displayed in white, an image is taken by a
camera section, and a brightness correction parameter is calculated
for the obtained image by a host apparatus. The correction
parameters obtained by calculation are used to correct the
brightness of an image obtained by imaging the imaging target
during operation.
SUMMARY
[0004] An image reading system of the present disclosure having an
image reading apparatus and an information processing apparatus,
includes: the image reading apparatus including: a camera
configured to capture a document in page units and acquire captured
image data, and an image scanner configured to have a higher
resolution than the camera; and the information processing
apparatus including: an area classifying unit configured to
classify the captured image data captured by the camera into areas
for each content, an area selecting unit configured to select, for
each area classified by the area classifying unit, whether to
output the captured image data or to scan with the image scanner,
based on classification of the area and state of the captured image
data, an image processing unit configured to process, for each
area, the captured image data or scan image data scanned by the
image scanner, and output the area as an area data or delete the
area, and a document output unit configured to collect the area
data processed by the image processing unit, reconstruct the data
into document data, and output the document data.
[0005] An image forming system according to the present disclosure
having an image forming apparatus and an information processing
apparatus, includes: the image reading apparatus including: a
camera configured to capture a document in page units and acquires
captured image data, an image scanner configured to have a higher
resolution than the camera, and an image forming unit configured to
form an image of the document data; and the information processing
apparatus including: an area classifying unit configured to
classify the captured image data captured by the camera into areas
for each content, an area selecting unit configured to select, for
each area classified by the area classifying unit, whether to
output the captured image data or to scan with the image scanner,
based on classification of the area and state of the captured image
data, an image processing unit configured to process, for each
area, the captured image data or scan image data scanned by the
image scanner, and output the area as an area data or delete the
area, and a document output unit configured to collect the area
data processed by the image processing unit, reconstruct the data
into document data, and output the document data.
[0006] An image reading method according to the present disclosure
executed by an image reading system including an image reading
apparatus and an information processing apparatus, includes the
steps of: with a camera in the image reading apparatus, capturing a
document in page units and acquires captured image data; in the
information processing apparatus, classifying the captured image
data that is obtained into areas for each content; in the
information processing apparatus, selecting, for each classified
area, whether or not to output the captured image data based on
classification of the area and state of the captured image data;
with an image scanner having a higher resolution than the camera in
the image reading apparatus, scanning the area of the document for
the area selected not to output the captured image data; in the
information processing apparatus, processing, for each area, the
captured image data or scan image data scanned by the image
scanner, and outputting the area as an area data or deletes the
area; and in the information processing apparatus, collecting
processed area data, reconstructing the data into document data,
and outputting the document data.
BRIEF DESCRIPTION OF THE DRAWINGS
[0007] FIG. 1 is a system configuration diagram of an image forming
system according to an embodiment of the present disclosure;
[0008] FIG. 2 is a schematic diagram of the image forming apparatus
as shown in FIG. 1;
[0009] FIG. 3 is a block diagram showing a control configuration of
the image forming apparatus as shown in FIG. 1;
[0010] FIG. 4 is a block diagram showing a control configuration of
the server as shown in FIG. 1;
[0011] FIG. 5 is a block diagram illustrating a functional
configuration of the image forming system according to the
embodiment of the present disclosure;
[0012] FIG. 6 is a flowchart of a document reading process
according to the embodiment of the present disclosure;
[0013] FIG. 7 is a diagram illustrating an example of captured
image data obtained by the camera imaging process as shown in FIG.
6;
[0014] FIG. 8 is a conceptual diagram of the area classification
process as shown in FIG. 6; and
[0015] FIG. 9 is a diagram illustrating an example of document data
generated by the reconstruction process as shown in FIG. 6.
DETAILED DESCRIPTION
Embodiment
[System Configuration of Image Forming System X]
[0016] First, a system configuration of an image forming system X
according to an embodiment of the present disclosure is described
with reference to FIGS. 1 to 4.
[0017] Here, the image forming system X of the present embodiment
is an image reading system including a plurality of image forming
apparatuses 1 and a server 2.
[0018] In the image forming system X of the present embodiment,
each image forming apparatus 1 and server 2 are connected via a
network 5.
[0019] The image forming apparatus 1 is an example of the image
reading apparatus according to the present embodiment. The image
forming apparatus 1 includes an MFP capable of transmitting and
receiving various data, a network scanner, a document scanner, a
network FAX, a printer, and the like. The image forming apparatus 1
can read a document D (FIG. 2) and output the document data 230
(FIG. 5) by, pull-printing, direct printing, copying, scanning,
performing facsimile transmission, or the like.
[0020] The server 2 is an information processing apparatus that
controls the image forming apparatus 1, processes the read image
data, and manages the generated document data 230 (FIG. 5). The
server 2 can transmit and receive various data from and to the
image forming apparatus 1. In this embodiment, the server 2 may be
a general-purpose machine, PC (Personal Computer) server, NAS
(Network Attached Storage), and the like.
[0021] The server 2 executes a general-purpose OS (Operating
System) such as Linux (registered trademark) or Windows (registered
trademark) and provides various services (daemons) on the OS.
[0022] Note that the server 2 may be capable of storing, managing,
charging, authorizing, and the like, the document data 230.
Further, the server 2 may store the document data 230 in a document
box (a shared folder, a storage folder) associated with the
user.
[0023] Further, the server 2 may be capable of performing
transmission processing to a business or office DMS (Document
Management System), or the like.
[0024] The network 5 is an Intranet such as a LAN (Local Area
Network), a WAN (Wide Area Network) such as the Internet or a
mobile phone network, or the like. When the network 5 is a LAN, it
may be a wireless LAN such as WiFi. When the network 5 is a WAN, it
may be connectable to a server on a so-called "cloud" via a router,
a gateway, or the like. Further, the network 5 may be connected to
a plurality of networks by a VPN (Virtual Private Network), a
bridge, or the like.
[0025] (Configuration of Image Forming Apparatus 1)
[0026] Next, the configuration of the image forming apparatus 1
according to the embodiment of the present disclosure is described
in detail with reference to FIGS. 2 to 3.
[0027] First, the appearance and operation of the image forming
apparatus 1 is described with reference to FIG. 2.
[0028] In the image forming apparatus 1, the document reading unit
12 is provided above the main body 14, and the document feeding
unit 13 is provided above the document reading unit 12. The stack
tray 50 is provided on the recording paper discharge port 41 side
formed on the main body 14. The operation panel unit 16 is provided
on the front side of the image forming apparatus 1.
[0029] The document reading unit 12 is an image scanner including a
scanner 12a, a platen glass 12b, and a document reading slit 12c.
The scanner 12a includes an exposure lamp, a CCD (charge coupled
device) or a CMOS (complementary metal oxide semiconductor) image
sensor, and the like. The document reading unit 12 is configured to
be movable in a document conveying direction by the document
feeding unit 13.
[0030] The platen glass 12b is a platen made of a transparent
member such as glass. The document reading slit 12c has a slit
formed in a direction perpendicular to the direction in which the
document is transported by the document feeding unit 13.
[0031] When reading a document placed on the platen glass 12b, the
scanner 12a is moved to a position facing the platen glass 12b.
Then, the scanner 12a reads the document while scanning the
document placed on the platen glass 12b and acquires the scan image
data 210 (FIG. 5). The scanner 12a stores the acquired scanned
image data 210 in the storage unit 19 (FIG. 3) of the main body
14.
[0032] When reading a document conveyed by the document feeding
unit 13, the scanner 12a is moved to a position facing the document
reading slit 12c. Then, the scanner 12a reads the document via the
document reading slit 12c in synchronization with the document
feeding operation by the document feeding unit 13 and acquires the
scan image data 210. The scanner 12a stores the acquired scanned
image data 210 in the storage unit 19 of the main body 14.
[0033] Here, in the present embodiment, the document reading unit
12 can scan at an arbitrary rectangular position on the paper
surface of the document D by setting the reading resolution by
setting such as color, partial color, and monochrome.
[0034] The document feeding unit 13 includes a document placing
section 13a, a document discharge section 13b, and a document
transport mechanism 13c. The document D, which is placed on the
document placing section 13a, is sequentially fed out one by one
from the upper surface by the document transporting mechanism. 13c,
is transported to a position facing the document reading slit 12c,
and is then discharged to the document discharge section 13b.
[0035] The document feeding unit 13 is configured to be foldable,
and the upper surface of the platen glass 12b can be opened by
lifting the document feeding unit 13 upward.
[0036] The camera 60, which is described later, can also capture
the document D in this state.
[0037] The main body 14 includes the image forming unit 17, and
also includes a paper feed unit 42, a paper transport path 43, a
transport roller 44, and a discharge roller 45. The paper feed unit
42 includes a plurality of paper feed cassettes 42a that store
recording papers having different sizes or directions, and a paper
feed roller 42b that feeds the recording paper from the paper feed
cassette 42a one by one to a paper transport path 43. The paper
feed roller 42b, the transport roller 44, and the discharge roller
45 function as a transport unit. The recording paper is transported
by this transport unit.
[0038] The recording paper fed to the paper transport path 43 by
the paper feed roller 42b is transported to the image forming unit
17 by the transport roller 44. Then, the recording paper on which
the recording is performed by the image forming unit 17 is
discharged to the stack tray 50 by the discharge roller 45. In
addition, the main body 14 may include an internal tray in addition
to the stack tray 50.
[0039] The operation panel unit 16 includes an input unit such as a
button and a touch panel, and a display unit such as an LCD (Liquid
Crystal Display) and an organic EL (OEL or OLED) display. The
buttons of the input unit of the operation panel unit 16 are
numeric pads, buttons for starting, canceling, switching operation
modes, and giving instructions for executing a job. Among these,
the switchable operation modes include, for example, modes such as
copying, facsimile transmission, scanner, network scanner, or the
like. Executable jobs include, for example, printing, transmitting,
saving, recording the selected document, or the like.
[0040] The operation panel unit 16 is provided on the front side of
the image forming apparatus 1. The input unit of the operation
panel unit 16 acquires instructions of various jobs of the image
forming apparatus 1 by the user. In addition, it is also possible
to input and change information of each user according to the
user's instruction acquired from the operation panel unit 16.
[0041] The image forming unit 17 includes a photosensitive drum
17a, an exposure unit 17b, a developing unit 17c, a transfer unit
17d, and a fixing unit 17e. The exposure unit 17b is an optical
unit including a laser device, a mirror, a lens, an LED
(light-emitting diode) array, and the like. The exposure unit 17b
emits light based on bitmap data, or the like, on which the
document data 230 (FIG. 3) is image-formed and exposes the
photosensitive drum 17a to form an electrostatic latent image on
the surface of the photosensitive drum 17a. The developing unit 17c
is a developing unit that develops an electrostatic latent image
formed on the photosensitive drum 17a by using toner, and forms a
toner image based on the electrostatic latent image on the
photosensitive drum 17a. The transfer unit 17d transfers the toner
image formed on the photosensitive drum 17a by the developing unit
17c to a recording paper. The fixing unit 17e heats the recording
paper onto which the toner image has been transferred by the
transfer unit 17d and fixes the toner image on the recording paper.
In this heating, it is possible to use IH (Induction Heating), or
the like.
[0042] The camera 60 is a camera provided with an image sensor such
as a CCD or a CMOS image sensor, an optical element such as a lens,
an LED illumination, and the like, which can image a document D
placed on the document feeding unit 13. The camera captures an
image of the document D in page units. Specifically, the camera 60
captures a still image or a moving image of the entire surface
(page) of the document D one by one and outputs the captured image
data 200 (FIG. 5), which is captured, to the storage unit 19 in the
main body 14. The resolution of the captured image data 200 may be
lower than that of the scan image data 210 (FIG. 5) acquired by the
document reading unit 12. Note that the camera 60 may be provided
with a mechanism such as a zoom and a tilt so that an arbitrary
position of the document can be magnified and captured.
[0043] Next, a control configuration of the image forming apparatus
1 is described with reference to FIG. 3. Description of the
configuration described in FIG. 2 is partially omitted.
[0044] The image forming apparatus 1 includes a control unit 10, an
image processing unit 11, a document reading unit 12, a document
feeding unit 13, a network transmitting and receiving unit 15, an
operation panel unit 16, an image forming unit 17 (image forming
unit), a fax transmitting and receiving unit 18, a storage unit 19,
a transport unit or the like.
[0045] Each unit is connected to the control unit 10 and its
operation is controlled by the control unit 10.
[0046] The control unit 10 is information processing unit, such as
GPP (General Purpose Processor), CPU (Central Processing Unit), MPU
(Micro Processing Unit), DSP (Digital Signal Processor), GPU
(Graphics Processing Unit), ASIC (Application Specific Integrated
Circuit), or the like.
[0047] The control unit 10 reads a control program stored in the
ROM or the HDD of the storage unit 19, expands the control program
in the RAM, and executes the control program to operate as each
unit of a function block as described later. Further, the control
unit 10 controls the entire apparatus according to prescribed
instruction information inputted from an external terminal or the
operation panel unit 16.
[0048] The image processing unit 11 is a control arithmetic unit
such as a DSP (Digital Signal Processor), a GPU (Graphics
Processing Unit), an ASIC, and the like. The image processing unit
11 performs some image processing on image data. The image
processing may be, for example, processing such as
enlargement/reduction, rotation and distortion (skew) correction,
density adjustment, gradation adjustment, and image improvement.
The skew correction includes processing of a spread portion of a
spread document such as a book, and the like. In addition, the
image processing unit 11 may be capable of executing a process such
as averaging, or the like, the captured image data 200 as the
moving image data to increase the resolution or remove noise.
[0049] Further, the image processing unit 11 can store the scan
image data 210 (FIG. 5) read by the document reading unit 12 and
the captured image data 200 captured by the camera 60 in the
storage unit 19 as print data and can send them to the server 2. At
this time, the image processing unit 11 may include an accelerator
function for converting the print data into electronic document
data or image data. Specifically, the image processing unit 11 may
include at least a part of a learned model that performs geometric
layout analysis of an image, various models included in the image
processing unit 21 as described later, and the like.
[0050] The network transmitting and receiving unit 15 is a network
connection unit including a LAN board, a wireless transceiver, or
the like, for connecting to the network 5.
[0051] The network transmitting and receiving unit 15
transmits/receives data on a data communication line and
transmits/receives a voice signal on a voice telephone line.
[0052] The fax transmitting and receiving unit 18 performs
transmitting and receiving facsimile. The fax transmitting and
receiving unit 18 receives a facsimile from another facsimile
device via a voice line and stores it in the storage unit 19. The
facsimile transmitting and receiving unit 18 can cause the image
forming unit 17 to form it an image. Further, the fax transmitting
and receiving unit 18 can convert the document read by the document
reading unit 12 or the data of the network facsimile transmitted
from an external terminal into image data, and facsimile-transmit
the image data to another facsimile device via a voice line.
[0053] The storage unit 19 is a non-temporary recording medium such
as a semiconductor memory as a ROM (Read Only Memory) and a RAM
(Random Access Memory), or the like, and as an HDD (Hard Disk
Drive).
[0054] In the present embodiment, the storage unit 19 stores
various image data, document data 230 (FIG. 5), and the like.
[0055] The contents in the RAM of the storage unit 19 are retained
by a function such as self-refresh even in the power saving
state.
[0056] A control program for controlling the operation of the image
forming apparatus 1 is stored in the ROM or the HDD of the storage
unit 19. In addition, the storage unit 19 also stores user account
settings. Further, the storage unit 19 may include a storage folder
(document box) area for each user.
[0057] In addition, in the image forming apparatus 1, the control
unit 10 and the image processing unit 11 may be integrally formed,
such as a CPU with a built-in GPU, a chip-on-module package, an SOC
(System On a Chip), and the like.
[0058] Further, the control unit 10 and the image processing unit
11 may include a RAM, a ROM, a flash memory, or the like.
Furthermore, the control unit 10 can perform at least a part of the
image processing of the image processing unit 11.
[0059] (Configuration of Server 2)
[0060] Next, the configuration of the server 2 is described with
reference to FIG. 4.
[0061] The server 2 includes a control unit 20, an image processing
unit 21, a network transmitting and receiving unit 25, a storage
unit 29, and the like.
[0062] The control unit 20 is an information processing unit such
as a GPP, a CPU, an MPU, a DSP, a GPU, an ASIC, and the like.
[0063] The image processing unit 21 is a control operation unit
such as a GPU, a DSP, an ASIC, and the like.
[0064] In the present embodiment, the image processing unit 21 can
perform analysis and recognition process on an image. The image
processing unit 21 can also recognize, for example, whether the
image is color, partial color, or monochrome (black or white), or
can acquire the ground color.
[0065] In addition, the image processing unit 21, may include an
accelerator function of an AI (Artificial Intelligence) including a
CNN (Convolutional Neural Network), another artificial neural
network, reinforcement learning, and the like. In this case, the
image processing unit 21 can use a learned model or perform model
learning. In the present embodiment, it is possible to use various
models for recognizing a document area and recognizing characters,
tables, and images. Among these, for the image, a model that
recognizes a non-text line, a symbol, an illustration, a logo, a
photograph, or the like, can be used. For a photograph, a model
that recognizes an imaged object such as a person, an animal, a
material body, or the like, can also be used. Furthermore, it is
also possible to use a model that recognizes an advertisement by
combining these various models.
[0066] In the present embodiment, the image processing unit 21 can
execute an OCR (Optical Character Recognition) process. The image
processing unit 21 can also perform the OCR by using various types
of image recognition algorithm and AI. The image processing unit 21
can also detect the location of the form in the table by the OCR.
For the location of the form, it is possible to use the
above-described model for recognizing non-text lines. Furthermore,
the image processing unit 21 may be able to determine the content
of the text by using the AI of the natural language processing. In
this case, analysis by using a distributed word expression,
semantic analysis of text, similarity search, and the like, can be
performed. In addition, the AI of the natural language processing
can be added to the above-described advertisement recognition
model. In the model for recognizing the advertisement, if a
trademark, a logo, a specific phrase, a currency symbol, an
address, a telephone number, and the like, are included, the
advertisement can be determined.
[0067] The network transmitting and receiving unit 25 is a network
connection unit including a LAN board, a wireless transceiver, or
the like, for connecting to the network 5.
[0068] The storage unit 29 is a storage unit by using a
non-temporary recording medium. The storage unit 29 may include a
RAM, a ROM, an eMMC, an SSD, an HDD, and the like. The storage unit
29 stores a control program for controlling the operation of the
server 2. The control program includes programs and data such as an
OS, a service (daemon) for realizing pull print, a device driver,
and various applications. This service (daemon) may be, for
example, a WWW server by using the HTTP protocol and various
server-side applications.
[0069] Further, the storage unit 29 may include an external storage
medium such as various flash memories and optical recording
media.
[0070] Note that the control unit 20 and the image processing unit
21 may be integrally formed as a CPU with a built-in GPU, a
chip-on-module package, an SOC, or the like.
[0071] Further, the control unit 20 and the image processing unit
21 may include a RAM, a ROM, a flash memory, or the like.
[0072] Further, the control unit 20 reads out the control program
stored in the storage unit 29, expands and executes the control
program, and thereby operates as each unit of a functional block as
described later. Furthermore, the control unit 20 can execute at
least a part of the process of the image processing unit 21.
[0073] [Functional Configuration of Image Forming System X]
[0074] Here, with reference to FIG. 5, a functional configuration
of the image forming system X is described.
[0075] The control unit 20 of the server 2 includes an area
classifying unit 100, an area selecting unit 110, and a document
output unit 120.
[0076] The storage unit 29 stores the captured image data 200, the
scanned image data 210, the area data 220, and the document data
230.
[0077] The area classifying unit 100 classifies the captured image
data 200 captured by the camera 60 of the image forming apparatus 1
into areas for each content. In the present embodiment, the area
classifying unit 100 can recognize an area from each page of the
document D by using AI or the like, and classify each area as one
of a text, a table, an image, and the like. At this time, the area
classifying unit 100 can also determine information such as the
accuracy of the classification and the state of the area. In
addition, the area classifying unit 100 can perform high-speed
processing by using the AI accelerator, or the like, of the image
processing unit 21 at the time of classification.
[0078] The area selecting unit 110 selects, for each area
classified by the area classifying unit 100, whether or not to
output the captured image data 200 based on the classification of
the area and the state of the captured image data 200.
[0079] Specifically, in the present embodiment, the area selecting
unit 110 does not output the captured image data 200 when the area
is classified as a text or a table and the state of the captured
image data 200 is not suitable for performing OCR, and the document
is scanned by the document reading unit 12.
[0080] The document output unit 120 collects the area data 220
processed by the image processing unit 21, reconstructs the area
data 220, and outputs the document data 230. At this time, the
document output unit 120 does not have to collect the area data
220, which is performed deletion process by the image processing
unit 21.
[0081] In the present embodiment, the image processing unit 21
processes the selected captured image data 200 or scan image data
210 for each area, and outputs or deletes it as area data 220.
[0082] Specifically, in the present embodiment, the image
processing unit 21 recognizes an image that is not related to the
content of the text and performs the deletion process. The images
that is not related to the content of the text includes an
advertisement, an illustration, a logo, and the like.
[0083] In the present embodiment, the document reading unit 12 in
the imager forming apparatus 1 scans the area of the document D for
the area selected by the area selecting unit 110 not to output the
captured image data 200. For this reason, in the present
embodiment, the document reading unit 12 functions as an image
scanner having a higher resolution than the camera 60. The scan
image data 210 read by the document reading unit 12 is transmitted
to the server 2.
[0084] In the present embodiment, the camera 60 in the image
forming apparatus 1 captures an image of a document in page units
and acquires captured image data 200. The captured image data 200
is transmitted to the server 2.
[0085] The captured image data 200 is image data captured by the
camera 60 in the imager forming apparatus 1 from the document D in
page units. As this image data, it is possible to use image data of
various still images such as JPEG (JPG), TIFF, RAW bitmap, BMP,
GIF, PNG, and the like. Alternatively, the captured image data 200
may be image data of a still image, which a frame is extracted from
image data of a moving image or image quality is improved by
averaging, or the like.
[0086] The scan image data 210 is image data scanned by the
document reading unit 12 in the imager forming apparatus 1. This
image data may have higher resolution than the captured image data
200, more color information, less reflection of lighting, and may
be suitable for OCR. That is, when OCR is performed by using the
scan image data 210, it may be possible to reduce erroneous
recognition.
[0087] The area data 220 is image data for each area classified by
the area classifying unit 100 in each page of the document D. In
the present embodiment, the area data 220 is data that includes
only a portion of processed area derived from the captured image
data 200 or the scan image data 210. The area data 220 may include
text data after OCR, table structure data, image data subjected to
high image quality processing, and the like. Alternatively, when
the deletion processing has been performed, the area data 220 may
be data in a state in which a blank or a "NULL" value is
written.
[0088] The document data 230 is electronic document data such as
PDF (Portable Document Format), PS (Post Script), XML (Extensible
Markup Language), a word processor file, a spreadsheet file, a
presentation software file, and the like. Document data 230 may
include text data, image data, and other binary data. Of these, the
text data may include structured data described in a markup
language such as XML, or the like. This structured data includes
information on forms and tables, information on the above-described
areas, page information, printing properties, other meta
information, and the like. The image data may be a type of data
included in the electronic document data 230 such as JPEG, TIFF,
GIF, PNG, and the like.
[0089] Here, the control unit 20 of the server 2 executes the
control program stored in the storage unit 29 to function as the
area classifying unit 100, the area selecting unit 110, and the
document output unit 120.
[0090] Further, each unit of the image forming apparatus 1 and the
server 2 described above are a hardware resource for executing the
image reading method of the present disclosure.
[0091] Note that a part or an arbitrary combination of the
above-described functional configurations may be configured in
terms of hardware and circuits by using an IC, a programmable
logic, an FPGA (Field-Programmable Gate Array), or the like.
[0092] [Document Reading Process by Image Forming System X]
[0093] Next, with reference to FIGS. 6 to 9, the document reading
process by the image forming system X according to the embodiment
of the present disclosure is described.
[0094] In the document reading process according to the present
embodiment, the camera 60 captures an image of the document D in
page units and acquires captured image data 200. Then, the captured
image data 200, which is captured, is classified into areas for
each content. For each classified area, whether to output or not
the captured image data 200 is selected based on the classification
and the state of the captured image data 200. Here, the document
reading unit 12 scans the area of the document D for the area
selected not to output the captured image data 200. Then, for each
area, the captured image data 200 or the scanned image data 210 is
processed and output it as area data 220 or deleted it. Then, the
processed area data 220 is collected, reconstructed into document
data 230, and output it.
[0095] In the document reading process according to the present
embodiment, the control unit 10 and the control unit 20 mainly
execute the programs stored in the storage unit 19 and the storage
unit 29 by using hardware resources in cooperation with each
unit.
[0096] Hereinafter, the details of the document reading process
according to the present embodiment is described step by step with
reference to the flowchart in FIG. 6.
[0097] (Step S100)
[0098] First, the operation panel unit 16 and the camera 60 of the
image forming apparatus 1 perform a camera imaging process.
[0099] Here, the user places the document D on the document placing
section 13a of the document feeding unit 13 and gives an
instruction to read the document on the operation panel unit 16. At
this time, the user can set the format of the document data 230
created after reading, the necessity of an image in the document,
and the like. The format of the document data 230 includes a file
format, color, partial color, or monochrome (black and white)
settings, resolution settings, print and transmission settings, and
the like. Further, as the format of the document data 230, it is
also possible to set whether or not to delete a handwritten
comment, a figure, a highlighted marker, or the like.
[0100] Further, the user can perform output instruction such as
setting a transmission location of the read document data 230,
storing the document data 230 in a document box, printing a
document in the image forming unit 17, or the like.
[0101] The operation panel unit 16 that has received these
instructions transmits a command for imaging the document D to the
camera 60.
[0102] Then, the camera 60 captures the document D in page units,
acquires captured image data 200, and stores the captured image
data 200 in the storage unit 19. At this time, the image processing
unit 11 can also temporarily process the entire captured image data
200 to improve image quality.
[0103] The imaged document D is sequentially fed out one by one
from the upper surface by the document transport mechanism 13c and
discharged to the document discharge unit 13b. At this time, the
document D may be scanned as described later.
[0104] The captured image data 200 is transmitted to the server
2.
[0105] FIG. 7 shows an example of the captured image data 200.
[0106] (Step S201)
[0107] Here, the area classifying unit 100 of the server 2 performs
an area classification process.
[0108] The area classifying unit 100 receives the captured image
data 200 captured by the camera 60 from the image forming apparatus
1 and stores it in the storage unit 29.
[0109] Then, the area classifying unit 100 classifies the captured
image data 200 into areas for each content. In the present
embodiment, first, the area classifying unit 100 performs a
geometric layout analysis by using, for example, various models of
the AI of the image processing unit 21.
[0110] More specifically, the area classifying unit 100 uses the
image processing unit 21 to acquire, for example, a ground color,
recognize a page margin of the captured image data 200, and
recognize a page number and an index. Then, the area classifying
unit 100 recognizes a unit of characters and images of the captured
image data 200 as an area from a main body part of text other than
these. The area classifying unit 100 can recognize the area based
on a blank, an outline, and geometric information.
[0111] Then, the area classifying unit 100 classifies the content
of the recognized area. Specifically, the area classifying unit 100
classifies the area into one of a text, a table, and an image, and
performs labeling. At this time, the area classifying unit 100
analyzes the accuracy of the classification, the state of the area,
and the like, and performs labeling. The area classifying unit 100
can recognize whether the state of the area is color, partial
color, or monochrome (black and white). Further, the area
classifying unit 100 can recognize the presence or absence of a
stain, a portion of poor printing state, or a blurred or unclear
spot (hereinafter, simply referred to as "stain, or the like") as
the state of the area. Further, in the case of a text or a table,
the area classifying unit 100 can analyze whether or not OCR is
possible based on the stain, or the like. Specifically, when the
stain or the like is equal to or more than a specific stain
threshold, the area classifying unit 100 analyzes that the OCR is
not possible because the false recognition rate is too high in the
OCR.
[0112] The area classifying unit 100 stores the data of the
labeling of the analysis results together with the rectangular
coordinate data of the captured image data 200 in the storage unit
29 as data indicating the area.
[0113] FIG. 8 shows an example in which areas A1, A2, A3, A4, A5,
and A6 are recognized from the captured image data 200 in FIG. 7.
In this example, the areas A1, A2, A5, and A6 are recognized as
texts, and the areas A3 and A4 are recognized as images.
[0114] (Step S202)
[0115] Next, the area selecting unit 110 determines whether or not
each area includes a text.
[0116] In the present embodiment, the area selecting unit 110
determines Yes if the area is classified into either a text or a
table. The area selecting unit 110 determines No if the area is
classified as an image or otherwise.
[0117] In the case of Yes, the area selecting unit 110 proceeds
with the process to step S203.
[0118] In the case of No, the area selecting unit 110 proceeds with
the process to step S205.
[0119] (Step S203)
[0120] If the area includes a text, the area selecting unit 110
determines whether or not to scan it.
[0121] If the accuracy of the classification is less than a
specific threshold, the area selecting unit 110 determines that the
classification has failed and thus determines Yes. Alternatively,
based on the analysis result of the state of the area, if OCR is
not possible, the area selecting unit 110 determines that the
result is Yes because of necessity to obtain a clearer image. In
addition, if the state of the area is unclear, the area selection
unit 110 determines Yes.
[0122] On the other hand, the area selecting unit 110 determines No
if the classification accuracy of the area of the captured image
data 200 is higher than the specific threshold and the state of the
area is suitable for OCR.
[0123] In the case of Yes, the area selecting unit 110 proceeds
with the process to step S204.
[0124] In the case of No, the area selecting unit 110 proceeds with
the process to step S205. In this case, the area selecting unit 110
selects to output the captured image data 200.
[0125] (Step S204, S101)
[0126] If scanning should be performed, the area selecting unit 110
in the server 2 perform a scan controlling process, and the
document reading unit 12 in the image forming apparatus 1 perform a
scanning process.
[0127] In this case, the area selecting unit 110 does not output
the captured image data 200 and selects to obtain the scan image
data 210. The area selecting unit 110 transmits a command to the
image forming apparatus 1 and causes the document reading unit 12
to scan the location of the area to obtain the scan image data
210.
[0128] More specifically, the area selecting unit 110 can cause the
document feeding unit 13 to convey the document D, and causes the
document reading unit 12 to scan only the portion corresponding to
the rectangular coordinate data of the area of the captured image
data 200. Alternatively, if it is necessary to scan a plurality of
areas in one page of the document, the area selecting unit 110 may
cause the document reading unit 12 to scan entire page of the
document D one by one, cuts out a portion corresponding to the
area, and output it. At this time, the area selecting unit 110 can
cause the area of the captured image data 200 to be scanned at a
resolution higher than that of the captured image data 200 with a
setting suitable for OCR, depending on whether the location of the
area of captured image data 200 is color, partial color, or
monochrome (black and white).
[0129] The area selecting unit 110 causes the scanned image data
210 to be transmitted from the image forming apparatus 1 to the
server 2. The area selecting unit 110 acquires this, stores it in
the storage unit 29 in association with the data indicating the
area.
[0130] (Step S205)
[0131] Here, the image processing unit 21 performs a content
recognition process.
[0132] When outputting the captured image data 200 with reference
to the data indicating the area, the image processing unit 21 cuts
out a portion of the rectangular coordinate data of the area of the
captured image data 200 and performs image recognition. On the
other hand, when the scan image data 210 is used, image recognition
is performed for the portion in this area.
[0133] When the area is classified as a text, the image processing
unit 21 recognizes, for example, a state of a column, a paragraph,
and a line, and recognizes a character position. When the area is a
table, the image processing unit 21 recognizes, for example, a form
and a character portion. Then, for both cases, the image processing
unit 21 performs OCR for the character portion.
[0134] In the case of an image, the image processing unit 21
recognizes non-text lines, symbols, illustrations, logos,
photographs, and the like. The image processing unit 21 also
recognizes these imaging objects.
[0135] Furthermore, the image processing unit 21 can recognize an
advertisement by combining various models.
[0136] (Step S206)
[0137] Next, the image processing unit 21 determines whether or not
the content of the area is related to the other text.
[0138] If the OCR is possible, the image processing unit 21
analyzes the content of the recognized text by natural language
processing and determines Yes if there is a relationship with
another text (the other text) already recognized in another area.
The determination of the content may be performed by, for example,
analysis of the frequency of occurrence of the same word, syntax
analysis, semantic analysis, or the like. Further, the image
processing unit 21 infers that the part of the area composed of
large characters and preceded to a group of characters, such as a
paragraph, is a title related to another text, or the like, and
determines that the result is Yes.
[0139] If the area is classified as an image and the recognition
result of the image indicates that the area is not an
advertisement, an illustration, a logo, or the like, the image
processing unit 21 determines that there is a possibility that the
area is related to another text, or the like, and determines
Yes.
[0140] On the other hand, if it is not related to other texts, the
image processing unit 21 determines No. Specifically, for example,
if the image is determined to be an advertisement by a model that
recognizes the advertisement, the image processing unit 21
determines No. Furthermore, the image processing unit 21 also
determines No if the classification of the area is an image and the
user has set that necessity of the image is "no image required," or
the like.
[0141] In the case of Yes, the image processing unit 21 proceeds
with the process to step S207.
[0142] In the case of No, the image processing unit 21 proceeds
with the process to step S208. As a result, the determined area is
deleted without being output.
[0143] (Step S207)
[0144] When it is related to the other text, the image processing
unit 21 performs an area processing and outputting process.
[0145] The image processing unit 21 obtains the location of the
rectangular coordinate data of the captured image data 200 or the
scanned image data 210 for each area, performs processing, and
outputs the processed area data 220.
[0146] As the processing, firstly, the image processing unit 21
converts the image data included in the area data 220 into
monochrome or partial color when the area is monochrome or partial
color. The same applies when monochrome or partial color is
designated as the format of the document data 230 by the user.
Alternatively, the image processing unit 21 also converts text,
images, and the like, having in a single color close to black to
monochrome. As described above, when not necessary to use the full
color, the size of the data of the area can be reduced by
converting into the monochrome or the partial color.
[0147] Further, when the area is classified as text, the image
processing unit 21 outputs the area data 220 replaced with the OCR
characters, and the image data can be deleted. Thereby, a stain and
the like are also deleted. Alternatively, depending on the format
of the document data 230, the image processing unit 21 may output
the area data 220 including text data in the image data as
"transparent text." In addition, the image processing unit 21 may
delete handwritten comments, figures, markers, and the like, if the
setting in the format of the document data 230 is to be deleted
these. Further, the image processing unit 21 may perform a check by
using a spell checker to detect and correct an OCR character
recognition error, an erroneous writing of the original document D
in the first place, and the like.
[0148] If the area is classified as a table, the image processing
unit 21 detects the location of the form. The image processing unit
21 converts this location of the form into vector data, table
structure data, or the like, and outputs the converted data as the
area data 220 corresponding to the format of the document data 230.
At this time, the image processing unit 21 includes the portion of
the text recognized by the OCR within this form and outputs them.
Furthermore, if processing that takes into account the semantics of
the document is required, the image processing unit 21 performs
processing that takes this into account. Specifically, for example,
when a process is required in consideration of the meaning of the
text term arranged in the form, the image processing unit 21
performs a process according to the meaning. For example, when
there is a "total (sum)" column, and when blanks or characters
cannot be recognized, the image processing unit 21 may calculate
this total and output it, or the like.
[0149] If the area is classified as an image, the image processing
unit 21 performs processing on the position of the rectangular
coordinate data of the captured image data 200 or the scan image
data 210 for image processing such as rotation and distortion
(skew) correction, density adjustment, gradation adjustment, image
improvement, or the like. Specifically, after performing skew
correction, the image processing unit 21 performs correction of
brightness, hue, exposure, or the like, ground color removal,
removal of stains or the like, noise removal, and the like, and can
optimize an image. Furthermore, an object similar to the recognized
imaging object may be generated and replaced by an AI such as GAN
(Generative Adversarial Network), or the like, to sharpen the
image. The image processing unit 21 outputs the image data on which
the image processing has been performed as the area data 220.
[0150] With reference to FIGS. 8 and 9, examples of the processing
area output process are described.
[0151] The area A1 is classified as text, is determined to be a
title, and shows an example in which only text is output as the
area data 220-1.
[0152] The area A2 is also classified as text, is determined to be
related to another text (other text), and shows an example in which
only text is output as the area data 220-2. Here, the stain and the
like indicated by light-colored circles in FIG. 8 has been
removed.
[0153] The area A3 shows an example that is recognized as an
advertisement image and deleted, that means, it is not output as
the area data 220.
[0154] The area A4 shows an example that is classified as an image,
is subjected to image processing, and is output as area data
220-4.
[0155] The area A5 shows an example that, despite having classified
as text, a stain, or the like, is large as shown by light-colored
circle in FIG. 8, and thus the scan image data 210 is obtained.
Also, in this example, the text is determined to be related to
another text, only the text is output as the area data 220-5, and
the stains, and the like, is thus removed.
[0156] The area A6 is classified as a text, is determined as an
advertisement not related to another text, is deleted, and is not
output as the area data 220.
[0157] (Step S208)
[0158] Here, whether or not the document output unit 120 has
processed all the areas is processed.
[0159] The document output unit 120 determines Yes if all the areas
of the same page of the document D have been processed. Otherwise,
the document output unit 120 determines No.
[0160] In the case of Yes, the document output unit 120 advances
the processing to step S209.
[0161] In the case of No, the document output unit 120 returns the
processing to step S201 and continues the processing of another
area.
[0162] (Step S209)
[0163] If all the areas have been processed, the document output
unit 120 performs a reconstruction process.
[0164] The document output unit 120 collects the area data 220 of
the same page and reconstruct the area data 220 into document data
230.
[0165] In the example of the area data 220 in FIG. 9, as described
above, an example that the area data 220-1, the area data 220-2,
the area data 220-3, and the area data 220-4 are arranged on the
same page as like the captured image data 200 is shown.
[0166] (Step S210)
[0167] Next, the document output unit 120 performs a document
output process.
[0168] The document output unit 120 transmits the document data 230
to the image forming apparatus 1 when the processing has been
completed for all pages.
[0169] In addition, the document output unit 120 may perform
processing corresponding to the output instruction, such as
transmitting the document to a location designated by the user.
[0170] (Step S102)
[0171] Here, the image forming unit 17 of the image forming
apparatus 1 performs an image forming process.
[0172] The image forming unit 17 receives the document data 230
transmitted from the server 2, forms an image, records the document
data 230 on a recording sheet, transmits a facsimile, and stores it
in a document box of the storage unit 19, and the like.
[0173] Note that the image forming apparatus 1 that performs this
image forming process may be an output apparatus other than the one
that has read the document D.
[0174] With the above, the document reading process according to
the embodiment of the present disclosure is ended.
[0175] With the above configuration, the following effects can be
retained.
[0176] In a typical technology, only to correct the brightness of
the entire page by a documenting apparatus by using a camera is
possible. For this reason, automatically improving the quality of
the document in which texts and images are mixed in the same page
has been quite difficult.
[0177] On the other hand, the image forming system X according to
the embodiment of the present disclosure is an image reading system
including the image forming apparatus 1 (image reading apparatus)
and the server 2 (information processing apparatus). The image
forming apparatus 1 including: a camera 60 that captures a document
in page units and acquires captured image data 200, and an image
reading unit 12 having a higher resolution than the camera 60; and
the server 2 including: an area classifying unit 100 that
classifies the captured image data 200 captured by the camera 60
into areas for each content, an area selecting unit 110 that
selects, for each area classified by the area classifying unit 100,
whether to output the captured image data 200 or not, based on
classification of the area and state of the captured image data
200, an image processing unit 21 that processes, for each area, the
captured image data 200 or scan image data 210 scanned by the image
reading unit 12, and outputs the area as an area data 220 or
deletes the area, and a document output unit 120 that collects the
area data 220 processed by the image processing unit 21,
reconstructs the data into document data 230, and outputs the
document data 230.
[0178] With this configuration, to capture an image of a document
by using the camera 60, analyze the document for each area, and
scan and process the image with an image scanner is possible. As a
result, this allows to scan only the necessary area with the image
scanner while acquiring the captured image data 200 with the camera
60 at higher speed than the image scanner. That is, high-speed
reading can be performed while ensuring the quality of the output
document data 230. Further, by performing the processing or
deleting based on the classification of the area, obtaining the
optimal document data 230 can be performed.
[0179] Further, in the image forming system. X according to the
embodiment of the present disclosure, classification of the area by
the area classifying unit includes a text, a table, and an image;
and the image processing unit 21 processes the captured image data
200 or the scanned image data 210 of the area classified into the
text or the table, and outputs area data 220 including character
data and/or table data.
[0180] With such a configuration, various types of original
document can be efficiently processed according to the analysis of
the text, table, and image for each area, and optimal document data
230 can be obtained. Further, by performing the processing
according to the classification of the area, the processing load
can be reduced, and the calculation resources can be saved. In
addition, by performing processing in accordance with a text, a
table, and an image, providing document data 230 that is easy to
read and has a reduced data amount is possible.
[0181] Further, in the image forming system X according to the
embodiment of the present disclosure, the area selecting unit 110
does not output the captured image data 200 and scans with the
document reading unit 12 when the area is classified as a text or a
table and state of the captured image data 200 is not suitable for
OCR.
[0182] With this configuration, even if the OCR, or the like, does
not work well at the resolution of the camera 60 or if the imaging
does not work well due to external light, or the like, the scan
image data 210 is acquired, and performing various processing
becomes possible. Therefore, high-speed image capturing by the
camera 60 and high-quality scanning by the image scanner can both
be achieved.
[0183] Further, in the image forming system. X according to the
embodiment of the present disclosure, the image processing unit 21
recognizes and deletes an area that is not related to the content
of another text (other text).
[0184] With such a configuration, unnecessary areas can be deleted
from the document data 230, and the document data 230 that is easy
to view can be provided. Further, the storage capacity of the
document data 230 can be reduced.
[0185] Further, the image forming system X according to the
embodiment of the present disclosure is that an image not related
to the content of the other text includes an advertisement, an
illustration, and a logo.
[0186] With such a configuration, advertisements, and the like, can
be efficiently removed.
Other Embodiments
[0187] In the above-described embodiment, an example has been
described in which the scanned image data 210 read by the document
reading unit 12 has higher quality than the captured image data 200
captured by the camera 60.
[0188] However, the quality of the captured image data 200 may be
higher than that of the scanned image data 210.
[0189] In this case, the captured image data 200 may be used as the
area data 220 instead of the scan image data 210. For example, when
there is a streak due to dirt on the document reading slit 12c, or
the like, a rectangular coordinate portion of the captured image
data 200 may be cut out.
[0190] With such a configuration, image data can be obtained in any
case, and a flexible configuration can be supported.
[0191] In the above-described embodiment, an example has been
described in which the locations of the rectangular coordinate data
of the captured image data 200 are classified into a plurality of
areas.
[0192] However, the number of areas may be one in one page.
Further, the area may be a polygon, or the like, that is not a
rectangle. Also, the area may be set within or with overlapped
another rectangle, polygon, or the like, within the rectangle, the
polygon, or the like.
[0193] In addition, the text, the table, and the image may be
classified or selected as an area in a mixed state,
respectively.
[0194] With this configuration, a flexible configuration can be
accommodated.
[0195] In the above-described embodiment, it has been described
that an area including text is scanned by the document reading unit
12 as necessary.
[0196] However, depending on the state of the captured image data
200 included in the area, the document may be scanned by the
document reading unit 12 without including the text.
[0197] Specifically, for example, when there is a poor external
light state, a hand or the like is in the image, dirt on the lens
of the camera 60, a fold or three-dimensional dirt on the document
D, or the like, the document reading unit 12 may perform
scanning.
[0198] With this configuration, when the quality of the captured
image data 200 is low, the scanned image data 210 is acquired, and
the high-quality document data 230 can be created.
[0199] In the above-described embodiment, an example has been
described in which the image processing unit 21 performs various
processes such as AI.
[0200] However, part or all of the processing of the image
processing unit 21 described above may be executed by the control
unit 10, the image processing unit 11, the document reading unit
12, an external client PC, another MFP, a server, or a service or
server on a so-called "cloud."
[0201] With such a configuration, depending on processing,
executing the processing more efficiently than the image processing
unit 21 is possible.
[0202] In the above-described embodiment, an example has been
described in which an area to be deleted is sequentially
determined.
[0203] However, the area to be deleted may be determined after
looking at the page or the entire document.
[0204] With this configuration, it is possible to reduce the
possibility that the advertisement is not deleted or that the
content of the document is erroneously deleted, and the document
data 230 from which the advertisement is reliably removed can also
be provided.
[0205] In the above-described embodiment, an example has been
described in which the area data 220 is arranged in the same manner
as a document.
[0206] However, in the document data 230, deleted areas may be
packed and arranged.
[0207] With this configuration, it is possible to reduce the number
of pages of the document data 230 by an amount corresponding to the
reduction of the advertisement, and the like.
[0208] In the above-described embodiment, an example in which the
camera 60 is provided in the image forming apparatus 1 has been
described.
[0209] However, a configuration that a camera imaging type scanner
including an imaging unit by using a camera, a USB camera, or the
like, may be separately connected to the image forming apparatus 1
is possible. Alternatively, a configuration in which an apparatus
including the document reading unit 12, the document feeding unit
13, and the camera 60 is connected to a printer may be employed.
Further, a configuration in which a camera such as a user's mobile
phone or smartphone is used as the camera 60 is also possible. In
these cases, the captured image data 200 may collectively be
transmitted to the image forming apparatus 1 beforehand, and then
scanned by an image scanner such as the document reading unit 12 if
necessary.
[0210] With such a configuration, the above-described processing
can be executed even in an image forming apparatus that does not
include the camera 60 in the apparatus.
[0211] Alternatively, the image reading apparatus of the present
disclosure can be applied to an information processing apparatus
other than the image forming apparatus 1. In other words, a
configuration may be employed in which a dedicated scanning device
such as a document camera or the like, a network scanner, a server
2 to which a scanner is separately connected via USB or the like,
another server, or the like is used.
[0212] Further, the configuration and operation of the
above-described embodiment are examples, and it is needless to say
that the configuration and operation can be appropriately changed
and executed without departing from the aim of the present
disclosure.
* * * * *