U.S. patent application number 15/185284 was filed with the patent office on 2016-10-13 for information processing apparatus, information processing system, method of processing information, and program.
The applicant listed for this patent is Sony Corporation. Invention is credited to Yutaka Hasegawa, Hiroshi Kyusojin, Yoichi Mizutani, Masahiro Takahashi.
Application Number | 20160301879 15/185284 |
Document ID | / |
Family ID | 47746098 |
Filed Date | 2016-10-13 |
United States Patent
Application |
20160301879 |
Kind Code |
A1 |
Kyusojin; Hiroshi ; et
al. |
October 13, 2016 |
INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING SYSTEM,
METHOD OF PROCESSING INFORMATION, AND PROGRAM
Abstract
An information processing apparatus, an information processing
system, and a method of processing information are provided. In one
embodiment, the information processing apparatus includes a
processor, and a memory device storing instructions. When executed
by the processor, the instructions cause the processor to receive,
from a first information processing apparatus, area specifying
information and location information, the area specifying
information specifying a display area in an image, the display area
including a plurality of partial images, the location information
indicating at least one location of the plurality of partial
images. The instructions further cause the processor to transmit,
to a second information processing apparatus, the area specifying
information and the location information.
Inventors: |
Kyusojin; Hiroshi; (Tokyo,
JP) ; Mizutani; Yoichi; (Saitama, JP) ;
Hasegawa; Yutaka; (Kanagawa, JP) ; Takahashi;
Masahiro; (Kanagawa, JP) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Sony Corporation |
Tokyo |
|
JP |
|
|
Family ID: |
47746098 |
Appl. No.: |
15/185284 |
Filed: |
June 17, 2016 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
13880267 |
Apr 18, 2013 |
9398233 |
|
|
PCT/JP2012/004448 |
Jul 10, 2012 |
|
|
|
15185284 |
|
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06F 19/321 20130101;
G16H 30/40 20180101; G16H 80/00 20180101; H04N 5/265 20130101; H04N
7/15 20130101; H04L 12/1827 20130101; H04M 3/567 20130101 |
International
Class: |
H04N 5/265 20060101
H04N005/265; H04L 12/18 20060101 H04L012/18; H04N 7/15 20060101
H04N007/15; H04M 3/56 20060101 H04M003/56; G06F 19/00 20060101
G06F019/00 |
Foreign Application Data
Date |
Code |
Application Number |
Aug 22, 2011 |
JP |
2011-180438 |
Claims
1. An information apparatus comprising: a processor; and a memory
device storing instructions which when executed by the processor,
cause the processor to: receive information including an image;
output the information to a display; in a case that an arbitrary
portion of the image is selected, obtain annotation information
affixed to the arbitrary portion from a first information
apparatus; and obtain other information from a second information
apparatus.
2. The information processing apparatus of claim 1, wherein the
annotation information is the information that the first
information apparatus downloads from the second apparatus in
advance.
3. The information processing apparatus of claim 1, wherein the
annotation information is selected from the group consisting of an
observation magnification, a coordinate moving path, a position in
the image to which the annotation is affixed, and combinations
thereof.
4. The information processing apparatus of claim 1, wherein the
annotation information comprises an operation record of the
image.
5. The information processing apparatus of claim 3, wherein the
operation record of the image is indicated as a display area.
6. The information processing apparatus of claim 1, wherein the
annotation information is identifiable as one or more of a symbol,
a diagram, a text, a voice, an image, and a line.
7. The information processing apparatus of claim 1, wherein the
image comprises data representative of a slice of a biological
tissue.
8. The information processing apparatus of claim 1, wherein the
instructions further cause the processor to obtain second
information of a second image.
9. The information processing apparatus of claim 8, wherein the
second information of the second image comprises a second
position.
10. The information processing apparatus of claim 9, wherein the
second position has a same coordinate as a first position to which
the annotation information is affixed.
11. A method of operating an information processing apparatus, the
method comprising causing a processor to execute instructions to
receive information of an image; output the information to a
display; when an arbitrary portion of an image is selected, obtain
annotation information affixed to the arbitrary portion from a
first information apparatus; and obtain other information from a
second information apparatus.
12. The method of claim 11, wherein the annotation information is
selected from the group consisting of an observation magnification,
a coordinate moving path, a position in the image to which the
annotation is affixed, and combinations thereof.
13. The method of claim 11, wherein the annotation information
comprises an operation record of the image.
14. The method of claim 11, wherein the annotation information is
identifiable as one or more of a symbol, a diagram, a text, a
voice, an image, and a line.
15. The method of claim 11, wherein the image comprises data
representative of a slice of a biological tissue.
16. The method of claim 11, further comprising causing the
processor to execute the instructions to obtain second information
of a second image.
17. The method of claim 16, wherein the second information of the
second image comprises a second position at a same coordinate as of
a first position to which the annotation information is
affixed.
18. A system comprising: a first information processing apparatus
configured to store annotation information affixed to an arbitrary
portion of an image; a second information processing apparatus
configured to store other information of the arbitrary portion of
the image; and a third information processing apparatus configured
to: receive information of the image; output the information to a
display; when the arbitrary portion of the image is selected,
obtain the annotation information affixed to the arbitrary portion
from the first information apparatus in advance; and obtain other
information from a second information apparatus.
19. The system of claim 18, wherein the annotation information is
selected from the group consisting of an observation magnification,
a coordinate moving path, a position in the image to which the
annotation is affixed, and combinations thereof.
20. The system of claim 1, wherein the annotation information
comprises an operation record of the image.
21. The system of claim 1, wherein the annotation information is
identifiable as one or more of a symbol, a diagram, a text, a
voice, an image, and a line.
Description
CROSS REFERENCES TO RELATED APPLICATIONS
[0001] The present application is a continuation of U.S.
application Ser. No. 13/880,267 filed Apr. 18, 2013, which is a
national stage of International Application No. PCT/JP2012/004448
filed on Oct. 7, 2012 and claims priority to Japanese Patent
Application No. 2011-180438 filed on Aug. 22, 2011, the disclosure
of which is incorporated herein by reference.
BACKGROUND
[0002] The present technology relates to an information processing
apparatus which may have a pathology image in common with another
information processing apparatus to make a user observe the image,
an information processing system having the information processing
apparatus, a method of processing information in the information
processing apparatus, and a program for the information processing
apparatus.
[0003] In the past, to support a teleconference among a plurality
of users, a system (e.g., a television conference system) in which
terminals of the users have a screen in common with one another as
a common information resource has been known.
[0004] For example, in Patent Literature (PTL) 1 described later,
an information communication service system, in which a conference
is expedited while each of user terminals takes out contents
registered in advance if necessary, is disclosed. In this system, a
server receives videos and voices transmitted from a plurality of
terminals in real time, synthesizes these ones, and delivers the
synthesized ones to the terminals.
[0005] Further, in PTL 2 and PTL 3 described later, a content
distributing system, in which contents are copied and dispersed to
a plurality of servers to be stored, is described.
[0006] Moreover, PTL 4 described later discloses that a
reproduction apparatus downloads in advance contents, of which the
date and time to be provided for viewers is predetermined, from a
server in a ciphered state, receives a key from the server at a
viewable date and time, and deciphers the contents with the key to
reproduce the contents.
[0007] Furthermore, in the field of pathological diagnosis,
terminals of a plurality of users (e.g., doctors) have an image for
pathological diagnosis in common with one another, and hold a
teleconference to make a diagnosis while transmitting opinions
among user's terminals. Therefore, the diagnosis is efficiently
made.
[0008] In relation to this, in PTL 5 described later, a server and
client system is disclosed. In this system, a server delivers a
plurality of tile images composing a pathology image to clients,
and each client synthesizes the pathology image from the tile
images and views the pathology image.
CITATION LIST
Patent Literature
[0009] [PTL 1]
[0010] Japanese Patent No. 3795772
[0011] [PTL 2]
[0012] Japanese Patent Application Laid-open No. 2003-115873
[0013] [PTL 3]
[0014] Japanese Patent Application Laid-open No. 2003-085070
[0015] [PTL 4]
[0016] Japanese Patent Application Laid-open No. 2010-119142
[0017] [PTL 5]
[0018] U.S. Pat. No. 7,542,596
SUMMARY
[0019] However, in the case of constructing a common system of a
pathology image, as described in PTL 1, in a tree type system for
concentrating delivered data into a single point, because it is
substantially required to gather data of a pathology image,
photographed in each of terminals (hospitals), to a server of a
cloud computing, it is time-consuming to upload the data. Further,
when contents are concentrated into a single point, the contents
act as a bottle neck, and scalability in simultaneously holding a
plurality of conferences or the like may not be exhibited.
[0020] In a common system of normal contents, the problem in
scalability may be solved by the technique described in each of PTL
2 and PTL 3. However, because data of a pathology image having a
very high resolution have a large volume, it is time-consuming, in
the same manner as in the uploading, to copy the data among a
plurality of servers.
[0021] Further, in the technique described in PTL 4, the load on
the server is decreased with the reduction of data communication
performed during the conference. However, when the communication
using the pathology image data is performed, it is very
time-consuming to download the data in advance. Moreover, because
only a slight portion of very huge volume of pathology image data
is used for the pathological diagnosis, most of the downloaded data
are not used. Therefore, efficiency in the download in advance is
very low in view of effective use of a network band.
[0022] In view of the circumstances as described above, it is
desirable to provide an information processing apparatus, an
information processing system, and a method of processing
information that are capable of shortening the time, taken to
prepare the holding of a teleconference in which data of a
pathology image are possessed in common, and expediting the
teleconference efficiently and smoothly.
[0023] An information processing apparatus according to an
embodiment of the present technology includes a processor, a
display device, and a memory device storing instructions. When
executed by the processor, the instructions cause the processor to:
(a) in response to an operation input, determine area specifying
information and location information, the area specifying
information specifying a display area in an image, the display area
including a plurality of partial images, the location information
indicating at least one location of the plurality of partial
images, (b) transmit the area specifying information and the
location information to a first information processing apparatus,
the first information processing apparatus being configured to, in
response to receiving the area specifying information and the
location information, transmit the area specifying information and
the location information to a second information processing
apparatus, (c) transmit a request for the plurality of partial
images to the at least one location indicated by the location
information, (d) receive the plurality of partial images from the
at least one location, and (e) display the received plurality of
partial images.
[0024] Therefore, because the information processing apparatus may
receive each of the partial images of the pathology image from an
arbitrary location being the same or different from locations of
the other partial images, as compared with the case where the
locations are concentrated into one point, the time required to
prepare a teleconference using the pathology image with another
information processing apparatus may be shortened. Further, because
the information processing apparatus is not required to determine
the display area in the pathology image, the teleconference using
the pathology image may be expedited efficiently and smoothly by
receiving the partial image according to the area specifying
information and by displaying the partial image.
[0025] The information processing apparatus may further have
storage. In this case, the processor may be configured to determine
whether or not the partial images included in the display area are
stored in the storage, and to be capable of controlling the
display, when the partial images are stored in the storage, to
display the partial images without transmitting the request
information to the location.
[0026] Therefore, when the partial images are stored in the
storage, it is not required that the information processing
apparatus access the location. Accordingly, the traffic on the
network may be reduced, and the teleconference using the pathology
image may be further efficiently expedited.
[0027] The processor may be configured to be capable of controlling
the information processing apparatus to receive, in advance, image
specifying information, temporary location information, and
diagnosis record information from the other information processing
apparatus before receiving the area specifying information and the
location information, the image specifying information specifying a
pathology image that is capable of being used in a teleconference
held with the other information processing apparatus, the temporary
location information indicating a location of the specified
pathology image at this time, the diagnosis record information
indicating a past diagnosis record of the specified pathology
image, and to receive a partial image, associated with the received
diagnosis record information, among a plurality of partial images
composing the pathology image specified by the received image
specifying information, from the location indicated by the
temporary location information. The processor may be configured to
be capable of controlling the storage to store the received partial
image.
[0028] Therefore, the information processing apparatus may download
in advance a specific partial image, which may be used in the
teleconference at a high probability, to store the specific partial
image in the storage. Accordingly, the traffic during the
teleconference may be reduced, and the teleconference using the
pathology image may be efficiently expedited.
[0029] The processor may be configured to be capable of controlling
the information processing apparatus to receive display record
information, indicating one of a past display area and a display
position in the specified pathology image, as the diagnosis record
information. Therefore, the information processing apparatus may
further efficiently expedite the teleconference using the pathology
image by downloading in advance a partial image which may be
displayed in the teleconference at a high probability in the same
manner as in the past.
[0030] The processor may be configured to be capable of controlling
the information processing apparatus to receive annotation
information, affixed to a predetermined position in the specified
pathology image, as the diagnosis record information.
[0031] Therefore, the information processing apparatus may further
efficiently expedite the teleconference by downloading in advance
the partial image to which annotation information is affixed and
which may attract attention in the teleconference at a high
probability in the same manner as in the past.
[0032] The pathology image may exist for each of a plurality of
slices, which are taken out from one biological tissue and are
stained with different colors, respectively, to form a plurality of
pathology images. In this case, the processor may be configured to
be capable of controlling the information processing apparatus to
receive a partial image existing at a predetermined position, to
which the annotation information is affixed, in a first pathology
image of a slice stained with a first color, and to receive a
partial image existing at the position same as the predetermined
position in a second pathology image of a slice stained with a
second color.
[0033] Therefore, the information processing apparatus may download
in advance not only the partial image of the first pathology image,
to which the annotation information is affixed, but also the
partial image of the second pathology image which is taken from the
biological tissue in common with the first pathology image but is
of a slice stained in a color differing from that in the first
pathology image.
[0034] Images of a slice taken out from one biological tissue may
be taken at a plurality of different resolutions, and the pathology
image may exist for each of the plurality of resolutions to form a
plurality of pathology images. In this case, the processor may be
configured to be capable of controlling the information processing
apparatus to receive a partial image existing at a predetermined
position, to which the annotation information is affixed, in a
first pathology image taken at a first resolution, and to receive a
partial image existing at the position same as the predetermined
position in a second pathology image taken at a second
resolution.
[0035] Therefore, the information processing apparatus may download
in advance not only the partial image of the first pathology image,
to which the annotation information is affixed, but also the
partial image of the second pathology image which is taken from the
biological tissue in common with the first pathology image but of
which the resolution differs from the resolution of the first
pathology image.
[0036] The processor may be configured to be capable of controlling
the information processing apparatus to receive the area specifying
information from the other information processing apparatus via a
first server apparatus. In this case, the location may indicate a
second server apparatus different from the first server
apparatus.
[0037] Therefore, the pathology image and the area specifying
information are managed in different servers, respectively.
Accordingly, it may be prevented that the concentration of the load
on a specific server disturbs the expedition of the
teleconference.
[0038] An information processing apparatus according to another
embodiment includes a processor, and
[0039] a memory device storing instructions. When executed by the
processor, the instructions cause the processor to: (a) receive,
from a first information processing apparatus, area specifying
information and location information, the area specifying
information specifying a display area in an image, the display area
including a plurality of partial images, the location information
indicating at least one location of the plurality of partial
images; and (b) transmit, to a second information processing
apparatus, the area specifying information and the location
information.
[0040] According to another embodiment of the present technology,
there is provided an information processing apparatus comprising: a
processor, a display device, and a memory device storing
instructions. When executed by the processor, the instructions
cause the processor to: (a) receive area specifying information and
location information from a first information processing apparatus,
the area specifying information specifying a display area in an
image, the display area including a plurality of partial images,
the location information indicating at least one location of the
plurality of partial images, (b) transmit a request for the
plurality of partial images to the at least one location indicated
by the location information, (c) receive the plurality of partial
images from the at least one location, and (d) display the received
plurality of partial images.
[0041] According to still another embodiment of the present
technology, there is provided a method of operating an information
processing apparatus. The method includes (a) causing a processor
to execute instructions to receive, from a first information
processing apparatus, area specifying information and location
information, the area specifying information specifying a display
area in an image, the display area including a plurality of partial
images, the location information indicating at least one location
of the plurality of partial images, and (b) causing the processor
to execute instructions to transmit, to a second information
processing apparatus, the area specifying information and the
location information.
[0042] According to still another embodiment of the present
technology, there is provided a system including a first, second,
and third information processing apparatus. The first information
processing apparatus is configured to, in response to an operation
input, determine area specifying information and location
information, the area specifying information specifying a display
area in an image, the display area including a plurality of partial
images, the location information indicating at least one location
of the plurality of partial images. The second information
processing apparatus is configured to receive the area specifying
information and the location information from the first information
processing apparatus. The third information processing apparatus is
configured to: (a) receive the area specifying information and the
location information from the second information processing
apparatus, (b) transmit a request for the plurality of partial
images to the at least one location indicated by the location
information, (c) receive the plurality of partial images from the
at least one location; and (d) display the received plurality of
partial images.
[0043] As described above, according to the present technology, the
time required to prepare the holding of a teleconference in which
data of the pathology image are possessed in common may be
shortened, and the teleconference may be expedited efficiently and
smoothly.
BRIEF DESCRIPTION OF THE FIGURES
[0044] FIG. 1 is a block diagram showing the configuration of a
teleconference system according to a first embodiment of the
present technology.
[0045] FIG. 2 is a block diagram showing the configuration of
hardware of a PC in the system.
[0046] FIG. 3 is a diagram explaining a display principle of a
pathology image treated in the system.
[0047] FIG. 4 is a diagram indicating the procedure in the case of
producing an image group of the pathology image treated in the
system.
[0048] FIG. 5 is a flowchart showing the processing in a PC being
the chairman when a pathology image is displayed in the system.
[0049] FIG. 6 is a flowchart showing the processing in a PC being
an audience when a pathology image is displayed in the system.
[0050] FIG. 7 is a block diagram showing the configuration of a
teleconference system according to a second embodiment of the
present technology.
[0051] FIG. 8 is a diagram showing the specification of tiles, to
be downloaded in advance, on the basis of past operation record
information in the system according to the second embodiment.
[0052] FIG. 9 is a diagram showing the specification of tiles, to
be downloaded in advance, on the basis of past operation record
information in the system according to the second embodiment.
[0053] FIG. 10 is a diagram showing the specification of tiles, to
be downloaded in advance, on the basis of past operation record
information in the system according to the second embodiment.
[0054] FIG. 11 is a diagram showing the specification of a tile, to
be downloaded in advance, according to past annotation information
in the system according to the second embodiment.
[0055] FIG. 12 is a diagram showing the production of a pathology
slide from a plurality of slices of a biological tissue.
[0056] FIG. 13 is a flowchart showing one example of the processing
in a PC and a server when a pathology image is downloaded in
advance in the system according to the second embodiment.
[0057] FIG. 14 is a flowchart showing another example of the
processing in a PC and a server when a pathology image is
downloaded in advance in the system according to the second
embodiment.
[0058] FIG. 15 is a flowchart showing the processing in a PC being
an audience when a pathology image is displayed in the system.
[0059] FIG. 16 is a block diagram showing the configuration of a
teleconference system according to a third embodiment.
[0060] FIG. 17 is a block diagram showing the configuration of a
teleconference system according to a fourth embodiment.
[0061] FIG. 18 is a block diagram showing the configuration of a
teleconference system according to a fifth embodiment.
[0062] FIG. 19 is a flowchart showing the processing in a PC
according to the fifth embodiment.
[0063] FIG. 20 is a block diagram showing the configuration of a
teleconference system according to a sixth embodiment.
[0064] FIG. 21 is a block diagram showing the configuration of a
teleconference system according to a seventh embodiment.
DESCRIPTION OF EMBODIMENTS
[0065] Hereinafter, embodiments according to the present technology
will be described with reference to drawings.
First Embodiment
[0066] Initially, the first embodiment according to the present
technology will be described.
[0067] [Outline of System]
[0068] FIG. 1 is a block diagram showing the configuration of a
teleconference system using a pathology image according to the
first embodiment. In this system, a diagnosis is efficiently made
by holding a teleconference, while personal computers (PCs) of a
plurality of users (e.g., doctors) have an image for pathological
diagnosis (a pathology image) in common with one another, and by
making a diagnosis while opinions are transmitted among the
PCs.
[0069] As shown in this figure, this system has a server 200, a
plurality of PCs 100, and a scanner 300. These may communicate with
one another via the Internet 150.
[0070] Data of a pathology image taken by the scanner 300 are
uploaded to the server 200 via the Internet 150 and are stored. As
described later in detail, the pathology image is obtained by
taking an image of a slice of a biological tissue or the like held
in a glass slide. The pathology image data are not stored only in
the server 200, but may be stored in any of the PCs 100.
[0071] In this system, one of the plurality of PCs 100 acts as the
"chairman" of the teleconference, and the other ones act as
"audiences". A pathology image specified by the PC 100 as the
chairman is displayed on the PCs 100 as audiences.
[0072] As described later in detail, the PC 100 acting as the
chairman transmits information (area specifying information),
specifying a display area in the pathology image for a diagnosis in
the teleconference, and information (URL; uniform resource
locator), indicating locations of the pathology image data, to the
server 200. The server 200 transmits the area specifying
information and the URL to the PCs 100 acting as the audiences.
That is, the PC 100 acting as the chairman transmits the area
specifying information and the URL to the PCs 100 acting as the
audiences via the server 200.
[0073] Each PC 100 acting as the audience receives an image (a
partial image) of the pathology image existing in the area,
specified by the area specifying information, from the URL and
displays the partial image. Therefore, the PCs 100 have the
pathology image in common with one another, and a pathological
diagnosis may be made by the users of the PCs 100.
[0074] [Hardware Configuration of PC]
[0075] FIG. 2 is a block diagram showing the hardware configuration
of one PC 100.
[0076] Each PC 100 has a central processing unit (CPU) 11, a read
only memory (ROM) 12, a random access memory (RAM) 13, an
input/output interface 15, and a bus 14 connecting these ones with
one another.
[0077] The input/output interface 15 is connected with a display
16, an input section 17, storage 18, a communicating section 19, a
driver 20, and the like.
[0078] The display 16 is, for example, a display device using
liquid crystal, electro-luminescence (EL) or the like.
[0079] The input section 17 is, for example, a pointing device, a
keyboard, a touch panel, a microphone, or another operating device.
When the input section 17 includes a touch panel, the touch panel
may be integrally formed with the display 16.
[0080] The storage 18 is a nonvolatile storage device, for example,
being a hard disk drive (HDD), a flash memory, or another solid
memory. In the storage 18, not only the pathology image data are
stored, but also an application program, to be executed to receive
and display the pathology image data in this system, is stored.
[0081] The driver 20 is, for example, a device capable of driving a
removable recording medium 21 such as an optical recording medium,
a floppy (registered trademark) disk, a magnetic recording tape, or
a flash memory. In contrast, the storage 18 is often used as a
device which is mounted in the PC 100 in advance and drives a
non-removable recording medium.
[0082] The communicating section 19 is a modem, a router, or
another communicating device which communicates with another device
and may be connected to a local area network (LAN), a wide area
network (WAN), or the like. The communicating section 19 may use
any of wire communication and wireless communication. The
communicating section 19 is often used while being formed
independent of the PC 100.
[0083] The hardware configuration of the server 200 is also the
same as the hardware configuration of the PC 100 and has blocks
such as a control section, storage, and a communicating section
being necessary to act as a computer.
[0084] Next, a pathology image, which may be stored in the server
200 or the storage 18 of the PC 100, and a display principle of the
image will be described. FIG. 3 is a diagram showing an image
pyramid structure for explaining the display principle.
[0085] An image pyramid structure 50 according to this embodiment
is an image group (a total image group) of which images are
produced from a single pathology image, obtained from a single
observation object 40 (see FIG. 4) by an optical microscope, at a
plurality of different resolutions respectively. The image having
the largest size is located in the lowest layer of the image
pyramid structure 50, while the image having the smallest size is
located in the highest layer. The resolution of the largest sized
image is, for example, 50.times.50 kilopixels or 30.times.40
kilopixels. The resolution of the smallest sized images is, for
example, 256.times.256 pixels or 256.times.512 pixels.
[0086] That is, when the same display 16 displays these images, for
example, in 100% size (each of the images is displayed at the
number of physical dots which is the same as the number of pixels
in the image), the image having the largest size is displayed in
the largest size, and the image having the smallest size is
displayed in the smallest size. Here, in FIG. 3, the display area
of the display 16 is indicated by D.
[0087] FIG. 4 is a diagram for explaining the procedure in the case
of producing the image group of the image pyramid structure 50.
[0088] Initially, a digital image of an original image (a huge
image) obtained at a predetermined observation magnification by an
optical microscope (not shown) is prepared. This original image is
equivalent to the largest sized image being the lowest image of the
image pyramid structure 50 shown in FIG. 3, and is an image having
the highest resolution. Therefore, as the lowest image of the image
pyramid structure 50, an image obtained by observing at a
comparatively high resolution by the optical microscope is
used.
[0089] In the field of pathology, generally, a slice thinly cut off
from a living internal organ, a biological tissue, a cell, or a
part of any one of these is an observation object 40. Then, the
observation object 40 held in a glass slide is read out by the
scanner 300 having the function of an optical microscope, and an
obtained digital image is stored in the scanner 300 or other
storage.
[0090] As shown in FIG. 4, this scanner 300 or a generally-used
computer (not shown) produces a plurality of images which
respectively have resolutions lowered step by step from the largest
sized image obtained as described above, and stores these images,
for example, every "tile" (partial image) unit denoting a unit of a
predetermined size. The size of one tile is, for example,
256.times.256 pixels. To each tile, identification information (an
ID or a number) identifying the tile is added.
[0091] The image group produced as described above forms the image
pyramid structure 50, and this image pyramid structure 50 is stored
in the storage 18 of the PC 100 or storage of the server 200.
Practically, the PC 100 or the server 200 may store the images
having a plurality of different resolutions and pieces of
information of the resolutions while the images are associated with
the pieces of information respectively. The PC 100 may perform the
production and storage of the image pyramid structure 50.
[0092] The total image group forming this image pyramid structure
50 may be produced by a known compression method, and may be, for
example, produced by a known compression method for producing a
thumbnail image.
[0093] When the user of each PC 100 views the pathology image
stored in the storage 18 of the PC 100 not in a teleconference but
in stand-alone, the PC 100 extracts a desired image from the image
pyramid structure 50 in response to the operation of the user input
from the input section 17 and displays this image on the display
16. In this case, the PC 100 displays an image of an arbitrary
portion, selected by the user, from among images having an
arbitrary resolution selected by the user. While the user changes
the observation magnification, the user may obtain a feeling that
the user actually observes the observation object 40. That is, in
this case, the PC 100 acts as a virtual microscope, and a virtual
observation magnification is practically equivalent to the
resolution described above.
[0094] [Operation of System]
[0095] Next, an operation of the system configured as described
above will be described. Hereinafter, an operation will be
described while especially setting the CPU 11 of the PC 100 as a
main operating subject, and the operation is performed in
cooperation with other blocks of each device and software
(application).
[0096] FIG. 5 is a flowchart showing the processing in the PC 100
being the chairman when a pathology image is displayed according to
this embodiment, while FIG. 6 is a flowchart showing the processing
in the PC 100 being one audience in this case.
[0097] As shown in FIG. 5, the CPU 11 of the PC 100 being the
chairman waits for an operation input of the user which specifies a
display area in a specific pathology image (step 51) and specifies
the display area when receiving this operation input (step 52).
[0098] Then, on the basis of the specified display area, the CPU 11
specifies tiles of the pathology image used for the display
process, that is, tiles (partial images) of which all or a part is
included in the display area (step 53).
[0099] Then, the CPU 11 transmits a request for data of the
specified tiles to the server 200 (step 54). On the other hand, the
CPU 11 transmits display area information indicating the display
area and a URL of the specified tile data to the server 200 (step
55).
[0100] Then, the CPU 11 receives the tile data transmitted from the
server 200 in response to the request (step 56) and displays the
tile data on the display (step 57).
[0101] On the other hand, as shown in FIG. 6, the CPU 11 of the PC
100 being each audience receives the display area information and
the URL transmitted from the PC 100 being the chairman via the
server 200 (step 61).
[0102] Then, the CPU 11 specifies tiles of the pathology image
required for the display process of the display area on the basis
of the display area information (step 62).
[0103] Then, the CPU 11 transmits the request for the specified
tiles to the server 200 on the basis of the received URL (step
63).
[0104] Then, the CPU 11 receives tiles transmitted in response to
the request (step 64), and displays the tiles on the display 16
(step 65).
Second Embodiment
[0105] Next, the second embodiment according to the present
technology will be described.
[0106] FIG. 7 is a block diagram showing the configuration of a
teleconference system according to this embodiment.
[0107] In the first embodiment described above, the pathology image
is stored in the server 200, and is downloaded to the PC 100 on the
day on which the teleconference is held. However, as shown in this
figure, when the day fixed for the teleconference and a pathology
image used in the teleconference are predetermined, the PC 100 may
also download the pathology image to the storage 18 before the
conference is held. This downloading may be performed in a tile
unit without being performed in a unit of the whole pathology
image.
[0108] In the above-described download in a tile unit in advance,
each tile being used in the teleconference at a high probability
may be downloaded. The tiles being used at a high probability are,
for example, the tiles used for a pathological diagnosis in the
past. The tiles used for a past pathological diagnosis are, for
example, specified according to a following criterion.
[0109] That is, the PC 100 specifies tiles used for a past
diagnosis on the basis of a display operation record indicating
that at which magnification (resolution) a doctor making a
diagnosis in the past viewed a pathology image and how the doctor
moved coordinates or a display area.
[0110] FIG. 8, FIG. 9, and FIG. 10 are diagrams showing examples of
tiles specified on the basis of this operation record
information.
[0111] As shown in FIG. 8, when the operation record of a pathology
image I is indicated as a coordinate moving path 71 being a line,
tiles 70 (tiles to which the line belongs) on which the line goes
across are specified. Further, when the operation record is
indicated by a point (specific coordinates), a tile including this
point is specified.
[0112] Further, as shown in FIG. 9, even in the same case as that
in FIG. 8, tiles 70 being formed in a rectangle and including the
line may be specified.
[0113] Further, as shown in FIG. 10, when the operation record of
the pathology image I is indicated as a display area 72 of a
rectangle, each tile 70 of which all or a part is included in this
rectangle is specified.
[0114] Furthermore, when some diagnosis annotation is affixed to a
pathology image by a doctor in a past diagnosis, the PC 100
specifies a tile to which this annotation is affixed. The
annotation denotes information which is, for example, formed in a
style of a symbol, a diagram, a text, a voice, an image, a link
(e.g., URL), or the like on the basis of user's input on the PC
displaying the pathology image.
[0115] For example, as shown in FIG. 11, a tile 70, to which an
annotation 73 indicated by a symbol X is affixed, among tiles
composing the pathology image I is specified. When the annotation
indicated by a line, a rectangle, or a circle is affixed, in the
same manner as in FIG. 8 to FIG. 10, tiles on which the line goes
across or tiles included in the rectangle or the circle are
specified.
[0116] FIG. 12 is a diagram showing the preparation of slides
(pathology slides) in which slices are held to be photographed for
pathology images.
[0117] As shown in this figure, a pathology slide 80 is prepared by
cutting off the observation object (a slice) 40 at the thickness of
two to three micrometers from a pathology piece (a biological
tissue) S taking out by a surgery or the like, mounting the slice
40 on a slide glass 81, and covering the slice 40 with a cover
glass 82.
[0118] Accordingly, slices successively cutting off have
approximately the same tissue shape and the same medical feature of
cell and the like. That is, as shown in this figure, diagnosis
information affixed to a pathology slide 80A of a slice 40A is also
useful for an adjacent pathology slide 80B of a slice 40B.
[0119] There is the case where the different slices 40A and 40B are
stained with different colors to perform different inspections
respectively. Accordingly, when a plurality of pathology images
relating to the successive slices stained with different colors
exist, the PC 100 specifies a tile of one pathology image to which
an annotation is affixed, and specifies a tile of another pathology
image existing at the same coordinates as those of the position at
which the annotation is affixed.
[0120] Further, as described above, the user may see one pathology
image while changing the observation magnification (the
resolution). Accordingly, when an annotation is affixed to a
certain pathology image, this annotation is also useful information
for another pathology image of the same observation object of which
the magnification (the resolution) differs from that of the certain
pathology image. Accordingly, when an annotation is affixed to a
certain pathology image, the PC 100 specifies a tile of the certain
pathology image to which the annotation is affixed, and also
specifies a tile, which includes the same coordinates as
coordinates at which the annotation is affixed, in another
pathology image of the same observation object having a different
resolution.
[0121] FIG. 13 is a flowchart showing the processing in the PC 100
and the server 200 in the download of a pathology image in
advance.
[0122] As shown in this figure, initially, pathology image data to
be used for a teleconference and a list of participants of the
teleconference are transmitted from the PC 100 of the promoter of
the teleconference to the server 200 and are registered (step 131).
This process is, for example, performed by selecting a desired
pathology image and users from data of all pathology images and a
list of all users existing in the server 200.
[0123] Then, the server 200 transmits a notice of a conference
holding to the PC 100 of each participant (step 132).
[0124] Then, the PC 100 of each participant determines whether or
not the storage 18 such as a HDD having a capacity capable of
downloading a pathology image in advance exists in the PC 100 (step
S133).
[0125] When a HDD capable of downloading exists (YES), the PC 100
of the participant receives a list of pathology image data to be
used in the teleconference from the server 200 (step 134). In this
case, past diagnosis record information relating to these pathology
image data and the URL of the pathology image data (tiles) at this
time are also received.
[0126] Then, the PC 100 specifies tiles according to the criterion
described above on the basis of the received list of pathology
image data and the received diagnosis record information, and
downloads the specified tiles from the server 200 (step 135).
[0127] Then, when the start time of the teleconference comes, the
PC 100 of the participant may participate in the conference by
being connected to the server 200 (step 136).
[0128] In the example shown in FIG. 13, at the time when the
promoter registers the pathology image data to the server 200, the
list of data to be used in the conference is determined. However,
practically, in the period of time from the time when a request for
the holding of a conference is transmitted to the time when the
conference is held, there is the case where the list is updated by
deleting unnecessary data by the promoter, or by adding, by another
participant, data that this participant wishes to use.
[0129] Therefore, as shown in the flowchart of FIG. 14, there is
the idea that the list is maintained to an updated one by polling
the list.
[0130] That is, as shown in this figure, at step 141 to step 145,
after the PC 100 performs the same processes as those at step 131
to step 135 of FIG. 13, the PC 100 determines whether or not the
start time of the teleconference comes (step 146).
[0131] Then, when the start time of the teleconference does not yet
come (NO), the PC 100 accesses the server 200 and determines
whether or not the list of pathology image data to be used has been
updated (step 147).
[0132] When the list has been updated (YES), the PC 100 receives a
new list from the server 200 and repeats the processes performed at
step 144 and steps following step 144.
[0133] Here, in place of the processes at step 146 and step 147,
the PC 100 may receive a new list in response to a notice that
indicates the update of the list and is sent from the server 200 to
the PC 100 of each participant.
[0134] Further, the destination to which tiles are downloaded in
advance is not limited to the storage 18 of the PC 100. For
example, the downloading to a neighboring storage device connected
with the PC 100 via a network is possible. When there are a
plurality of servers 200, the downloading to the server 200 closest
to the PC 100 is allowed.
[0135] FIG. 15 is a flowchart showing the processing in the PC 100
being an audience when a pathology image is displayed in the system
according to this embodiment.
[0136] As shown in this figure, after the CPU 11 of the PC 100
performs the same processes as those at step 61 and step 62 shown
in FIG. 6 according to the first embodiment (steps 151 and 152),
the CPU 11 determines whether or not specified tiles are stored in
the storage 18 (whether or not specified tiles have been downloaded
in advance) (step 153).
[0137] When the tiles do not exist in the storage 18 (NO), the CPU
11 transmits a request for the tiles to the server 200 (step 154),
and receives the tiles (step 156).
[0138] When the tiles are stored in the storage 18 (YES), the CPU
11 reads out the tiles from the storage 18 (step 155).
[0139] Then, the CPU 11 displays the tiles, received or read out,
on the display 16 (step 157).
[0140] As described above, according to the download in advance,
unnecessary communication on the date of the teleconference is
reduced. Accordingly, the load on the server 200 expected when the
number of PC 100 being audiences is increased may be reduced, and
unnecessary packets transmitted through the network may be
avoided.
Third Embodiment
[0141] Next, a third embodiment according to the present technology
will be described.
[0142] FIG. 16 is a block diagram showing the configuration of a
teleconference system according to this embodiment.
[0143] In each of the embodiments described above, both the display
area information and the tile data of the pathology image are
stored in the server 200. However, it is not necessary to process
both ones in the same server. Therefore, as shown in this figure,
in this embodiment, a tile data server 200A for storing the tile
data and a control data server 200B for processing the display area
information are separately located. Further, in the control data
server 200B, the past diagnosis record information described in the
second embodiment, the list of tile data, the list of participants,
and the like are managed (hereinafter, these data are called
control data in a lump).
[0144] Therefore, the server is optimized for control data which
has a small data capacity but needs a rapid response, and tile data
which has a large data capacity and needs a high throughput.
[0145] Further, when the server 200B treating the control data does
not exist, the control data may be exchanged in the P2P
communication among a plurality of PC 100.
Fourth Embodiment
[0146] Next, a fourth embodiment according to the present
technology will be described.
[0147] FIG. 17 is a block diagram showing the configuration of a
teleconference system according to this embodiment.
[0148] As shown in this figure, a teleconference is held among a
plurality of hospitals by connecting in-hospital networks 170 with
one another. When pathology image data to be used in the conference
are gathered to a single point, the cost for uploading and
downloading is increased.
[0149] In contrast, pathology image data to be used in the
teleconference are originally obtained by taking an image by a
scanner 300 of each hospital and are retained in the in-hospital
servers 200A. Therefore, as shown in this figure, in this
embodiment, the in-hospital server 200A of each hospital is used in
place of the tile data server 200A in the third embodiment. In this
case, the control data server 200B also exists on the Internet
150.
[0150] Because of this configuration, the upload of the tile data
to the server on the Internet 150 becomes unnecessary.
Fifth Embodiment
[0151] Next, a fifth embodiment according to the present technology
will be described.
[0152] FIG. 18 is a block diagram showing the configuration of a
teleconference system according to this embodiment.
[0153] In the fourth embodiment, it is substantially required that
the in-hospital server delivers the tile data to participants of a
teleconference. Therefore, there is a probability that a problem
occurs in view of the load on the server depending on the network
bandwidth and the number of the PCs 100.
[0154] Therefore, in this embodiment, as shown in this figure, tile
data to be downloaded are uploaded to a cloud server 200C in
advance. Accordingly, the load on the in-hospital server 200D is
reduced.
[0155] FIG. 19 is a flowchart showing the display process in the PC
100 according to this embodiment.
[0156] As shown in this figure, in the same manner as the
processing in FIG. 6 and FIG. 15, when specifying tiles to be used
(steps 191 and 192), the CPU 11 of the PC 100 transmits a request
for the specified tiles to the cloud server 200C (step 193).
[0157] Then, the CPU 11 determines whether or not the tiles to be
requested are successfully received (step 194).
[0158] When failing in the reception of the tiles (NO), the CPU 11
transmits a request for the tiles to the in-hospital server 200D
(step 196), and receives the tiles (step 197). Thereafter, the
received tiles are displayed (step 197).
[0159] Because of this processing, the working volume of the
uploading of the tile data to the server 200C is suppressed, and
the load on the in-hospital server 200D is also reduced.
Sixth Embodiment
[0160] Next, a sixth embodiment according to the present technology
will be described.
[0161] FIG. 20 is a block diagram showing the configuration of a
teleconference system according to this embodiment.
[0162] In each of the embodiments described above, there is only
the single tile server. However, as shown in this figure, in this
embodiment, a plurality of tile servers for delivering the same
tile data are located (tile servers 200A-1 and 200A-2). For
example, the tile server 200A-1 is set as a primary server, and the
tile server 200A-2 is set as a secondary server.
[0163] Therefore, because a server optimum to one PC 100 being a
client is determined for each PC 100, the system may cope with the
increase in the number of PCs 100. In this case, each of the
following server determining algorithms may be used.
[0164] A server is statistically allocated to one client in advance
for each client.
[0165] The client (the PC 100) selects the server having the
smallest RTT (Round Trip Time).
[0166] The client (the PC 100) selects the server having the widest
bandwidth.
[0167] The primary server (the tile server 200A-1) determines while
considering the total load.
[0168] The roundrobin processing is performed.
[0169] The load information of the server is periodically (e.g.,
every 10 seconds) obtained.
Seventh Embodiment
[0170] Next, a seventh embodiment according to the present
technology will be described.
[0171] FIG. 21 is a block diagram showing the configuration of a
teleconference system according to this embodiment.
[0172] As shown in this figure, in an in-hospital system in an
in-hospital network 170, tile data for which a diagnosis was
finished is retreated from the in-hospital network 170 to the cloud
server 200C via the Internet 150.
[0173] Then, when the reservation of an in-hospital conference is
performed in the in-hospital system while using the retreated data,
the tile data retreated to the cloud server 200C is returned to the
in-hospital server 200D at the time of the reservation.
[0174] [Modifications]
[0175] The present technology is not limited to only the
embodiments described above, and may be variously modified within
the scope not departing from the subject matter of this
disclosure.
[0176] The configurations of the first to seventh embodiments may
be combined with one another in any manner if the combined
embodiments are not contradictory to one another. For example, the
download process in advance in the second embodiment may be
embodied in the configuration that the server is separately divided
into the tile data server and the control data server as described
in the third embodiment.
[0177] [Others]
[0178] In the present technology, the following configurations may
be adopted.
[0179] In an embodiment, an information processing apparatus
includes a processor and a memory device storing instructions. When
executed by the processor, the instructions cause the processor to:
(a) receive, from a first information processing apparatus, area
specifying information and location information, the area
specifying information specifying a display area in an image, the
display area including a plurality of partial images, the location
information indicating at least one location of the plurality of
partial images; and (b) transmit, to a second information
processing apparatus, the area specifying information and the
location information.
[0180] In the information processing apparatus according to an
embodiment, the image includes data representative of a slice of a
biological tissue.
[0181] In the information processing apparatus according to an
embodiment, (a) the information processing apparatus includes a
server, (b) the first information processing apparatus includes a
first personal computer; and (c) the second information processing
apparatus includes a second personal computer.
[0182] In the information processing apparatus according to an
embodiment, the plurality of partial images includes (a) a first
partial image stored at a first location, and (b) a second partial
image stored at a second location.
[0183] In the information processing apparatus according to an
embodiment, the area specifying information is determined based on
a display record.
[0184] In the information processing apparatus according to an
embodiment, the area specifying information is determined based on
annotation information.
[0185] In another embodiment, an information processing apparatus
includes a processor, a display device, and
[0186] a memory device storing instructions. When executed by the
processor, the instructions cause the processor to: (a) in response
to an operation input, determine area specifying information and
location information, the area specifying information specifying a
display area in an image, the display area including a plurality of
partial images, the location information indicating at least one
location of the plurality of partial images, (b) transmit the area
specifying information and the location information to a first
information processing apparatus, the first information processing
apparatus being configured to, in response to receiving the area
specifying information and the location information, transmit the
area specifying information and the location information to a
second information processing apparatus, (c) transmit a request for
the plurality of partial images to the at least one location
indicated by the location information, (d) receive the plurality of
partial images from the at least one location; and (e) display the
received plurality of partial images.
[0187] In the information processing apparatus according to an
embodiment, the image includes data representative of a slice of a
biological tissue.
[0188] In the information processing apparatus according to an
embodiment, (a) the information processing apparatus includes a
first personal computer, (b) the first information processing
apparatus includes a server; and (c) the second information
processing apparatus includes a second personal computer.
[0189] In the information processing apparatus according to an
embodiment, the plurality of partial images includes: (a) a first
partial image stored at a first location, and (b) a second partial
image stored at a second location.
[0190] In the information processing apparatus according to an
embodiment, the instructions, when executed by the processor, cause
the processor to: (a) receive the first partial image from the
first location, and (b) receive the second partial image from the
second location.
[0191] In the information processing apparatus according to an
embodiment, the instructions, when executed by the processor, cause
the processor to: (a) determine whether the first partial image is
stored by the memory device, (b) in response to a determination
that the first partial image is stored by the memory device, read
out the first partial image from the memory device, and (c) for the
first partial image, do not transmit a request to the at least one
location indicated by the location information.
[0192] In the information processing apparatus according to an
embodiment, the instructions, when executed by the processor, cause
the processor to determine the area specifying information based on
a display record.
[0193] In the information processing apparatus according to an
embodiment, the instructions, when executed by the processor, cause
the processor to determine the area specifying information based on
annotation information.
[0194] In another embodiment, an information processing apparatus
includes a processor, a display device, and
[0195] a memory device storing instructions. When executed by the
processor, the instructions cause the processor to: (a) receive
area specifying information and location information from a first
information processing apparatus, the area specifying information
specifying a display area in an image, the display area including a
plurality of partial images, the location information indicating at
least one location of the plurality of partial images, (b) transmit
a request for the plurality of partial images to the at least one
location indicated by the location information, (c) receive the
plurality of partial images from the at least one location, and (d)
display the received plurality of partial images.
[0196] In the information processing apparatus according to an
embodiment, the image includes data representative of a slice of a
biological tissue.
[0197] In the information processing apparatus according to an
embodiment, (a) the information processing apparatus includes a
personal computer, and (b) the first information processing
apparatus includes a server.
[0198] In the information processing apparatus according to an
embodiment, the plurality of partial images includes: (a) a first
partial image stored at a first location, and (b) a second partial
image stored at a second location.
[0199] In the information processing apparatus according to an
embodiment, the instructions, when executed by the processor, cause
the processor to: (a) receive the first partial image from the
first location, and (b) receive the second partial image from the
second location.
[0200] In the information processing apparatus according to an
embodiment, the instructions, when executed by the processor, cause
the processor to: (a) determine whether the first partial image is
stored by the memory device, (b) in response to a determination
that the first partial image is stored by the memory device, read
out the first partial image from the memory device, and (c) for the
first partial image, do not transmit a request to the at least one
location indicated by the location information.
[0201] In the information processing apparatus according to an
embodiment, the area specifying information is determined based on
a display record.
[0202] In the information processing apparatus according to an
embodiment, the area specifying information is determined based on
annotation information.
[0203] In another embodiment, a method of operating an information
processing apparatus includes (a) causing a processor to execute
instructions to receive, from a first information processing
apparatus, area specifying information and location information,
the area specifying information specifying a display area in an
image, the display area including a plurality of partial images,
the location information indicating at least one location of the
plurality of partial images, and (b) causing the processor to
execute instructions to transmit, to a second information
processing apparatus, the area specifying information and the
location information.
[0204] In the method of operating an information processing
apparatus according to an embodiment, the image includes data
representative of a slice of a biological tissue.
[0205] In the method of operating an information processing
apparatus according to an embodiment, (a) the information
processing apparatus includes a server, (b) the first information
processing apparatus includes a first personal computer, and (c)
the second information processing apparatus includes a second
personal computer.
[0206] In the method of operating an information processing
apparatus according to an embodiment, the plurality of partial
images includes: (a) a first partial image stored at a first
location, and (b) a second partial image stored at a second
location.
[0207] In the method of operating an information processing
apparatus according to an embodiment, the area specifying
information is determined based on a display record.
[0208] In the method of operating an information processing
apparatus according to an embodiment, the area specifying
information is determined based on annotation information.
[0209] In another embodiment, a system includes first, second, and
third information processing apparatus. The first information
processing apparatus is configured to, in response to an operation
input, determine area specifying information and location
information, the area specifying information specifying a display
area in an image, the display area including a plurality of partial
images, the location information indicating at least one location
of the plurality of partial images. The second information
processing apparatus is configured to receive the area specifying
information and the location information from the first information
processing apparatus. The third information processing apparatus is
configured to: (a) receive the area specifying information and the
location information from the second information processing
apparatus, (b) transmit a request for the plurality of partial
images to the at least one location indicated by the location
information, (c) receive the plurality of partial images from the
at least one location, and (d) display the received plurality of
partial images.
[0210] The present application contains subject matter related to
that disclosed in Japanese Priority Patent Application JP
2011-180438 filed in the Japan Patent Office on Aug. 22, 2011, the
entire content of which is hereby incorporated by reference.
[0211] It should be understood that various changes and
modifications to the presently preferred embodiments described
herein will be apparent to those skilled in the art. Such changes
and modifications can be made without departing from the spirit and
scope of the present subject matter and without diminishing its
intended advantages. It is therefore intended that such changes and
modifications be covered by the appended claims.
REFERENCE SIGNS LIST
[0212] 11 CPU [0213] 16 display [0214] 18 storage [0215] 19
communicating section [0216] 40 observation object [0217] 70 tile
[0218] 71 coordinates moving path [0219] 72 display area [0220] 73
annotation [0221] 100 PC [0222] 150 Internet [0223] 170 in-hospital
network [0224] 200 server [0225] 300 scanner
* * * * *