U.S. patent application number 13/191566 was filed with the patent office on 2011-12-01 for apparatus and method of viewing electronic documents.
This patent application is currently assigned to SONY COMPUTER ENTERTAINMENT EUROPE LIMITED. Invention is credited to Adriana Maria Eyzaguirre, Peter John Hodges, Jonathan Richard Milner-Moore.
Application Number | 20110296295 13/191566 |
Document ID | / |
Family ID | 39767474 |
Filed Date | 2011-12-01 |
United States Patent
Application |
20110296295 |
Kind Code |
A1 |
Hodges; Peter John ; et
al. |
December 1, 2011 |
APPARATUS AND METHOD OF VIEWING ELECTRONIC DOCUMENTS
Abstract
A method of generating an e-book document comprises representing
some or all of a page of an electronic document as a graphic
textures sequence derived from the document page. The sequence
comprises successively smaller graphic textures, arranging them to
form a first mip-map suitable for use by 3D graphics hardware, and
outputting an e-book document comprising the first mip-map
representing some or all of the document pages. A method of reading
an e-book document includes accessing at least a portion of the
mip-map comprising the sequence of graphic textures, constructing a
surface in a 3D virtual space comprising one or more polygons,
applying to the polygon surface a graphic texture derived from the
accessed part of the mip-map responsive to scaling of the polygon
surface with respect to an e-book reader display, and displaying
the textured polygon surface on the display, thereby displaying
some or all of the page.
Inventors: |
Hodges; Peter John; (London,
GB) ; Milner-Moore; Jonathan Richard; (London,
GB) ; Eyzaguirre; Adriana Maria; (London,
GB) |
Assignee: |
SONY COMPUTER ENTERTAINMENT EUROPE
LIMITED
London
GB
|
Family ID: |
39767474 |
Appl. No.: |
13/191566 |
Filed: |
July 27, 2011 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
13056800 |
May 4, 2011 |
|
|
|
PCT/GB2009/001878 |
Jul 30, 2009 |
|
|
|
13191566 |
|
|
|
|
Current U.S.
Class: |
715/234 ;
715/243 |
Current CPC
Class: |
G06T 15/04 20130101;
G06T 11/60 20130101 |
Class at
Publication: |
715/234 ;
715/243 |
International
Class: |
G06F 17/00 20060101
G06F017/00 |
Foreign Application Data
Date |
Code |
Application Number |
Aug 4, 2008 |
GB |
0814220.0 |
Claims
1. A method of generating an e-book document, comprising the steps
of: incorporating metadata including an indication of the placement
of features within a digital page of said e-book document;
incorporating metadata including an indication of positions at
which to place a viewpoint in said e-book document; and composing a
reading sequence of said positions.
2. A method according to claim 1, in which the metadata comprises
an extended mark-up language.
3. A method of displaying an e-book document, comprising the steps
of: accessing metadata of the e-book document that includes an
indication of the placement of features within a digital page of
said e-book document, an indication of positions at which to place
a viewpoint in the e-book document, and a reading sequence of said
positions; and presenting said e-book document responsive to said
reading sequence.
4. A method according to claim 3 in which said features comprise
panels of a comic book.
5. A method according to claim 3 in which said metadata comprises
an extended mark-up language.
6. A method according to claim 3 in which said metadata forms a
separate data stream to the e-book document
7. A method according to claim 3 in which said reading sequence
comprises one or more moves selected from the list consisting of:
i. a pan; ii. a jump; and ii. a zoom.
8. A method of displaying an e-book document, comprising the steps
of: accessing metadata including an indication of the placement of
features within a digital page of said e-book document; composing a
reading pattern of said e-book document according to rules
responsive to the structure of said features within said digital
page of said e-book document; and presenting said digital page of
said e-book document responsive to the reading pattern.
9. A method according to claim 8 in which said features comprise
panels of a comic book.
10. A method according to claim 8 in which said metadata comprises
an extended mark-up language.
11. A method according to claim 8 in which said reading sequence
comprises one or more moves selected from the list consisting of:
i. a pan; ii. a jump; and iii. a zoom.
12. An e-book reader for reading an e-book document, the e-book
reader comprising: at least one processor programmed to: access
metadata of said e-book document that includes an indication of the
placement of features within a digital page of said e-book
document, an indication of positions at which to place a viewpoint
in the e-book document, and a reading sequence of said positions;
and present said e-book document responsive to said reading
sequence.
13. An e-book reader for reading an e-book document, the e-book
reader comprising: at least one processor programmed to: access
metadata including an indication of the placement of features
within a digital page of said e-book document; compose a reading
pattern of said e-book document according to rules responsive to
the structure of said features within said digital page of said
e-book document; and present said digital page of said e-book
document responsive to the reading pattern.
14. A tangible computer-readable storage medium on which computer
readable instructions of a program are stored, the instructions,
when executed by a processor, cause the processor to perform a
method of method of displaying an e-book document, the method
comprising: accessing metadata of the e-book document that
includes: an indication of the placement of features within a
digital page of said e-book document, an indication of positions at
which to place a viewpoint in the e-book document, and a reading
sequence of said positions; and presenting said e-book document
responsive to said reading sequence.
15. A tangible computer-readable storage medium on which computer
readable instructions of a program are stored, the instructions,
when executed by a processor, cause the processor to perform a
method of method of displaying an e-book document, the method
comprising: accessing metadata including an indication of the
placement of features within a digital page of said e-book
document; composing a reading pattern of said e-book document
according to rules responsive to the structure of said features
within said digital page of said e-book document; and presenting
said digital page of said e-book document responsive to the reading
pattern.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application is a divisional of U.S. patent application
Ser. No. 13/056,800, filed Jul. 30, 2009, which is a national phase
entry under 35 U.S.C. .sctn.371 of International Application No.
PCT/GB2009/001878 filed Jul. 30, 2009, published in English, which
claims the benefit of and priority to GB Patent Application No.
0814220.0 filed Aug. 4, 2008, the entire disclosures of which are
hereby incorporated by reference herein.
BACKGROUND OF THE INVENTION
[0002] The present invention relates to an apparatus and method of
viewing electronic documents.
BRIEF SUMMARY OF THE INVENTION
[0003] So-called `e-books` are conventional devices for viewing
electronic documents, and may take the form of either dedicated
hardware (such as the Sony PRS-500) or software applications that
adapt conventional hardware to fulfil the equivalent role. It will
be understood that hereafter a reference to an `e-book` or `e-book
reader` encompasses both hardware and software versions.
[0004] Historically there has been little agreement between the
manufacturers of e-book solutions as to a format for disseminating
electronic documents. As a result many formats have been proposed,
many of which are based on html/xml (hypertext/extended mark-up
languages). Meanwhile the most popular format, the Adobe portable
document format (PDF), is based on the printer language Postscript.
A comprehensive list of past and present e-book document formats
can be found at
http://en.wikipedia.org/wiki/Comparison_of_e-book_formats.
[0005] Within these formats, in addition to symbolic
representations of text, some scanned representations of documents
are also available (so-called raster formats). The most popular
raster formats for scanned documents are DjVu (commonly referred to
as Deja Vu) and again Adobe PDF. Both use similar methods,
employing wavelet-based compression methods for colour or greyscale
pictures (e.g. JPEG 2000) whilst for black and white text a graphic
character dictionary is generated, either using a selected instance
of a character to replace all repeat instances in lossy
compression, or using a differential coding based on such a
selected instance for each repeat instance in lossless
compression.
[0006] However, when such raster formats are viewed on hand-held
e-books or on portable devices running e-book applications, such as
mobile phones, personal digital assistants (PDAs) or portable
gaming platforms such as the Sony Playstation Portable.RTM.,
schemes such as DjVu and PDF suffer from visible scaling artefacts
as they are viewed on a small and comparatively low resolution
screen. Conversely, to provide a clear high-resolution image at
full scale still requires a comparatively large amount of memory to
store and/or bandwidth to download to a portable device. In the
latter case of downloads, this can also prove expensive to the user
and/or service provider.
[0007] An attempt to address the memory and bandwidth issues in
web-based document readers can be seen in the embedded Google.RTM.
maps application (see http://maps.google.co.uk/). In Google maps,
only a visible section of a map at a first scale is downloaded and
viewed, thereby conserving bandwidth. If the user navigates beyond
the range of the current section of the map, a new section is
downloaded. If the user requires more detail, the embedded
application scales up the relevant part of current raster image to
the next scale (resulting in a blocky representation of the
relevant part of the map), and then downloads a section of a new
map (itself a new document, containing additional features)
corresponding to the scaled section of the old map. This process
can be seen, for example, by placing a mouse cursor over a portion
of such a map and using the mouse scroll wheel. The effect is one
of continuity whilst providing access to sections of successive
documents containing ever more detail.
[0008] However, Google maps does not address issues of visual
quality in the rendition of a single e-book document.
[0009] Thus it is desirable to formulate an e-book format that
mitigates or alleviates the above problems of visual quality in
conjunction with memory/bandwidth usage, particularly on small or
portable e-book readers, and to provide an e-book reader that takes
advantage of this format.
[0010] In a first aspect there is provided a method according to
claim 1.
[0011] In another aspect there is provided a method according to
claim 6.
[0012] In another aspect there is provided a computer program
according to claim 11.
[0013] In another aspect there is provided an e-book reader
according to claim 13.
[0014] Further respective aspects and features of the invention are
defined in the appended claims.
BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Embodiments of the present invention will now be described
by way of example with reference to the accompanying drawings, in
which:
[0016] FIG. 1 is an illustration of a Sony.RTM. Playstation
Portable.RTM. entertainment device;
[0017] FIG. 2 is a schematic diagram of components of a Sony.RTM.
Playstation Portable.RTM. entertainment device;
[0018] FIG. 3 is a schematic diagram of a mip-map;
[0019] FIG. 4 is a schematic diagram of a page of an electronic
document;
[0020] FIGS. 5A-5D are schematic diagrams of data access selections
with respect to a page of an electronic document;
[0021] FIGS. 6A and 6B are schematic diagrams of text language
regionalisation within an electronic document;
[0022] FIG. 7 is a flow diagram of a method of generating an e-book
document; and
[0023] FIG. 8 is a flow diagram of a method of reading an e-book
document.
DETAILED DESCRIPTION
[0024] An apparatus and method of viewing electronic documents are
disclosed. In the following description, a number of specific
details are presented in order to provide a thorough understanding
of the embodiments of the present invention. It will be apparent,
however, to a person skilled in the art that these specific details
need not be employed to practise the present invention. Conversely,
specific details known to the person skilled in the art are omitted
for the purposes of clarity where appropriate.
[0025] In an example summary embodiment of the present invention, a
page of a digital document is formatted as a so-called mip-map or
sequence of graphic textures, comprising images of the document as
graphic textures at two or more scales. In an e-book reader, a
texture of the mip-map at a first scale is applied to a plane of
one or more polygons by a 3D graphics processor of the reader
device. The x, y and z co-ordinates of the plane within a 3D space
can then be manipulated, either by the user or the reader itself,
to pan within the document at a first scale or to zoom within the
document by effectively moving it closer to or further from the
point of view provided by the screen of the device. In this latter
case, the e-book reader or application takes advantage of the
ability of 3D graphics processors to smoothly interpolate between
mip-map textures at different scales (e.g. by trilinear filtering)
in order to provide a high quality display of the document at any
scale between the first and last available textures in the mip-map.
In the case that the working memory and/or bandwidth of the e-book
reader or application is heavily constrained, the mip-map textures
can be only partially loaded or downloaded to a cache, for example
in response to a pre-set, heuristic or empirically determined
reading pattern.
[0026] In embodiments of the present invention, a suitable host
device for an e-book reader application is the Sony Playstation
Portable.RTM..
[0027] Referring to FIG. 1, in an embodiment of the present
invention a Sony.RTM. PlayStation Portable.RTM. (PSP) entertainment
device is a host device 100 for an e-book reader application. The
PSP body 104 comprises, inter alia, a left joypad 106 and a right
joypad 108. These are used to interface with software running on
the PSP. In addition, the PSP comprises an integral display 102 and
a speaker 103.
[0028] Referring now also to FIG. 2, a summary schematic diagram of
a PSP is provided. The PSP comprises a central processing unit
(CPU) 101, a graphics processing unit (GPU) 110 for polygon
rendering, texture scaling and the like, a media engine 131 and an
audio/video processor (AVC) 132 for image rendering, video and
audio playback and the like, and a direct memory access controller
(DMAC) 140, linked by a common bus 160. The DMAC 140 also links to
an external bus 170 through which inputs and outputs are
communicated, including with a wireless communication means (Tx/Rx)
120, a USB connector 125, a flash memory stick interface 135 that
can act as a storage means for the device, an optical drive (not
shown) and to the integral display 102.
[0029] Referring now to FIG. 3, in an embodiment of the present
invention, a page of a digital document is formatted as a plurality
of graphic textures (300A-G) in the form of a mip-map.
[0030] The original purpose of mip-maps was to address the problem
in video games, simulators, or virtual reality systems that as a
potentially large texture (for example 2048.times.2048 pixels)
moves away from the screen's point of view, to display ever smaller
versions of it requires the averaging or sampling of ever larger
portions of the texture for each screen pixel actually displayed,
with the eventual situation being that the entire texture is so far
from the screen in the z (depth) direction that it is averaged down
to a single pixel. If there are many such pixels in the displayed
image then the computational burden in rendering the image becomes
immense. One solution is to enforce a so-called `draw distance`,
beyond which such textures are not rendered; this limits the number
of textures that are required to render a virtual environment and
hence limits the computational burden. However, it also causes the
undesirable effect that objects apparently vanish from or pop into
existence in the virtual environment at the notional draw distance.
Mip-maps were posited as a solution by providing pre-scaled,
smaller versions (300B-G) of an original resolution principal
texture 300A, potentially down to a 1.times.1 pixel version 300G.
In this way textures could appear to scale from full size to a
single pixel by averaging within ever smaller versions of the
texture, thereby avoiding the computational overhead of always
operating on the full resolution texture.
[0031] In one scheme--bilinear filtering--the averaging for the
purposes of re-scaling the texture is done on one source texture of
the mip-map until it is reduced to the scale of the next texture,
but this can result in visible `handover` artefacts as the graphics
system swaps from an averaged higher resolution texture to a native
lower resolution texture, and also in averaging artefacts with
changes in scale as new pixels are incorporated into averages
previously based on comparatively few pixels of the source texture
(e.g. so-called `pixel fighting`). Thus in another
scheme--trilinear filtering--the texture is interpolated from
averages derived from the two mip-map textures bounding the current
texture scale, resulting in much smoother handovers and a reduction
in averaging artefacts. A further scheme, anisotropic filtering,
incorporates perspective within tri-linear filtering; however, for
a texture in a plane parallel to the viewing screen, anisotropic
filtering therefore essentially reduces back to tri-linear
filtering.
[0032] Other advantages of mip-mapping include that the textures
can be anti-aliased in advance, thereby further reducing
computational load in generating smooth images at different
scales.
[0033] In embodiments of the present invention, a page of a
electronic document (a `page` being understood to be a single
planar section of a document; typically equating to one paper page
but potentially two or more paper pages as in a centrefold or
gatefold configuration, or a portion of a paper page) is scanned or
rendered at the highest desired resolution, hereafter referred to
as a `digital page`. It will be appreciated that the source
electronic document may be a scan of a paper document, or a
computer originated raster or vector document (such as an adobe
.pdf or a Microsoft Word.RTM. document).
[0034] The digital page is converted to a principal texture or,
where the resolution exceeds the maximum permitted size of texture
on a particular host device, two or more principal textures. Hence
a principal texture can represent some or all of a digital
page.
[0035] Then, for the or each principal texture, a series of one or
more derivative textures is generated at half size (a 50% reduction
in each dimension). Thus for a 2048.times.2048 pixel principal
texture, the derivative textures could be 1024, 512, 256, 128, 64,
32, 16, 8, 4, 2 and 1 pixel square.
[0036] However, in the present embodiment there is no need to
continue the series down to a 1.times.1 pixel texture; the process
can be stopped at the point where the document represented in the
texture remains readable, or satisfies some other a visual quality
criterion (e.g. the point at which the whole digital page fits
within the screen of the e-book in one or both dimensions). As a
consequence, notably the resolution of the principal texture need
not be a power of two, though clearly it is preferable for the
texture size to be exactly divisible by two for at least the
duration of the intended texture sequence in the mip-map. Likewise
potentially it need not be square, but can reflect the dimensions
of the source material.
[0037] Thus, for example, a principal texture could have a
resolution of 1680.times.2376, which is exactly divisible by two
down to a final texture with a resolution of 210.times.297, which
replicates the relative dimensions of size A4 paper. It will be
appreciated that such resolutions are non-limiting examples only,
and resolutions may be selected based upon a variety of criteria
including the dimensions of the source material, visual quality
requirements, the memory capacity of graphics hardware and any
constraints imposed by graphics application interfaces.
[0038] The selected texture sequence is packaged as a conventional
mip-map suitable for processing by the 3D graphics hardware (e.g.
GPU 110) of the reader or host device, or for presentation to such
hardware by an interface such as Microsoft.RTM. Direct X.RTM. or
OpenGL.
[0039] Clearly, in the event that the hardware or the interface
expects a mip-map to comprise a full texture sequence down to
1.times.1 pixel then square textures with dimensions based upon a
power of two can be used and either the remaining texture sequence
can be provided, or dummy blank textures (amenable to high
compression ratios) can be used.
[0040] In any event, the digital page may then be rendered by the
e-book by constructing a planar surface of one or more polygons
conforming to the relative dimensions of the digital page, and
applying a texture derived from the or each mip-map to the polygon
or polygons as applicable in a conventional manner, to display the
digital page.
[0041] Then, rather than scaling a raster-format image up or down
as in conventional e-book readers, in an embodiment of the present
invention the scaling of the document is achieved by moving the
polygon surface in the z-axis closer to or further from the
notional viewpoint of the user as represented by the screen, and
allowing the dedicated 3D graphics hardware (e.g. GPU 110) of the
device to generate a texture at the appropriate scale for the
perceived distance of the polygon surface, based upon the textures
of the mipmap and the z-axis value, and preferably using trilinear
filtering.
[0042] This has the advantages of utilising dedicated graphics
hardware and memory (texture memory) to implement the scaling (thus
potentially much faster and/or more smoothly than would be possible
using the CPU 101 alone), reducing the ongoing computational load
of the CPU to typically just specifying the current z-axis value,
and generating smooth images at fractional scales by virtue of
access to two representations of the document on either side of the
currently displayed scale, as outlined with respect to trilinear
filtering above.
[0043] In an alternative embodiment, scaling can be achieved by
redefining the actual size of the polygon surface to which the or
each texture is applied whilst the surface remains at the same
notional distance from the display. In this case the mip-maps are
again used to interpolate the texture at the resulting scale.
However, whilst this method also gains most of the advantages
outlined above, recomputation of the polygon plane will in general
require relatively more computational resources from the CPU 101
when generating an updated draw list for the GPU 110 to render, in
comparison to updating a single parameter (the z-axis value) as in
the preceding embodiment.
[0044] In either case, the e-book format will therefore comprise
the one or more mip-maps for at least a first digital page, an
indication of the smallest usable or displayed texture in a mip-map
sequence, and an indicator of the digital page dimensions, either
explicitly as relative or absolute measurements, by indication of a
format (such as A4), or implicitly from the dimensions of the
principal texture (or sum thereof if more than one). Where two or
more mip-maps are used to form a digital page, their respective
placement on the polygon plane should also be indicated, either
explicitly or by virtue of their order in a formatted file. To
address constraints on memory or bandwidth in small and/or portable
devices, several options are available.
[0045] Firstly, in addition to conventional texture compression,
the mip-maps can be differentially encoded. In an embodiment of the
present invention, the smallest used texture in the proposed
texture sequence (for example 300D) is stored and optionally
compressed in a conventional fashion. The next largest texture in
the sequence (300C) is then represented by obtaining the difference
in values between it and a double-scale version of 300D. These
difference values will represent the loss in fidelity caused by the
scaling down from 300C to 300D and in general will have a much
smaller variance than the values of the texture 300C itself,
enabling significantly greater compression. 300B is then similarly
differentially encoded from a reconstruction of 300C, and likewise
300A is differentially encoded from a reconstruction of 300B.
[0046] These textures (or those parts thereof required for the
purpose of display or trilinear filtering) can then be
reconstructed, either by the GPU 110 if capable, or by the CPU.
[0047] In this way the mip-map is more efficiently compressed for
the purposes of storage or download.
[0048] Secondly, alternatively or in addition only those textures
or parts of textures in a mip-map currently required or determined
to be required next need be accessed (e.g. downloaded from a
network or loaded from local media such as a hard-drive, memory
stick or optical disk), to be cached and optionally
decompressed/reconstructed.
[0049] In a first instance, only those portions of a texture or
textures required to generate the current screen display need be
accessed. However, this is likely to cause a lag when the user pans
or zooms within the document as neighbouring portions of a texture
or portions of further textures are accessed to generate the
updated screen display.
[0050] Therefore in an embodiment of the present invention,
neighbouring portions of a texture or portions of further textures
are cached in advance in response to a predetermined reading
pattern.
[0051] Referring now also to FIG. 4, as an example a digital page
of a comic book 400 comprises panels 401 to 404. At a particular
level of zoom, the region or `tile` 410 highlighted by dotted lines
is visible on the screen of the e-book.
[0052] In a first instance, a predetermined reading pattern may
take the form of `next` and `back` buttons that move the current
tile to a new pre-set position on the digital page.
[0053] Such a sequence of positions (optionally including
sequentially different scaling information) may be arbitrary for
each page and embedded as metadata within the e-book format or as a
separate data stream. A plurality of such streams or sets of
metadata may be made available for different e-book devices
responsive to their screen dimensions (and hence tile dimensions)
and native resolution, so enabling an enjoyable presentation of the
electronic document to be tailored to each device.
[0054] Optionally, several sequences may be provided and selected
between. For example, a `fast` sequence may concentrate on views
containing dialogue, whilst a `slow` sequence may provide
additional views of background and intervening artwork of the
comic. Other sequences may include or exclude pages, panels or text
as applicable, for example to exclude scenes of a violent or sexual
nature if the user was under a certain age.
[0055] Moreover, some sequences may only be available at a premium,
and allow access to additional areas of an electronic document.
[0056] It will be appreciated that during the composition of such a
sequence by the author, publisher or distributor (generally on a PC
or other device with a high resolution screen), typically some or
all of the page of the document surrounding the view intended to be
seen on the respective portable e-book reader would be visible, in
order to provide visual context for the choice of composition.
[0057] Alternatively, instead of choreographing the user's reading
experience in advance, such a sequence of positions and scales may
be evaluated at the time of reading, responsive to the relative
dimensions of the screen with respect to the relative dimensions of
the digital page or sections thereof.
[0058] For example, referring now to FIG. 4, a digital page of a
comic book 400 comprises panels 401 to 404. At a particular level
of zoom, the region or `tile` 410 highlighted by dotted lines is
visible on the screen of the e-book.
[0059] Referring now to FIGS. 5A-D, an example predetermined
reading pattern stipulates that the reader would move from the left
half of panel 401 to the right half, by default to the panel's
right edge if it is slightly more than twice as wide as a tile.
After that it will jump to panel 402 (FIG. 5A).
[0060] Therefore the portions of the texture or textures necessary
to render the right half of panel 401 and the whole of panel 402 at
the current scale (denoted by the areas 420) are accessed, and
cached for potential use.
[0061] In FIG. 5B, upon reaching panel 404, this panel is
significantly more than two times longer than one of the display
dimensions. In this case, rather than jump to the other end of the
panel (as in panel 401) and miss out the central third, the
predetermined reading pattern can next either jump successively in
equal steps through the panel, or pan down it. The relevant
textures are cached accordingly.
[0062] Thus in a first instance a predetermined reading pattern can
be pre-set according to rules responsive to the structure found on
the digital page and the current scale of tiling; for plain text
this will likely devolve to left-right and top to bottom jumps or
panning, whilst for documents with embedded images or borders these
can be used to determine scales and framing, either by automated
analysis of regions in the digital page or by indicators of where
such images or borders are within the digital page (e.g.
co-ordinates of diagonally opposite corners). The above rules are
exemplary and non-limiting, and other rules will be apparent to the
skilled person.
[0063] In addition to jumping/panning within a given scale, in FIG.
5C, the user may select to zoom out. Consequently a portion of the
texture at the next lowest scale required to zoom from the current
scale to form a tile at a scale anywhere down to that next lowest
scale may be cached.
[0064] Alternatively a user may choose to zoom in. In FIG. 5D a
portion of the texture at the next highest scale required to zoom
from the current scale to form a tile at a scale anywhere up to
that next highest scale may be cached.
[0065] At the new tile scale, the rules for predetermined reading
can then be reapplied.
[0066] In a second instance, a predetermined reading pattern refers
solely to the selection and caching of textures and does not
constrain the reader in how they view the document. In this case
rather than a `next` button, controls such as joy-pad buttons 106
or 108 may be used to navigate freely over the digital page.
[0067] In this case, caching may be based upon heuristics
responsive to factors such as:
[0068] How far to the right of the digital page is the currently
viewed tile;
[0069] indicative of the likelihood of panning or jumping further
right vs. wishing to jump down and to the left,
[0070] How close to the bottom of the digital page is the currently
viewed tile;
[0071] similarly indicative of the likelihood of jumping down, once
close to the bottom of the page, and/or the likelihood of turning
the page,
[0072] An estimate of the number of pixels used to represent
characters of text;
[0073] determined by pattern matching or metadata regarding the
relative size of text in the digital page, and indicative of the
likelihood of zooming in or out, depending on the text size.
[0074] Other heuristics will be apparent to the skilled person,
such as whether the current tile does not closely match at least
one dimension of a (possibly partially) displayed image or comic
panel (indicative that the reader may zoom in or out to achieve a
better match). It will be appreciated that such heuristics can be
adapted to the reading style of each language, for example
reversing the presumption of reading right to left.
[0075] After evaluating such criteria, textures may be cached in
order of likelihood of use.
[0076] Alternatively or in addition in this second instance caching
may be based upon empirical measurements. For example, 50 people
may be invited to read the digital page, and their reading
strategies are recorded. Caching is then performed in the order of
the most probable next action from a given region/scale of the
digital page.
[0077] Alternatively or in addition, the user's individual reading
style could be evaluated and stored in a similar fashion (for
example, the reader may be a browser who tends to skip down a page
at a relatively low image scale), and this can be incorporated into
the heuristics or empirical data disclosed above. Moreover, such
user information could be stored and applied to different
documents, either universally or differentiated according to
document type (e.g. comic, book, newspaper etc).
[0078] It will be appreciated that textures need not be accessed on
a tile-by-tile basis. Where memory and/or bandwidth permits, the
whole mip-map or large portions of it corresponding at least on
some scales to multiple tiles may be stored in memory, mitigating
the need for caching strategies such as those disclosed above.
Conversely for panning within a digital page at a large scale,
subsections of textures significantly smaller than a tile may be
supplied over the course of the panning process.
[0079] Referring now to FIGS. 6A and B, for some digital documents,
particularly those that combine images and text, language
regionalisation may be achieved by the use of multiple mip-maps. In
this case, the images are represented by one mip-map whilst the
text in a respective language is represented by a respective
further mip-map. The text may be anti-aliased in advance with
respect to the background image, or may be in speech bubbles or
boxes that may be similarly anti-aliased. Pixels of the textures
not associated with text (or speech bubbles or boxes) will be
uniform in value (allowing for efficient compression) and
transparent (e.g. with an alpha value of 0). Because such a mip-map
is mostly blank and has a comparatively simple colour palette,
compression is highly efficient, potentially allowing many more
language versions of a digital document to be downloaded or stored
in a given space than if the full image with text was stored for
each language. Alternatively or in addition, different versions of
the text in one language may be made available, for example being
modified according to an age rating and selected according to the
user's registered age.
[0080] To view the image and text together, an embodiment of the
present invention takes advantage of the fact that most 3D hardware
systems allow the application of multiple textures to a polygon,
using alpha values to determine the extent to which superposed
texture layers mask lower texture layers.
[0081] In this case, the image layer and then the respective text
layer are applied to the polygon plane, with the alpha values of
the text layer ensuring that the image layer is visible around the
text.
[0082] To change languages, only the respective mip-map for the
respective language then need be changed.
[0083] In the event that the 3D hardware system does not allow the
application of multiple textures, then the text may effectively be
overlaid upon the image by defining a second polygon plane with a
z-axis value fractionally smaller than that of the first polygon
frame, and applying textures of the respective text mip-map to it.
The 3D hardware will contain a so-called depth- or z-buffer that
determines which pixels of the two polygon planes are visible. In
this case it will be those of the text in preference to the image,
except where the texture is transparent. In such a circumstance a
z-buffer with a large bit depth (e.g. 24 or 32) would allow the
textures to be finely distinguished in the z-direction (thereby
allowing for the scale of both textures to be substantially the
same, and hence allowing for consistent pixel alignment between
textures).
[0084] It will be appreciated that such secondary texture layers
need not be restricted to text. For example alternative artwork may
be provided for certain elements of a page, for example to obscure
or sanitise scenes unsuitable for a young reader, or to provide
additional information such as artist's notes and/or rough sketches
to be overlaid on the main document. In this case multiple texture
layers can be used to allow both the background image, the text in
the appropriate language and the additional layer or layers to be
presented in a desired order. Again if the 3D hardware cannot
support layers (or enough layers) then multiple polygon surfaces
can be used as disclosed previously.
[0085] Likewise, it will be appreciated that the data source for
the different mip-maps need not all be the same; for example a
comic may be issued on an optical disc, but supplementary content
such as additional languages, author's notes and sketches etc., may
be downloaded.
[0086] Finally, in addition to static textures, in an embodiment of
the present invention an e-book document may comprise resources to
enable animated content.
[0087] In a first instance, this may comprise information regarding
the animation of turning pages (if used), including the optional
specification of stencil buffer values and stencil modifiers to
define shadowing effects during such animations.
[0088] In addition to animating the e-book document itself,
animations of the content of the document may also be
considered.
[0089] The e-book document may include further mip-map textures and
also specific, custom specifications for polygons, enabling the
introduction and animation of additional graphic elements. These
may simply be objects that appear to move in front of the page, but
potentially can interact more closely as follows:
[0090] The e-book document can optionally include a pre-determined
array of z-buffer values (typically for the principal texture) that
allows a determination of the relative depth position of each pixel
in the rendered texture. By transferring this prepared z-buffer
information to the 3D graphics hardware (typically with an offset
corresponding to the current distance of the page from the
viewpoint, as disclosed previously), the flat texture applied to
the polygon surface can be treated as having varying depth. For
example, a panel of a comic could illustrate an underwater scene
with a sunken boat to the left, seaweed spanning the middle of the
panel from top to bottom, and a cave to the right. In the prepared
z-buffer data the boat has a positive offset (making it more
distant), the seaweed has a negative z-buffer offset (making it
closer), whilst the rear of the cave mouth is positive and the
front is negative.
[0091] The e-book document also contains a polygon model for a fish
(either in 3D or as another flat surface) and a mip-map (or a
texture if this animation only occurs at a specific scale)
depicting the fish. The fish is given a zero z-buffer offset in
this example and, moving from left to right, may then appear to
swim in front of the boat, behind the seaweed and into the
cave.
[0092] Other layers may have additional pre-determined z-buffer
information that can be overlaid upon it. For example, a
regionalised text may have a strong negative z-buffer offset so
that it always appears on top of such animations, whilst the
remainder of the z-buffer is unchanged and conforms to the prepared
z-buffer information associated with the image. Referring now to
FIG. 7, a method of generating an e-book document comprises:
[0093] in a first step s710, representing some or all of a page of
an electronic document as a sequence of two or more graphic
textures from the page of the electronic document, wherein the
sequence comprises successively smaller textures;
[0094] in a second step s720, arranging the textures so generated
to form a mip-map suitable for use by 3D graphics hardware; and
[0095] in a third step s730, outputting an e-book document file
comprising said mip-map representing some or all of said page of
said document.
[0096] It will be apparent to a person skilled in the art that
variations in the above method corresponding to operation of the
various embodiments of the apparatus described above are considered
within the scope of the present invention, including but not
limited to:
[0097] Representing a page of a digital document using two or more
mip-maps;
[0098] Representing a page of a digital document using two layers
of mip-maps, for example an image layer and a text layer;
[0099] Incorporating within the e-book document file two or more
text mip-maps for different respective languages;
[0100] Incorporating within the e-book document file additional
resources for animations;
[0101] Incorporating within the e-book document file one or more
pre-prepared z-buffer data sets
[0102] Generating a partial mip-map sequence limited by readability
or aesthetic criteria, and optionally substituting dummy/blank
textures for the remainder of the mip-map sequence if required;
[0103] Generating non-square mip-maps, for example based upon the
dimensions of the digital document;
[0104] Incorporating metatdata including one or more selected from
the list consisting of:
[0105] an indication of the smallest usable or displayed texture in
a mip-map sequence,
[0106] an indicator of the digital page dimensions,
[0107] an indicator of relative texture placements,
[0108] an indication of text size relative to page size,
[0109] an indication of the placement of features within the
digital page (for example corner positions), and
[0110] an indication of one or more positions at which to place the
viewpoint of the e-book within a page and/or and indication of the
scale; and
[0111] Composing a reading sequence for a particular screen size
with reference to the larger document.
[0112] Referring now to FIG. 8, a method of reading an e-book
document, in which some or all of a page of the document is
represented by a mip-map, comprises:
[0113] in a first step s810, accessing at least a portion of a
mip-map comprising a sequence of graphic textures, wherein the
sequence comprises successively smaller textures;
[0114] in a second step s820, constructing a surface in a 3D
virtual space comprising one or more polygons;
[0115] in a third step s830, applying to the polygon surface a
texture derived from the mip-map that is responsive to the scale of
the plane with respect to a graphical display of the e-book;
and
[0116] in a fourth step 840, displaying the textured polygon
surface on the graphical display of the e-book, thereby displaying
some or all of the page of the document.
[0117] It will be apparent to a person skilled in the art that
variations in the above method corresponding to operation of the
various embodiments of the apparatus described above are considered
within the scope of the present invention, including but not
limited to:
[0118] Accessing at least a portion of two or more mip-maps,
corresponding to different parts of a digital page and/or to
different layers (such as image and text);
[0119] Applying at least a portion of two or more mip-maps to the
polygon surface, corresponding to different parts of a digital page
and/or to different layers (such as image and text);
[0120] Zooming in and out of the document by adjusting the depth
value of the polygon surface;
[0121] Generating the currently displayed texture(s) at the current
scale based upon mip-map textures at the next available scales
above and below the current scale; for example by using tri-linear
filtering;
[0122] Restricting the zoom function of the e-book reader to the
minimum and maximum allowed by the mip-map textures, including
where the smallest allowed texture is not the smallest available or
possible texture;
[0123] Constructing the polygon surface in response to the
dimensions of the primary texture of the or each mip-map or some
other metadata indicative of the dimensions of the digital
page;
[0124] Constructing additional resources to be animated in
conjunction with the displayed document;
[0125] Generating a set of z-buffer values based upon a prepared
set of values associated with the displayed document;
[0126] Accessing only those textures required to render the current
screen image at the current scale; and
[0127] Accessing textures required to render different views
(portions and/or scales) of a digital page for future viewing,
based upon one or more selected from the list consisting of: [0128]
a pre-set viewing/reading sequence, [0129] heuristics to predict
the user's viewing sequence, and [0130] empirical measurements of
probable user viewing sequence.
[0131] It will be appreciated that an e-book document generated
according to the methods described herein may be stored on a data
carrier such as a floppy disk, optical disk, hard disk, PROM, RAM,
flash memory or any combination of these or other storage media, or
transmitted via data signals on a network such as an Ethernet, a
wireless network, the Internet, or any combination of these of
other networks.
[0132] Finally, it will be appreciated that the methods disclosed
herein may be carried out on conventional hardware suitably adapted
as applicable by software instruction or by the inclusion or
substitution of dedicated hardware.
[0133] Thus the required adaptation to existing parts of a
conventional equivalent device may be implemented in the form of a
computer program product or other object of manufacture comprising
processor implementable instructions stored on a data carrier such
as a floppy disk, optical disk, hard disk, PROM, RAM, flash memory
or any combination of these or other storage media, or transmitted
via data signals on a network such as an Ethernet, a wireless
network, the Internet, or any combination of these of other
networks, or realised in hardware as an ASIC (application specific
integrated circuit) or an FPGA (field programmable gate array) or
other configurable circuit suitable to use in adapting the
conventional equivalent device.
* * * * *
References