U.S. patent application number 13/182773 was filed with the patent office on 2012-08-30 for electronic book navigation systems and methods.
This patent application is currently assigned to Google Inc.. Invention is credited to Scott Dougall, Nathan Moody, James Patterson.
Application Number | 20120221968 13/182773 |
Document ID | / |
Family ID | 46718701 |
Filed Date | 2012-08-30 |
United States Patent
Application |
20120221968 |
Kind Code |
A1 |
Patterson; James ; et
al. |
August 30, 2012 |
Electronic Book Navigation Systems and Methods
Abstract
An electronic book system provides interfaces particularly
suited to students' use of textbooks. A finger press on a touch
screen produces a contextual menu with user choices that relate to
where the finger was pressed or what the user was recently doing
with the book. A student provisionally navigates through a book by
a specific gesture which, when it stops, returns the user to the
previous position in the book. Annotations are displayed and hidden
using specific gestures and through selective movement of the
reader as sensed by its accelerometer.
Inventors: |
Patterson; James; (San
Francisco, CA) ; Moody; Nathan; (San Francisco,
CA) ; Dougall; Scott; (Burbank, CA) |
Assignee: |
Google Inc.
Mountain View
CA
|
Family ID: |
46718701 |
Appl. No.: |
13/182773 |
Filed: |
July 14, 2011 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
13171130 |
Jun 28, 2011 |
|
|
|
13182773 |
|
|
|
|
61446239 |
Feb 24, 2011 |
|
|
|
Current U.S.
Class: |
715/776 |
Current CPC
Class: |
G06F 3/04883 20130101;
G06F 40/134 20200101; G06F 3/0483 20130101; G06F 3/04842 20130101;
G06F 40/169 20200101; G06T 3/60 20130101; G06Q 30/0621 20130101;
G06F 3/0482 20130101 |
Class at
Publication: |
715/776 |
International
Class: |
G06F 3/048 20060101
G06F003/048 |
Claims
1. An electronic book reader, comprising a navigation subsystem
configured to provisionally change display of an electronic book
from a first portion to a second portion responsive to a first user
input, from the second portion back to the first portion responsive
to a second user input, and to make the change non-provisional
responsive to a third user input.
2. The electronic book reader of claim 1, wherein the first user
input includes the user touching a screen of the reader at a first
set of contact points, the second user input includes the user
changing to a second set of contact points, and the third user
input includes the user changing to a third set of contact
points.
3. The electronic book reader of claim 2, wherein the second set of
contact points is a subset of the first set of contact points.
4. The electronic book reader of claim 2, wherein the third set of
contact points is a subset of the first set of contact points.
5. The electronic book reader of claim 1, wherein the first user
input includes a first user gesture, the second user input includes
a second user gesture, and the third user input includes a third
user gesture.
6. The electronic book reader of claim 5, wherein the second user
gesture corresponds to a release from the first user gesture.
7. The electronic book reader of claim 1, wherein the first user
input includes touching a screen of the electronic book reader with
at least a first point of contact and a second point of contact,
the second user input includes breaking the first point of contact,
and the third user input includes breaking the second point of
contact.
8. A computer implemented method of using an electronic book,
comprising: provisionally changing a display of an electronic book
from a first portion to a second portion responsive to a first user
input; returning the display to the first portion responsive to a
second user input; and making the changing of the display
non-provisional responsive to a third user input.
9. The method of claim 8, wherein the first user input includes the
user touching a screen of an electronic book reader at a first set
of contact points, the second user input includes the user changing
to a second set of contact points, and the third user input
includes the user changing to a third set of contact points.
10. The method of claim 9, wherein the second set of contact points
is a subset of the first set of contact points.
11. The method of claim 9, wherein the third set of contact points
is a subset of the first set of contact points.
12. The method of claim 8, wherein the first user input includes a
first user gesture, the second user input includes a second user
gesture, and the third user input includes a third user
gesture.
13. The method of claim 12, wherein the second user gesture
corresponds to a release from the first user gesture.
14. The method of claim 8, wherein the first user input includes
touching a screen of the electronic book reader with at least a
first point of contact and a second point of contact, the second
user input includes breaking the first point of contact, and the
third user input includes breaking the second point of contact.
15. A computer program product that comprises a non-transient
computer readable storage medium containing executable computer
program instructions for using an electronic book, comprising:
instructions to provisionally change a display of an electronic
book from a first portion to a second portion responsive to a first
user input; instructions to return the display to the first portion
responsive to a second user input; and instructions to make the
change of the display non-provisional responsive to a third user
input.
16. The computer program product of claim 15, wherein the first
user input includes the user touching a screen of an electronic
book reader at a first set of contact points, the second user input
includes the user changing to a second set of contact points, and
the third user input includes the user changing to a third set of
contact points.
17. The computer program product of claim 16, wherein the second
set of contact points is a subset of the first set of contact
points.
18. The computer program product of claim 16, wherein the third set
of contact points is a subset of the first set of contact
points.
19. The computer program product of claim 15, wherein the first
user input includes a first user gesture, the second user input
includes a second user gesture, and the third user input includes a
third user gesture.
20. The computer program product of claim 19, wherein the second
user gesture corresponds to a release from the first user
gesture.
21. The computer program product of claim 15, wherein the first
user input includes touching a screen of the electronic book reader
with at least a first point of contact and a second point of
contact, the second user input includes breaking the first point of
contact, and the third user input includes breaking the second
point of contact.
22. An electronic book reader, comprising a navigation subsystem
configured to provide user controls to create, manage and use
bookmarks, said user controls comprising: a user control to add a
bookmark on a currently viewable page; a first graphical
representation, representing an entire electronic book, including
the position of all existing bookmarks relative to the beginning
and the end of the electronic book, as well as the currently
viewable page; a control to navigate to a second page indicated by
an existing bookmark; and a second graphical indication, indicating
whether the currently viewable page is bookmarked.
23. The electronic book reader of claim 22, wherein the first
graphical representation also displays positions of chapter breaks
within the electronic book.
24. A computer implemented method of creating, managing and using
bookmarks in an electronic book, comprising: providing a user
control to add a bookmark on a currently viewable page; providing a
first graphical representation, representing an entire electronic
book, including the position of all existing bookmarks relative to
the beginning and the end of the electronic book, as well as the
currently viewable page; providing a user control to navigate to a
second page indicated by an existing bookmark; and providing a
second graphical indication, indicating whether the currently
viewable page is bookmarked.
25. The method of claim 24, wherein the first graphical
representation also displays positions of chapter breaks within the
electronic book.
26. A computer program product that comprises a non-transient
computer readable storage medium containing computer executable
instructions for creating, managing and using bookmarks in an
electronic book, comprising: instructions to provide a user control
to add a bookmark on a currently viewable page; instructions to
provide a graphical representation of an entire electronic book,
including the position of all existing bookmarks relative to the
beginning and the end of the electronic book, as well as the
currently viewable page; instructions to provide a user control to
navigate to a second page indicated by an existing bookmark; and
instructions to provide a second graphical indication, indicating
whether the currently viewable page is bookmarked.
27. The computer program product of claim 26, wherein the first
graphical representation also displays positions of chapter breaks
within the electronic book.
Description
CROSS REFERENCE TO RELATED APPLICATION
[0001] This application is a divisional and claims the benefit of
U.S. patent application Ser. No. 13/171,130 filed Jun. 28, 2011,
entitled "Electronic Book Interface Systems and Methods" which
claims the benefit of U.S. Patent Application Ser. No. 61/446,239,
filed Feb. 24, 2011, entitled "Electronic Textbook Systems and
Methods" both of these are incorporated by reference in their
entirety as if fully set forth herein.
BACKGROUND
[0002] 1. Technical Field
[0003] The invention generally relates to the field of electronic
books and, more particularly, to systems and methods for accessing
and otherwise working with book information via electronic
devices.
[0004] 2. Background Information
[0005] Even as widespread use of the Web reaches its twentieth
anniversary, there has been little change in how people make use of
textbooks. Students still fill their backpacks with as many of the
five-pound books as will fit, and the impact of such paper-based
learning is felt not only in students' backs, but in the carbon
footprint of all of the infrastructure required to supply, use and
dispose of such materials. A change of just a few pages in a
textbook may make it obsolete and call for a new version to be
printed; students carry not just this week's chapters with them
everywhere, but last month's and next month's chapters as well.
[0006] Although some attempts have been made to transform study
material from Gutenberg's era to the digital era, some of the
advantages of using paper books for study purposes have not been
replicated. Students from time immemorial have used their texts in
different ways. Some highlight portions of particular interest;
others place notes in the margins to keep track of clarifications
of difficult concepts. Some used textbooks are more useful than new
ones because they naturally fall open to the most important pages
after repeated use, or because particularly important pages or
sections are more dog-eared than others. Electronic reading devices
have not to date provided interfaces to implement some of these
subtle yet important features that help students learn from their
texts most efficiently.
[0007] It would be advantageous to provide improved interface
mechanisms for students to obtain, read, study from, and otherwise
use textbook content with some of the tablet, laptop and other
electronic devices that are now entering widespread use.
SUMMARY OF THE INVENTION
[0008] An electronic book reader including input recognition,
annotation, and collaboration subsystems. Improved interaction
methods provide more intuitive use of electronic books for
studying. The provided annotation functionality allows a reader
(e.g. a student) to make notes as is common with conventional paper
books. The collaboration subsystem provides functionality to share
these notes with others, enabling group learning on a variety of
scales, from small study groups to collaborations with worldwide
scope.
[0009] In one aspect, the electronic book reader is configured to
provide tabs or other interface controls through which a user can
access a syllabus for a particular course, corresponding textbooks,
the student's own electronic notebook for the course, and lecture
material (whether actual audio/video of the lecture, a slide deck
used with the lecture, or related materials from the lecture).
[0010] In one aspect, the reader is configured to facilitate
navigation through a textbook by providing various user options for
provisionally or tentatively moving through the text, for instance
to temporarily move to a glossary section before returning to the
main text, or temporarily moving from a question page to a portion
of the main text relating to the question. By using suitable
gestures, the student navigates between these options in a
provisional fashion that allows easy return to the main section of
interest. In a related aspect, a bookmarking system facilitates
easy access to portions the student identifies as important.
[0011] In a further aspect, the reader is configured to allow a
student to attach annotations to an electronic textbook, in much
the same way as a student might write notes in a conventional paper
textbook. These notes can take on a wider range of forms than is
possible conventionally. For example, a student can attach audio
and video as well as more traditional textual annotations. In a
related aspect, the reader is configured to provide tools for
converting student annotations into computer searchable and
manipulatable form.
[0012] In yet another aspect, the reader is configured to
communicate with an accelerometer subsystem on the user's computer
to allow the user to "pour" annotations off of, or onto, the user's
view of the textbook to either remove clutter or provide
annotations as the user may require at any particular time. The
reader is configured to permit students to show all annotations, to
show only certain annotations, to marginalize annotations, or to
hide all annotations as preferred at any particular time.
[0013] In still another aspect, the reader is configured to allow
students to use gestures to select portions of a textbook to copy
over to the student's electronic notebook, for instance where such
copying might provide a more efficient way of connecting concepts
than merely annotating the textbook with notes. In a specific
aspect, a user interface allows the student to include more or less
information in such a guide, based on the student's needs and
available study time.
[0014] The reader is further configured to facilitate collaboration
not only with other students, but with a professor or other
teacher, or a teacher's assistant assigned to help students with
the class. In one aspect, the reader is configured to recognize the
student's hand gesture in the form of a question mark on a textbook
page to open a question to a moderator (e.g., teaching assistant).
The student can then type in a question, and the moderator will
know which portion of the textbook relates to the question based on
the reader transmitting that information along with the question to
the moderator. The reader provides a number of other predefined
gestures and is further configured to allow users to define their
own gestures (e.g., scribbling in the initials of a friend opens a
chat with that friend, again keyed to the currently displayed
portion of the textbook).
[0015] In another aspect, the reader is configured to assist a
student in creating a personalized study guide. The presence of
annotations made by the student and/or the prevalence of
annotations made by other users informs which portions of an
electronic book are included. The reader provides controls to allow
the student to tailor the precise criteria used in generating the
study guide to help meet their specific needs and requirements.
[0016] The features and advantages described in the specification
are not all inclusive and, in particular, many additional features
and advantages will be apparent to one of ordinary skill in the art
in view of the drawings, specification, and claims. Moreover, it
should be noted that the language used in the specification has
been principally selected for readability and instructional
purposes, and may not have been selected to delineate or
circumscribe the disclosed subject matter.
BRIEF DESCRIPTION OF THE DRAWINGS
[0017] FIG. 1 is a high-level diagram illustrating a networked
environment that includes an electronic book reader.
[0018] FIG. 2 illustrates a logical view of a reader module used as
part of an electronic book reader.
[0019] FIG. 3 illustrates a logical view of a system database that
stores data related to the content hosting system.
[0020] FIG. 4 illustrates a user computer configured as an
electronic book reader.
[0021] FIG. 5 illustrates a user computer configured as an
electronic book reader, including user highlighting and
annotations.
[0022] FIG. 6 illustrates a user computer configured as an
electronic book reader, including a notebook interface.
[0023] FIG. 7 illustrates a user computer configured as an
electronic book reader, including a contextual menu.
[0024] FIG. 8 illustrates operation of the contextual menu of FIG.
7.
[0025] The figures depict various embodiments of the present
invention for purposes of illustration only. One skilled in the art
will readily recognize from the following discussion that
alternative embodiments of the structures and methods illustrated
herein may be employed without departing from the principles of the
invention described herein.
DETAILED DESCRIPTION
[0026] FIG. 1 is a high-level diagram illustrating a networked
environment 100 that includes a book content hosting system 110.
The embodiments discussed herein are particularly suited for
textbooks, but one skilled in the art will recognize that many of
the features discussed herein are applicable to various other types
of books as well. The content hosting system 110 makes available
for purchase, licensing, rental or subscription textbooks that can
be viewed on user and content provider computers 180 (depicted in
FIG. 1, for exemplary purposes only, as individual computers 180A
and 180B) using a reader module 181 or browser 182. The content
hosting system 110 and computers 180 are connected by a network 170
such as a local area network or the Internet.
[0027] The network 170 is typically the Internet, but can be any
network, including but not limited to any combination of a LAN, a
MAN, a WAN, a mobile, a wired or wireless network, a private
network, or a virtual private network. The content hosting system
110 is connected to the network 170 through a network interface
160.
[0028] As discussed above, only a single user computer 180A is
shown, but in practice there are many (e.g., millions of) user
computers 180A that can communicate with and use the content
hosting system 110. Similarly, only a single content provider
computer 180B is shown, but in practice there are many (e.g.,
thousands or even millions of) content providers 180B that can
provide textbooks and related materials for content hosting system
110. In some embodiments, reader module 181 and browser 182 include
a content player (e.g., FLASH.TM. from Adobe Systems, Inc.), or any
other player adapted for the content file formats used by the
content hosting system 110.
[0029] User computer 180A with reader module 181 is used by users
to purchase or otherwise obtain, and access, materials provided by
the content hosting system 110. Content provider computer 180B is
used by content providers to create and provide material for the
content hosting system 110. A given computer can be both a client
computer 180A and content provider computer 180B, depending on its
usage. The hosting service 110 may differentiate between content
providers and users in this instance based on which front end
server is used to connect to the content hosting system 110, user
logon information, or other factors.
[0030] The content hosting system 110 comprises a user front end
server 140 and a content provider front end server 150 each of
which can be implemented as one or more server class computers. The
content provider front end server 150 is connected through the
network 170 to content provider computer 180B. The content provider
front end server 150 provides an interface for content providers to
create and manage materials they would like to make available to
users. The user front end server 140 is connected through the
network 170 to client computer 180A. The user front end server 140
provides an interface for users to access material created by
content providers.
[0031] The content hosting system 110 is implemented by a network
of server class computers that can include one or more
high-performance CPUs and 1 G or more of main memory, as well as
500 GB to 2 Tb of storage. An operating system such as LINUX is
typically used. The operations of the content hosting system 110,
front end 140 and back end 150 servers as described herein can be
controlled through either hardware (e.g., dedicated computing
devices or daughter-boards in general purpose computers), or
through computer programs installed in computer storage on the
servers of the service 110 and executed by the processors of such
servers to perform the functions described herein. One of skill in
the art of system engineering and, for example, video content
hosting will readily determine from the functional and algorithmic
descriptions herein the construction and operation of such computer
programs.
[0032] The content hosting system 110 further comprises a system
database 130 that is communicatively coupled to the network 170.
The system database 130 stores data related to the content hosting
system 110 along with user and system usage information.
[0033] The system database 130 can be implemented as any device or
combination of devices capable of persistently storing data in
computer readable storage media, such as a hard disk drive, RAM, a
writable compact disk (CD) or DVD, a solid-state memory device, or
other optical/magnetic storage mediums. Other types of
computer-readable storage mediums can be used, and it is expected
that as new storage mediums are developed in the future, they can
be configured in accordance with the descriptions set forth
above.
[0034] The content hosting system 110 is further comprised of a
third party module 120. The third party module 120 is implemented
as part of the content hosting system 110 in conjunction with the
components listed above. The third party module 120 provides a
mechanism by which the system provides an open platform for
additional uses relating to electronic textbooks, much as an
application programming interface allows third parties access to
certain features of a software program. In some embodiments, third
party input may be limited to provision of content via content
provide computers 180B and content provider third party server 150.
Given the wide range of possible operation of system 100, however,
in some embodiments it may be desirable to open additional
capabilities for third parties who are not providing content to
access the system. For example, aggregated data regarding what
sections of a textbook are most often annotated may be helpful to
the author of the textbook (or to other authors) to determine where
additional explanation of difficult concepts might be
warranted.
[0035] In this description, the term "module" refers to
computational logic for providing the specified functionality. A
module can be implemented in hardware, firmware, and/or software.
Where the modules described herein are implemented as software, the
module can be implemented as a standalone program, but can also be
implemented through other means, for example as part of a larger
program, as a plurality of separate programs, or as one or more
statically or dynamically linked libraries. It will be understood
that the named modules described herein represent one embodiment of
the present invention, and other embodiments may include other
modules. In addition, other embodiments may lack modules described
herein and/or distribute the described functionality among the
modules in a different manner. Additionally, the functionalities
attributed to more than one module can be incorporated into a
single module. In an embodiment where the modules as implemented by
software, they are stored on a computer readable persistent storage
device (e.g., hard disk), loaded into the memory, and executed by
one or more processors included as part of the content hosting
system 110. Alternatively, hardware or software modules may be
stored elsewhere within the content hosting system 110. The content
hosting system 110 includes hardware elements necessary for the
operations described here, including one or more processors, high
speed memory, hard disk storage and backup, network interfaces and
protocols, input devices for data entry, and output devices for
display, printing, or other presentations of data.
[0036] Numerous variations from the system architecture of the
illustrated content hosting system 110 are possible. The components
of the system 110 and their respective functionalities can be
combined or redistributed. For example, the system database 130,
third party module 120, user front end server 140, and content
provider front end server 150 can be distributed among any number
of storage devices. The following sections describe in greater
detail the reader module 181, system database 130, and the other
components illustrated in FIG. 1 in greater detail, and explain
their operation in the context of the content hosting system
110.
[0037] FIG. 2 illustrates a functional view of a reader module 181
used as part of a electronic textbook system. In the embodiment
described above in connection with FIG. 1, the reader module is
implemented on user computer 180A, but it should be recognized that
in other embodiments, portions discussed herein could also be
implemented on other computers (e.g., those in content hosting
system 110) that are in communication with reader module 181.
[0038] Reader module 181 is configured to address the fact that
students use textbooks differently than other readers use typical
books. Students typically study from, rather than merely read,
textbooks. Studying is typically less linear than other reading, as
texts are rarely read in "start-to-finish" manner. Studying is
often much more interactive than typical reading, with annotations,
cross-referencing between problem sets and main portions, reference
to glossary or definitions sections, and the like. Studying is also
inherently social and collaborative as well--far more so than most
other types of reading. Learning in general, and studying in
particular, typically combines attention to textbooks with creation
and reference to notebooks, problem sets, lab experiment results,
lecture materials, and other related sources.
[0039] Reader module 181 includes various subsystems to facilitate
the specialized uses students make of textbooks. In the embodiment
illustrated in FIG. 2, reader module 181 includes an annotation
subsystem 220, an OCR subsystem 230, a collaboration subsystem 240,
an ordering subsystem 250, an input recognition subsystem 260, and
a daemon subsystem 270. Many of these subsystems interact with one
another, as described below.
[0040] Annotation subsystem 220 provides various user tools and
interfaces to allow students to mark up portions of an electronic
textbook as they may find most helpful for learning and studying
purposes. Annotation subsystem 220 includes conventional features
such as highlighting and text entry tools, and also includes more
advanced tools. For example, as described below annotation
subsystem 220 keeps track of textbook portions for which a student
has provided annotations, and collects those portions into a
personalized study guide based on a user command.
[0041] OCR subsystem 230 is a recognition subsystem that takes
information not originally in machine-readable form and converts it
to machine readable form. For example, OCR subsystem 230
communicates with annotation subsystem 220 to convert handwritten
student notes (entered graphically via finger or stylus gestures on
a touch screen, for instance) into machine readable text. As used
here, OCR subsystem 230 includes not only optical character
recognition, but other types of recognition as well, for instance:
voice-to-text recognition to allow a student to speak rather than
write annotations; image to text recognition for photographs the
student may take of a professor's notes on a blackboard during a
lecture; and other types of recognition as well that may be
provided within an electronic textbook or as a third party
add-on.
[0042] Collaboration subsystem 240 provides various user functions
that allow students to work together. As detailed below, for
example, users can share their annotations and notes with their
study group, their class section, their entire class, or other
users worldwide of the electronic textbook. Further, collaboration
subsystem 240 includes social network facilities to permit students
to undertake study sessions with audio and visual chat, to ask and
answer questions, and to submit questions to professors or teaching
assistants.
[0043] Ordering subsystem 250 represents tools that allow students
to obtain electronic textbooks and related materials. In one
embodiment, ordering subsystem 250 is implemented as an electronic
marketplace (e.g., the DROID.TM. marketplace implemented on the
ANDROID.TM. operating system for smart phones and tablet
computers). Third parties offer electronic textbooks and related
materials such as study guides, problem sets, updates, workbooks,
and the like. Some of these materials are available for purchase;
others are free. In some embodiments, provision via other
mechanisms (e.g., subscription, barter, "pay-per-view") is
supported, as may be desired by any subset of a student community
or content provider group.
[0044] Input recognition subsystem 260 provides user interface
tools to facilitate use of electronic textbooks and related
features. For instance, by sensing particular gestures on a touch
screen of user computer 180A as discussed in detail below, the
system temporarily shifts display of a textbook from a current page
to a new section, while keeping track of the section of primary
interest. Thus, a student working on a problem set section of the
textbook can quickly look back at the text of the chapter, or a
student reading a section for the first time can quickly jump to a
glossary section of the textbook for a definition of an unfamiliar
term or concept.
[0045] Reader module 181 is configured to permit user-selected
applications to run to enhance a student's ability to work with an
electronic textbook. For example, a student may purchase an
application that provides study questions on a per-chapter basis
for textbooks that do not include such questions. In addition,
reader module 181 includes a daemon subsystem 270 to provide
additional add-on features without the user launching a visible
application for such features.
[0046] Further detail regarding reader module 181 and various
subsystems thereof is provided below in connection with discussion
of FIGS. 4-6.
[0047] FIG. 3 illustrates a functional view of the system database
130 that stores data related to the textbook content hosting system
110. The system database 130 may be divided based on the different
types of data stored within. This data may reside in separate
physical devices, or it may be collected within a single physical
device.
[0048] With respect to content providers, partner data 370
comprises information regarding content providers or partners
registered with the content hosting system 110 that have permission
to create and deliver content. Partner data 370 includes provider
contact information.
[0049] User profile data storage 310 includes information about an
individual user (e.g., a student), to facilitate the payment and
collaborative aspects of system 100. Subscriber data storage 320
includes identifying information about the student, such as the
electronic textbooks the student has obtained and the social
network groups the student has joined. In some embodiments,
subscriber data storage 320 also maintains information regarding
the location in each of the student's textbooks where the student
is or was reading, to allow, for example, a student to read part of
a textbook chapter on a smart phone while on a campus bus and
continue reading from the same spot on the student's desktop
computer in a dorm room.
[0050] Account data storage 330 keeps track of the user's payment
mechanisms (e.g., Google Inc.'s CHECKOUT.RTM.) related to the
user's ability to obtain content from system 100. Social network
data storage 340 maintains the information needed to implement a
social network engine to provide the collaborative features
discussed herein, e.g., social graphs, social network preferences
and rules. Textbook data 350 stores the actual content that is
provided to users upon their request, such as electronic textbook
files. Add-on data storage 360 maintains information for related
features, such as non-static data relating to textbooks.
[0051] In one embodiment, conventional mechanisms are used to
implement many of the aspects of system database 130. For example,
the existing mechanisms from Google Inc.'s BOOKS.TM. GMAIL.TM.,
BUZZ.TM. CHAT.TM., TALK.TM., ORKUT.TM., CHECKOUT.TM., YOUTUBE.TM.,
SCHOLAR.TM., BLOGS.TM. and other products include aspects that can
help to implement one or more of storage facilities 310-370 and
modules 220-270. Google Inc. already provides eBook readers for
ANDROID.TM. devices (phones, tablets, etc.), iOS devices
(iPhones.RTM., iPads.RTM. and other devices from Apple, Inc.) and
various desktop Web browsers, and in one embodiment Google Inc.'s
EDITIONS.TM. eBook reader application is modified to provide the
functionality described herein.
[0052] Referring now to FIG. 4, there is shown a portable computer
400 (e.g., a tablet computer running the ANDROID.TM. operating
system) with a touch screen 401, a microphone 402, and a
front-facing camera 403. As is known to those skilled in the art,
such devices currently available typically also provide rear-facing
cameras, accelerometers, GPS receivers, Wi-Fi and advanced cellular
communications capabilities, and various other features. As shown,
computer 400 is running reader module 181 and displaying a page 404
from an electronic textbook.
[0053] In one embodiment, reader module 181 provides four tabs
above a main content area 404 allowing selection of four class
modules via the following user interface icons: a Syllabus tab 410,
a Textbook tab 420, a Notebook tab 430 and a Lectures tab 440. The
Syllabus tab 410 provides course-specific information for the
student, including a calendar of what portions of the text are
going to be dealt with on what dates, when assignments are due, and
when tests are scheduled. In one embodiment the student's
performance during the class is also tracked here (e.g., grades on
assignments and exams to date). The Textbook tab 420, shown in FIG.
4 as the currently selected tab, provides the actual textbook, as
well as a number of navigational and other tools related to view of
the textbook. The Notebook tab 430, when selected, causes the
student's notebook for the course to be displayed (see discussion
of FIG. 6, below). The Lectures tab 440, when selected, causes
display of lecture-related materials, such as a professor may
choose to provide to students. For example, a professor may provide
slide decks used in a lecture, videos, or other materials that
repeat or supplement what the professor presents in a lecture
session.
[0054] More specifically, the display provided under the Textbook
tab 420 includes a number of reading and annotation tools 407.
First, the name of the currently selected textbook is displayed
("Freshman Chemistry") in a drop-down menu allowing selection of
alternate texts for courses that use multiple textbooks. Not shown
are controls, in one embodiment provided above tools 407, for
selecting among various courses, for purchasing textbooks and
related items, for opening a session or chat as described below,
for launching a search engine, for changing system settings and for
getting automated help.
[0055] To the right of the textbook title is an icon to display a
table of contents, as well as an icon to change settings such as
text size. To the right of that is an icon to toggle between
regular view of the textbook and view of a user-generated study
guide (discussed below). To the right of that is an eye-shaped
icon, currently shown in the "eye shut" state, indicating whether
to show user annotations (also detailed below). The last four icons
are to add handwritten (pen) annotations (via a stylus or finger,
as desired and as supported by computer 400), highlighting, sticky
note annotations, and audio annotations to the textbook.
[0056] Below the primary content display area 404 are a set of page
navigation tools 408. From left to right they include an icon to
add a bookmark, an indicator of the current page (circle) in
relation to various chapters (noted by breaks in the horizontal
line) and previously set bookmarks, a number indicating the last
page of the textbook, and arrows representing previous page and
next page commands. The user touches on an appropriate portion of
this display of tools 408 to accomplish a corresponding action.
[0057] Also shown on FIG. 4 is a large bar 405, which in one
embodiment is colored yellow. Bar 405 indicates that the user has
created a sticky note relating to this portion of the text. Smaller
bars, in one embodiment displayed in gray, appear both within and
below bar 405; in one embodiment these represent other types of
annotations provided by the student, for example an audio
annotation or a video annotation. Likewise, vertical lines 406
indicate still other student input, in one embodiment highlighting
(straight lines) and handwritten annotations (squiggles). As noted
above, the closed-eye icon in tools 407 indicates that all of this
student-generated markup of the text is currently being hidden from
view.
[0058] Referring now also to FIG. 5, the same tablet computer is
shown, this time with the aforementioned eye icon in the open state
(annotations displayed). The bar 405 now shows as a full sticky
note, complete with a user control for settings (which in one
embodiment include an OCR option to convert the handwritten text to
clean machine-searchable text and an option to toggle between
handwritten and machine text versions for display). In one
embodiment, a small "resize handle" icon appears at the bottom of
the note to allow the note to be made larger or smaller as the user
may desire, and an "X" in the upper right hand corner of the note
allows the user to delete the note if desired. The small gray bars
referenced above are replaced with a "TV" icon indicating a video
annotation as well as a small green circle with a number in it
indicating how many comments have been entered concerning this
annotation (e.g., by other students in a collaborative study
session). A similar loudspeaker icon with a small green circle and
corresponding number indicates an audio annotation and comments on
that. Likewise, the highlighting and handwritten text previously
indicated by vertical lines is now fully displayed. Also in this
display, an indication of the current bookmarked status of the page
is included in the upper left-hand corner, along with an "X" which,
when touched by the user, removes the bookmark.
[0059] In some embodiments, reader module 181 uses accelerometer
and other positioning input from computer 400 and interprets
certain movements as commands. As one example, tilting computer 400
from portrait mode (as shown in FIGS. 4 and 5) to landscape mode
triggers a change in the display from one page to two-page spread
format. An abrupt partial tilt, on the other hand, when in the
"show annotations" mode illustrated in FIG. 5, causes the
annotations to be "poured" into the margin and the display switched
to the "hide annotations" (corresponding to "closed eye" icon) mode
illustrated in FIG. 4. A tilt in the other direction pours the
annotations back into view.
[0060] Referring now to FIG. 6, the same user computer 400 is now
shown after the user has selected the Notebook tab 430. In this
mode of operation, in addition to the in-textbook annotations
described above, a student can readily create and maintain a
notebook keyed to specific portions of a textbook or lecture. A set
of tools 607 provides icons that (from left to right) allow a
student to change settings, such as displaying hand-entered text as
shown here or a machine-recognized typed version of the same; to
capture information from a whiteboard or chalkboard using camera
403 or (if equipped) a back-facing camera on computer 400; to share
notes with others, to enter notes with a pen tool as described
above, to highlight notes, to add a sticky note to the notebook,
and to capture audio annotations corresponding to the notes from
microphone 402. In one embodiment, the student can provide not only
handwriting, but hand drawn shapes 605 as well in both the notebook
and textbook annotation modes of operation. Likewise the student
can provide notes in outline form 606. The page navigation tools
608 in the notebook mode of operations include icons (from left to
right) to add a bookmark, add a new page to the notebook, and
navigate among the pages of the notebook as previously explained in
connection with textbook page navigation.
[0061] For annotations that are not already in machine-readable
text form, the reader provides user tools for
audio/visual/character recognition to convert such annotations into
a form that can be machine searched and accessed. The reader
includes a user interface tool to allow a user to toggle as desired
between such original images and the corresponding machine readable
text. In addition to recognizing text images, the reader also
provides a user with an option to recognize lines and various
geometric shapes from imaged or handwritten notes, such that
cleaned-up versions of flow diagrams, molecular structures and the
like can be easily made from sources having varied legibility.
[0062] Students' use of textbooks involves certain operations not
typical of reading. For instance, most readers do not have
questions that regularly arise in connection with their reading,
but this is common for students who do not understand a concept
very well from the textbook presentation. Accordingly, input
recognition system 260 of reader module 181 provides a number of
predetermined operations specifically oriented to textbook use, and
also permits students to easily configure their own desired
operations.
[0063] In one example, already mentioned, a student may find a
portion of a text particularly "dense" in concepts, and may want to
include more annotation for that section than could reasonably fit
in the margins or elsewhere within the textbook display. In such
instances, input recognition system 260 detects when a user has
circled a portion of a textbook's content, either with a finger or
a stylus. To indicate such detection, the selected area is
displayed with a "glowing" appearance for a period of time. If,
during that period of time, the user touches the glowing area and
drags it to hover over the Notebook tab 430, input recognition
system 260 detects this as a command to copy that portion of text
into the student's notebook, where further room for annotation is
available. In that event, the student's notebook becomes the active
display, allowing the student to place the copied portion from the
textbook anywhere in the notebook that may be desired, and to
annotate on and around the added excerpt.
[0064] As a second example, specific annotations are immediately
recognized as corresponding to commands rather than actual
annotations. For example, in one embodiment a handwritten
annotation in the form of a question mark with a circle around it
is interpreted as a request to send a question regarding the nearby
text to the appropriate teaching assistant for that course (or
other predetermined moderator), and a dialog box immediately opens,
preaddressed to the teaching assistant, allowing the student to ask
the question. In one embodiment, the message to the teaching
assistant is automatically tagged with the corresponding portion of
the text so that the student does not need to include any context
with the specific question, but can just include the question in a
way that might be confusing without context. For example, if the
text shows an illegal divide-by-zero operation, the student's
question could simply be: "Why can't you do this?" without any
further contextual information.
[0065] Likewise, other predefined gestures are provided in various
embodiments. A "c" drawn with a circle around it, or a cartoon text
balloon shape, is interpreted as a command to open a chat panel. A
"k" with a circle around it or a pound sign (#) is interpreted as a
command to open a keyboard panel. A squiggly line or repeated
zig-zag is a command to delete a word or diagram. A handwritten
name ("Jim") opens a chat panel with a familiar classmate. A
specified word ("calc") invokes an installed add-on.
[0066] Users are also given the option of predefining their own
gestures and self-recording arbitrary sequences of actions (similar
to macros) that will be associated with those gestures. As one
example, a user may define a letter "Q" with a circle around it to
mean "Quit thoughtfully" and make that gesture correspond to saving
all notebook edits, quitting the open textbook, and emailing notes
to other study group members (e.g., Mike, Bob and Mary).
[0067] As noted above in connections with FIGS. 4 and 5, reader
module 181 enables highlighting, sticky notes and annotations
generally (e.g., 405, 406) to be selectively shown or marginalized.
An advantage of marginalizing, rather than completely hiding,
annotation is that marginal marks remind the student upon a second
or third reading of a section that there are potentially helpful
annotations available for that section. Furthermore, use of the
accelerometer of computer 400 to either show or marginalize
annotations upon a quick tilting of computer 400 provides a very
quick and intuitive way for the student to switch between these two
types of display. In some embodiments, user interface controls
allow the specific gestures used to indicate show/marginalize
annotations to be adjusted and otherwise changed, so that the
sensitivity of these can be tuned to match a user's preference.
[0068] Reader module 181 also enables a student to mark certain
annotations as private. In one embodiment, annotations are by
default shared anonymously with the public (i.e., all others who
have access to the electronic textbook), but in some environments,
alternate embodiments may be more selective in sharing as may be
more appropriate.
[0069] In one embodiment, reader 181 is configured to tag all
portions of a textbook for which annotations have been provided
such that a student can request a personalized study guide,
comprised solely of the highlighted sections, to be generated. In
one embodiment, each tagged section remains hyperlinked to the
original full text to allow the student to quickly switch back to
the full text for additional context regarding a particular section
of interest. In one embodiment, this is accomplished by placing an
underlined page number at the left margin of each section of the
study guide; clicking on that number takes the user to the
indicated page in the textbook.
[0070] In a related embodiment, collaboration subsystem 240 is
configured to obtain information from other students as well
regarding portions of the textbook that they have highlighted for
generation of a user guide based on their annotated sections, in
addition to the user's own annotated sections. In one embodiment,
the student can select the student's own work group, other
classmates, other students at the same school or at other select
schools, or even all students worldwide for purposes of determining
which annotations should be used to generate the study guide. To
avoid a situation in which such "crowd-sourced" generation of
annotated selections produces too large a study guide, in one
embodiment a slider-style user interface (or other suitable user
interface) allows a student to adjust selectivity for generation of
the study guide. For instance, one setting includes all sections
highlighted by any student, but another setting requires that at
least five students provided annotations for a section to include
it in the study guide (or for consideration of all students
worldwide, 5% of the students providing annotations). Thus, a
student may tailor a user guide for the amount of time the student
may have available to use the guide. Undoubtedly, some students who
have not read the entire text may also use this feature to
determine which portions are considered most important for a first
reading before an examination.
[0071] To provide a user experience showing that such additional
annotated sections are being collected, an animated user interface
that moves or "slurps" these additional annotated sections from
outside the current field of view is shown when the user changes
the slider to include more sections, and the additional sections
are slurped out of the field of view as the user changes the slider
to be more selective in which sections to have in the study
guide.
[0072] In one embodiment, user interface tabs/buttons allow a user
to select "My highlights," "Classmates' highlights," or "Everyone's
highlights."
[0073] Many of the computers 400 on which reader module 181 will be
implemented support multi-touch navigation by a user. However, not
all of the multi-touch commands that may be most helpful for use of
electronic textbooks are provided in a native manner on such
devices. For instance, the standard "pinch-zoom" and swipe features
available to change magnification and move through pages and
chapters are certainly useful with textbooks, but more specific
navigation choices are supported by reader module 181. For example,
as noted above users of textbooks often need to make quick
reference to another portion of the text and then return to where
they were in the text. With a paper book, one often sticks a finger
in the book at the current page and then moves to the page of
temporary interest. Reader 181 permits a corresponding operation by
placing a finger of one hand down on the screen 401 at a location
showing the current page (e.g., near 404 on FIG. 4) and then using
other existing page navigation techniques to move to another page
(e.g., by swiping with two fingers of the other hand to move back a
number of pages).
[0074] Additionally, the navigation footer 408 is persistent, and
the user can quickly move around the book (either provisionally
using the one-finger hold on the current page or normally) using
this interface at any time.
[0075] When a user provisionally moves to a page, for instance to
skim it, the user can either release using the left hand to return
to the original page or release using the right hand to commit to
the new page and abandon the original page.
[0076] In a related aspect to the collaboration among students
discussed above, two or more students who are engaged in a chat
regarding a textbook or who are in a study session using the
textbook often need to help each other based on particular portions
of a text. To facilitate this, collaboration subsystem 240 keeps
track of where each student is in the textbook during collaboration
and sends that information to the computers 400 of the other
students in the collaboration, so that their current location is
indicated for the others to see. Likewise, one student's
annotations appear on the other students' computers 400 (with color
coding for each student's annotations), as do gestures made by one
student (e.g., pointing to a particular portion of text using
either a mouse or a finger press on a touch screen device).
[0077] Referring now to FIG. 7, the computer 400 implementing a
reader 181, discussed above with respect to FIG. 4 is shown once
again, this time with a display screen 401 including a contextual
menu 701. In one embodiment, once a user presses and holds on a
portion of the screen 401, a circle begins to appear, gradually
drawn in a clockwise direction around the user's finger. The circle
is complete after a finite period of short duration (say
approximately 500 milliseconds) and then turns into the contextual
menu 701. The purpose of such animation is to alert the user that
by holding a finger on the screen, the user is requesting such a
menu (release of the finger before the menu is complete makes the
incomplete circle disappear and the menu is not formed). In
addition, the animation assists users pressing near the edge of a
screen to see that menu 701 is being created, even if a portion of
the developing circle is obscured by the edge of the screen.
Contextual menu 701 provides, in this embodiment, six areas for
further user selection: a central area with an "X" in it to close
the menu (tapping outside the menu will also close it), and five
choices for further user selection. Menu 701 is a contextual menu
because the user choices are not always the same, but instead are
based on what is displayed on screen 401 as well as where on the
screen the user has asked the menu to appear. For instance, if the
user presses a finger over a chart or diagram, a different set of
choices may appear than if the user presses a finger over body
text, or over white space as shown in FIG. 7.
[0078] Referring now to FIG. 8, there is shown a progression of
contextual menus in one example, from a menu of action choices 801
to a display showing a selected choice 811, and then to a menu 821
of a series of additional user choices that result. Specifically,
menu 801 includes five user choices related to annotations, in this
case color, stroke, chat, sync and share, that the user can select.
In this instance, color denotes a choice of a color for
annotations, stroke denotes gesture recognition activation (and in
alternate embodiments, various gesture-related configuration and
operation choices), chat denotes activation of a chat window, sync
denotes synchronizing the user's display with that of other
connected students (e.g., to share annotations), and share denotes
sharing of annotations with other students. The latter two choices
also have small triangular blocks in the lower right of their
respective menu portions in menu 801; in this embodiment these
blocks indicate that the choices will spawn additional user choices
(i.e., not result in any action being taken immediately without the
opportunity for further user selection, for example by presentation
of a further menu of user choices). A central circle 802 with an
"X" in it provides a mechanism to close the circular menu, and is
primarily used for newer users who may not understand that menu 801
can also be closed by simply tapping outside of menu 801. In a
related embodiment, small graphics rather than words are used to
denote the user's options: An artist's palette for "color", a
swoosh symbol for "gesture", a word bubble for "chat", a circle
with rotating arrows for "sync", and a document with an arrow for
"share".
[0079] Assuming for present purposes that the user selects "color"
from menu 801, that portion of the menu gradually expands as shown
in circle 811, providing the user recognition that the input has
been received. Again, this takes approximately 500 milliseconds,
after which the next set of user choices is displayed via menu 821.
In this instance, the user choices are not textual at all, but
include different colors that the user may select by tapping on the
appropriately colored portion. Once the user does that, a similar
indication of recognition is provided by having that color
similarly grow into a circle that is entirely made up of the
selected color (not shown). For choices that do not result in
further menus or other selection choices, indication of finality is
provided by having the choice blink in confirmation and then fade
from view. In this instance, the selected color, after growing to
encompass the entire circle (other than the small circle 802)
blinks and then fades from view.
[0080] In some embodiments, a different number of user selections
than five is provided in menus 801 and 821, as may be appropriate
for a given context in which the menu is enabled. Other contexts
will also call for a different set of user choices within menu 801.
For example, referring again to FIG. 4, if the user presses and
holds on a portion of an annotation area, e.g., 405, an annotation
menu appears differing from menu 801 in that "delete" appears
rather than "color", "append" appears rather than "stroke", and
"question" appears rather than "chat" (with the remaining items,
"sync" and "share" still appearing as in menu 801). In this
instance, delete is used to remove an annotation, append is used to
send the annotation from the textbook display to the user's
notebook (shown on FIG. 6), and question is used to embed the
annotation in a question to be addressed to a fellow student,
teaching assistant or professor. Wherever possible, menu items that
are common across contexts are placed in consistent areas on menu
801 to facilitate ease of use.
[0081] Contextual menus, e.g., 801 are brought up in different
forms based not only on location of the user's finger press (e.g.,
over body text of the book as opposed to over a user's own
annotation), but also based on when the press is made (e.g.,
immediately after highlighting a section of text) and based on
other triggering events (e.g., recently receiving a question or
annotation from another student) that might warrant actions that
would not be needed otherwise. By providing menus with
context-driven choices, the need for interface "real estate" on the
screen is reduced, since inapplicable choices simply do not appear
rather than appearing in grayed-out text as is done with many
conventional menu systems.
[0082] Some portions of above description describe the embodiments
in terms of algorithms and symbolic representations of operations
on information. These algorithmic descriptions and representations
are commonly used by those skilled in the data processing arts to
convey the substance of their work effectively to others skilled in
the art. These operations, while described functionally,
computationally, or logically, are understood to be implemented by
computer programs executed by a processor, equivalent electrical
circuits, microcode, or the like. Furthermore, it has also proven
convenient at times, to refer to these arrangements of operations
as modules, without loss of generality. The described operations
and their associated modules may be embodied in software, firmware,
hardware, or any combinations thereof
[0083] As used herein any reference to "one embodiment" or "an
embodiment" means that a particular element, feature, structure, or
characteristic described in connection with the embodiment is
included in at least one embodiment. The appearances of the phrase
"in one embodiment" in various places in the specification are not
necessarily all referring to the same embodiment.
[0084] As used herein, the terms "comprises," "comprising,"
"includes," "including," "has," "having" or any other variation
thereof, are intended to cover a non-exclusive inclusion. For
example, a process, method, article, or apparatus that comprises a
list of elements is not necessarily limited to only those elements
but may include other elements not expressly listed or inherent to
such process, method, article, or apparatus. Further, unless
expressly stated to the contrary, "or" refers to an inclusive or
and not to an exclusive or. For example, a condition A or B is
satisfied by any one of the following: A is true (or present) and B
is false (or not present), A is false (or not present) and B is
true (or present), and both A and B are true (or present).
[0085] In addition, use of the "a" or "an" are employed to describe
elements and components of the embodiments herein. This is done
merely for convenience and to give a general sense of the
invention. This description should be read to include one or at
least one and the singular also includes the plural unless it is
obvious that it is meant otherwise.
[0086] Upon reading this disclosure, those of skill in the art will
appreciate still additional alternative structural and functional
designs for a system and a process for providing interfaces for
electronic books through the disclosed principles herein. Thus,
while particular embodiments and applications have been illustrated
and described, it is to be understood that the disclosed
embodiments are not limited to the precise construction and
components disclosed herein. Various modifications, changes and
variations, which will be apparent to those skilled in the art, may
be made in the arrangement, operation and details of the method and
apparatus disclosed herein without departing from the spirit and
scope defined in the appended claims.
* * * * *