U.S. patent application number 16/826153 was filed with the patent office on 2020-09-24 for online classroom system and method for monitoring student activity.
This patent application is currently assigned to Foundry College, Inc.. The applicant listed for this patent is Foundry College, Inc.. Invention is credited to Michael Laurence Davis, Stephen M. Kosslyn, Scott Lane Williams.
Application Number | 20200302812 16/826153 |
Document ID | / |
Family ID | 1000004899450 |
Filed Date | 2020-09-24 |
![](/patent/app/20200302812/US20200302812A1-20200924-D00000.png)
![](/patent/app/20200302812/US20200302812A1-20200924-D00001.png)
![](/patent/app/20200302812/US20200302812A1-20200924-D00002.png)
![](/patent/app/20200302812/US20200302812A1-20200924-D00003.png)
![](/patent/app/20200302812/US20200302812A1-20200924-D00004.png)
![](/patent/app/20200302812/US20200302812A1-20200924-D00005.png)
![](/patent/app/20200302812/US20200302812A1-20200924-D00006.png)
![](/patent/app/20200302812/US20200302812A1-20200924-D00007.png)
![](/patent/app/20200302812/US20200302812A1-20200924-D00008.png)
![](/patent/app/20200302812/US20200302812A1-20200924-D00009.png)
![](/patent/app/20200302812/US20200302812A1-20200924-D00010.png)
View All Diagrams
United States Patent
Application |
20200302812 |
Kind Code |
A1 |
Williams; Scott Lane ; et
al. |
September 24, 2020 |
ONLINE CLASSROOM SYSTEM AND METHOD FOR MONITORING STUDENT
ACTIVITY
Abstract
Systems and methods for online classroom instruction are
described that facilitate the learning and application of course
information in an active and synchronous learning environment. The
system monitors the activity of the students in each breakout
group, and the teacher is provided with a graphical indication, for
each breakout group, of the amount of activity. The teacher may
also use the graphical interface to either view a particular
breakout group, or become n active member of a particular breakout
group. The teacher may also move students from one breakout group
to another.
Inventors: |
Williams; Scott Lane; (San
Anselmo, CA) ; Davis; Michael Laurence; (Palo Alto,
CA) ; Kosslyn; Stephen M.; (San Francisco,
CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Foundry College, Inc. |
San Francisco |
CA |
US |
|
|
Assignee: |
Foundry College, Inc.
San Francisco
CA
|
Family ID: |
1000004899450 |
Appl. No.: |
16/826153 |
Filed: |
March 20, 2020 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62821829 |
Mar 21, 2019 |
|
|
|
62856658 |
Jun 3, 2019 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G09B 19/003 20130101;
G09B 5/00 20130101; G09B 7/06 20130101; G09B 5/02 20130101; G09B
5/065 20130101 |
International
Class: |
G09B 5/00 20060101
G09B005/00 |
Claims
1. A method of monitoring a plurality of breakout groups of an
active-learning class using a plurality of electronic devices in a
computer network, where the plurality of electronic devices
includes a first electronic device operated by an instructor, a
plurality of second electronic devices each operated by a student
of a plurality of students, and one or more servers, where each
breakout group of the plurality of breakout groups includes a
plurality of students, and where the monitoring is performed by the
instructor, said method comprising: for each breakout group (BOG)
of the plurality of BOGs, determining a BOG activity level over the
computer network from the activity of each student's electronic
device during the BOG, where the determining determines using one
or more electronic devices of the plurality of electronic devices;
and presenting on a display of the first electronic device a
graphical representation of BOG activity level for each BOG of the
plurality of BOGs.
2. The method of claim 1, where the BOG activity level is
determined from interactions between students of each BOG or from
interactions with the shared BOG workspace.
3. The method of claim 1, where the BOG activity level is a measure
of the amount of communication between the plurality of students of
the BOGs.
4. The method of claim 3, where the communication is by talking as
determined from the each electronic device's microphone or by
texting between electronic devices.
5. The method of claim 3, where the BOG activity level is the
amount of talking by the most talkative member of the BOG divided
by the average amount of the talking by all of the other members of
the BOG.
6. The method of claim 3, where the BOG activity level is a measure
of the amount of cursor activity in a shared document.
7. The method of claim 6, where the measure is an amount of cursor
activity in a shared document by a most active member of the BOG
divided by an average amount of cursor activity by the other
members of the BOG.
8. The method of claim 1, where the BOG activity level is
determined from the use of group chat.
9. The method of claim 1, where the BOG activity level is
determined from answers to quizzes by students of the BOG.
10. The method of claim 1, where the BOG activity level is
determined from how many times students of the BOG have asked
questions in class.
11. The method of claim 1, where the BOG activity level is
determined from the the variance in talking between students of the
BOG.
12. The method of claim 1, further comprising: forming a one-to-one
map between BOG activity level and a color, where the graphical
representation includes equally sized geometric shapes for each BOG
of the plurality of BOGs, and where the geometric shape for each
BOG of the plurality of BOGs is coded with the color as mapped from
the BOG activity level.
13. The method of claim 12, further comprising labeling each
graphical representation with the name of the corresponding BOG or
the name of at least one student of the corresponding BOG.
14. The method of claim 1, further comprising, accepting a
selection on the instructor's electronic device of one BOG of the
plurality of BOGs, and providing, on the instructor's electronic
device, a view of the BOG.
15. The method of claim 1, where the instructor's electronic device
is operable to move a student from one BOG to another BOG.
16. The method of claim 1, where the instructor's electronic device
is operable for selecting a BOG of the plurality of BOGs and
automatically visit a selected BOG.
17. An apparatus for monitoring a plurality of breakout groups
(BOGs) of an active-learning class using a plurality of electronic
devices in a computer network, where the plurality of electronic
devices includes a first electronic device operated by an
instructor, a plurality of second electronic devices each operated
by a student of a plurality of students, and one or more servers,
where each BOG of the plurality of BOGs includes a plurality of
students, and where the monitoring is performed by the instructor,
said apparatus comprising: one or more processors of at least one
electronic device of the plurality of electronic devices programmed
to for each BOG of the plurality of BOGs, determine a BOG activity
level over the computer network from the activity of each student's
electronic device during the BOG, and present on a display of the
first electronic device a graphical representation of BOG activity
level for each BOG of the plurality of BOGs.
18. The apparatus of claim 17, where the BOG activity level is
determined from interactions between students of each BOG or from
interactions with the shared BOG workspace.
19. The apparatus of claim 17, where the BOG activity level is a
measure of the amount of communication between the plurality of
students of the BOGs.
20. The apparatus of claim 19, where the communication is by
talking as determined from the each electronic device's microphone
or by texting between electronic devices.
21. The apparatus of claim 19, where the BOG activity level is the
amount of talking by the most talkative member of the BOG divided
by the average amount of the talking by all of the other members of
the BOG.
22. The apparatus of claim 19, where the BOG activity level is a
measure of the amount of cursor activity in a shared document.
23. The apparatus of claim 22, where the measure is an amount of
cursor activity in a shared document by a most active member of the
BOG divided by an average amount of cursor activity by the other
members of the BOG.
24. The apparatus of claim 17, where the BOG activity level is
determined from the use of group chat.
25. The apparatus of claim 17, where the BOG activity level is
determined from answers to quizzes by students of the BOG.
26. The apparatus of claim 17, where the BOG activity level is
determined from how many times students of the BOG have asked
questions in class.
27. The apparatus of claim 17, where the BOG activity level is
determined from the variance in talking between students of the
BOG.
28. The apparatus of claim 17, where said one or more processors
are further programmed to: form a one-to-one map between BOG
activity level and a color, where the graphical representation
includes equally sized geometric shapes for each BOG of the
plurality of BOGs, and where the geometric shape for each BOG of
the plurality of BOGs is coded with the color as mapped from the
BOG activity level.
29. The apparatus of claim 28, where said one or more processors
are further programmed to: label each graphical representation with
the name of the corresponding BOG or the name of at least one
student of the corresponding BOG.
30. The apparatus of claim 17, where said one or more processors
are further programmed to: accept a selection on the instructor's
electronic device of one BOG of the plurality of BOGs, and provide,
on the instructor's electronic device, a view of the BOG.
31. The apparatus of claim 17, where the instructor's electronic
device is operable to move a student from one BOG to another
BOG.
32. The apparatus of claim 17, where the instructor's electronic
device is operable for selecting a BOG of the plurality of BOGs and
automatically visit a selected BOG.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of U.S. Provisional
Application Nos. 62/821,829 filed Mar. 21, 2019 and 62/856,658
filed on Jun. 3, 2019, the contents of which are hereby
incorporated by reference in its entirety.
BACKGROUND OF THE INVENTION
Field of the Invention
[0002] The present invention generally relates to education, and
more particularly to an apparatus and method for on-line active
learning.
Discussion of the Background
[0003] Synchronous online learning, also known as live, real-time
online learning, uses computer and telecommunications technology to
connect teachers and students in real time. Such systems are used,
for example, for teaching college and high school courses and for
many forms of video conferencing, such as used during college
"office hours," study groups, and discussion sections.
[0004] Available synchronous online learning systems are simple
evolutions of existing video-conferencing services. Although such
systems generally offer a classroom setup where instructors can
present slideshows while on live video with students, technological
limitations prevent them from offering a richer classroom learning
experience.
[0005] For example, prior art systems do not promote active
learning by keeping all students engaged in the learning process.
Thus, prior art systems are typically limited to merely presenting
a slideshow, and have limited capabilities of forming or monitoring
breakout groups.
[0006] Other prior art systems treat all students the same,
independent of their classroom performance. Thus, prior art systems
determine student performance during a class, but do not use this
information to adjust lessons or to form breakout groups based on
the capabilities that are relevant to that activity. As a result,
breakout groups may include mismatched or poorly matched students.
Further, the tests provided to the students do not reinforce
material that must be mastered.
[0007] Thus, there is a need in the art for a system that promotes
active learning, determines and uses student performance to
automatically tailor instruction, and continually engages students
with challenging new material. Such a system and method should be
easy for the instructor and student to use, should provide a
compelling learning environment that promotes active learning and
discussion, should work with standard networked-computer systems,
and should be scalable for use by a large number of students.
BRIEF SUMMARY OF THE INVENTION
[0008] Certain problems in prior art computer platforms and methods
of instruction are solved with a computer platform and methods that
provide an active-learning classroom environment that: empowers
teachers by giving them access to tools that promote active
learning; prepares students for active learning by engaging them
with new material.
[0009] Yet other certain embodiments provide a method of monitoring
a plurality of breakout groups (hereafter referred to as "BOGs") an
active-learning class using a plurality of electronic devices in a
computer network, where the plurality of electronic devices
includes a first electronic device operated by an instructor, a
plurality of second electronic devices each operated by a student
of a plurality of students, and one or more servers, where each BOG
of the plurality of BOGs includes a plurality of students, and
where the monitoring is performed by the instructor, The method
includes: for each BOG of the plurality of BOGs, determining a BOG
activity level over the computer network from the activity of each
student's electronic device during the BOG, where the determining
determines using one or more electronic devices of the plurality of
electronic devices; and presenting on a display of the first
electronic device a graphical representation of BOG activity level
for each BOG of the plurality of BOGs.
[0010] Certain embodiments provide an apparatus for monitoring a
plurality of BOGs an active-learning class using a plurality of
electronic devices in a computer network, where the plurality of
electronic devices includes a first electronic device operated by
an instructor, a plurality of second electronic devices each
operated by a student of a plurality of students, and one or more
servers, where each BOG of the plurality of BOGs includes a
plurality of students, and where the monitoring is performed by the
instructor. The apparatus includes one or more processors of at
least one electronic device of the plurality of electronic devices
programmed to for each BOG of the plurality of BOGs, determine a
BOG activity level over the computer network from the activity of
each student's electronic device during the BOG, and present on a
display of the first electronic device a graphical representation
of BOG activity level for each BOG of the plurality of BOGs.
[0011] Certain other embodiments provide a computer platform and
methods that present students with learning tasks that are not too
difficult as to be frustrating and not so easy as to be boring;
ideally, tasks are at just the right level of difficulty for each
student. Thus, certain embodiments assign students automatically to
BOGs based on having comparable levels of the relevant capabilities
(as determined from previous performance measures, such as quiz
scores), and "multileveled" activities are presented to the BOGs
that they can perform more or less deeply. For example, in a course
on clear communication, students might be asked to go through
paragraphs and detect all ambiguity. Some ambiguity might be
flagrantly ambiguous words, which everyone can identify; some might
be more subtly ambiguous words, which will be identified by more
adept students; and some might be very subtle relations among
phrases, which only the most adept student will identify. Because
the groups are automatically formed so that members have comparable
capability, they will "self titrate" so that they are not
frustrated or bored, and will drill down to an appropriate level in
the materials.
[0012] Certain embodiments provide a computer platform and method
that facilitates the learning and application of course
information, as transmitted from an instructor to a plurality of
students, in an active and synchronous learning environment. In
certain embodiments the instructor and students communicate over
networked computers in either a classroom mode or a breakout mode.
In the classroom mode, the instructor presents course material to
the students and two-way communication is provided between the
instructor and students and between individual students. In the
breakout mode, the students are divided into independent BOGs, each
consisting of a small number of students, which are presented with
group learning activities. In certain embodiments, the students are
grouped according to the type and/or requirements of the learning
activity which may, for example and without limitation, be debates,
role playing, group problem solving, or guided analysis. The
learning activities generally are designed to be "multilayered," so
that students of different abilities all can get something out of
them.
[0013] Certain other embodiments provide a computer platform and
methods that promote learning and subsequent memory by inducing
students to process relevant information. Thus, certain embodiments
form a plurality of BOGs during a class comprising small groups of
students. Learning tasks for each BOG are provided with incentives
and consequences: students are incentivized to pay attention and
process deeply because they know that they will need that
information for a subsequent activity (e.g., a follow-on BOG). If
students do not pay attention, the consequence will be that they
cannot perform a subsequent task and are embarrassed in front of
their peers and perform poorly on a quiz.
[0014] Certain embodiments provide a computer platform and methods
include a scalable videoconferencing and teaching environment and
method for teaching classes. Various embodiments are capable of:
providing lectures for to up to 200 students, with the lectures
punctuated by active learning, which may be in the form of polls,
quizzes, and interactions over Chat; forming small breakout groups
(hereafter referred to as "BOGs"), ranging from 2-8 students. These
groups are either configured in advance or created on the fly, as
needed for active; earning; specifying sequences of BOGs, with
students who played different roles in an initial group
automatically being sorted into new groups based on those roles;
after a sequence of different groups, allowing students
automatically to be placed back into an earlier BOG, so that they
can revisit questions posed earlier; providing a dashboard for the
instructor, who can see a heat map indicating how much of a
specific type of activity (e.g., talking, engagement with writing)
is taking place in each BOG; permitting the instructor to visit any
specific BOG, interacting with those students; allowing the
instructor to "broadcast" to students in all BOGs; permitting
students to vote and take polls at any time, including during
debriefing after BOGs; using shared files at any point during
class, including in BOGs; permitting the work products from BOGs to
be shared; providing students with the ability to send messages to
the class; providing students and the instructor with screen
sharing capabilities; providing individualized quizzes at any point
during the class, with the same or different items presented to
different students; assigning students to BOGs using quiz
performance data, so that students at the same level are grouped
together or, alternatively, so that each group has at least one
member that understands the material; tagging particular questions
a given student received on a quiz and ensuring that students who
re-take a quiz automatically receive a new set of relevant
questions; and providing a student dashboard that shows students
their progress towards achieving specific competencies.
[0015] These features together with the various ancillary
provisions and features which will become apparent to those skilled
in the art from the following detailed description, are attained by
the system and method of the present invention, preferred
embodiments thereof being shown with reference to the accompanying
drawings, by way of example only, wherein:
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING
[0016] FIGS. 1 and 2 are schematic diagrams illustrative of one
embodiment of an on-line platform;
[0017] FIG. 3 is a high-level architecture diagram of the front
end/web app detail of the platform;
[0018] FIG. 4 is a general schematic of Content Management System
(CMS);
[0019] FIG. 5 is a schematic of an instructional design view of the
CMS;
[0020] FIG. 6 illustrates a live classroom view of the CMS for
presenting lesson content;
[0021] FIG. 7 is a schematic that illustrates the use of the
College API Server and a Load Balancer;
[0022] FIG. 8 is a schematic of the structure of the Database;
[0023] FIG. 9A is a schematic of publish/subscribe services are
used in the classroom setting;
[0024] FIG. 9B is a schematic that illustrates how
publish/subscribe services control student's behavior and
views;
[0025] FIG. 10 is a schematic that illustrates publish/subscribe
services for chat;
[0026] FIG. 11 is a schematic that illustrates publish/subscribe
services used for learning activities;
[0027] FIG. 12 is a schematic illustrating the creation of quizzes,
midterms, or final exam;
[0028] FIG. 13 is a schematic which illustrates one embodiment for
how the system forms BOGs during a lesson;
[0029] FIG. 14 illustrates forming BOGs with students having
similar ranking;
[0030] FIG. 15 illustrates forming BOGs with students based on more
than one student metric;
[0031] FIG. 16 illustrates the generation of the BOGs Heat Map
widget;
[0032] FIG. 17A illustrates the display of a plurality of video
feeds in the Classroom Videos Grid;
[0033] FIGS. 17B, 17C, and 17D illustrate three sequential views of
the Classroom Videos Grid of FIG. 17A;
[0034] FIGS. 18A and 18B illustrates the effect of the
"raise/lower" hand tool on the Classroom Videos Grid;
[0035] FIGS. 19A and 19B illustrates the effect of an instructor
spotlighting a student from Classroom Videos Grid;
[0036] FIG. 20A is a screenshot showing accesses tools to set up
rules for forming a BOG;
[0037] FIG. 20B is a screenshot showing the elements for organizing
BOGs;
[0038] FIG. 20C is a screenshot showing the embedding of a shared
whiteboard for the group.
[0039] FIG. 21 is a screenshot illustrating one embodiment of the
Everyone In Class (Instructor View) Instructor View State;
[0040] FIG. 22 is a screenshot illustrating one embodiment of the
BOG Activity In Progress Instructor View State;
[0041] FIG. 23 is a screenshot illustrating one embodiment of the
Everyone In Class (Students View) Student View State;
[0042] FIG. 24 is a screenshot illustrating one embodiment of the
Everyone In Class (Students View) Student View State during a
classroom discussion;
[0043] FIG. 25 is a screenshot illustrating one embodiment of the
Showing Poll Results Student View State;
[0044] FIG. 26 is a screenshot illustrating one embodiment of BOG
Activity In Progress Instructor Broadcasting/Intercom To All Groups
Student View State;
[0045] FIG. 27 is a screenshot illustrating one embodiment of the
"Spotlit" BOG Student View State;
[0046] FIG. 28 as a screenshot illustrating one embodiment of the
"Spotlit" BOG Instructor View State;
[0047] FIG. 29 is a screenshot of a document from a breakout
session; and
[0048] FIG. 30 is a screenshot of a document from a breakout
session imported into the Everyone In Class (Students View) Student
View State.
[0049] Reference symbols are used in the Figures to indicate
certain components, aspects or features shown therein, with
reference symbols common to more than one Figure indicating like
components, aspects or features shown therein.
DETAILED DESCRIPTION OF THE INVENTION
[0050] In general, the present invention includes a computer
network-based system ("platform"), in which an instructor and
students interact. Embodiments are presented herein for an
Internet-based system that provides webpages to the instructor and
to each of a plurality of students via web browsers, and which
permits the instructor and students to interact through their
browsers.
[0051] For purposes of the following discussion a "lesson plan"
includes "lesson content" in a number of sequentially accessed
"lesson steps" that are presented on the browsers. The lesson
content includes both information and events that are designed to
enhance learning. Each lesson step presents certain information to
the student or requires certain student interactions, either
individually or within groups of students. The lesson plans or
lesson steps may tagged as be designed for teaching certain subject
areas and/or or areas or specific areas of competency, which are
termed "Macro competency" (the overarching unit), "Mini competency"
(a specific aspect of the unit), and "Nano competency" (an
actionable component of the Mini competency, and corresponds to a
"Hack" or "Heuristic," as discussed subsequently.
[0052] Lesson steps may include, for example and without
limitation, text, graphics, and/or videos, instructions for
learning activities, such as breakout groups ("BOGs"), to be
performed by students either individually or in groups, and/or
quizzes, polls, or tests, an embedded web page, such as a Wikipedia
entry, an embedded video, linking to the Internet, an embedded
image, providing a URL pointing to an image served from the
internet, a student poll, a quiz, or a shared/editable files,
referred to herein and without limitation as a "whiteboard" or
"whiteboard/documents" and which may be for example, Google Docs,
Sheets, or Slides. or other shared workspaces, including but not
limited to: a shared coding environment that allows the instructor
and students to collaborate on the same piece of software code and
test that is runs; a shared drawing tool that allowing the
instructor and students to collaborate on building diagrams, org
charts, etc.; or a shared "game," such as battleship, which is used
to illustrate topics such as strategy.
[0053] FIGS. 1 and 2 are schematic diagrams which illustrate one
embodiment of an on-line platform of the present invention as a
system 100 including a server 110 for providing programming
instructions to a plurality of electronic devices 130 over a
network 120. In one embodiment, devices 130 are wireless devices,
and network 120 includes wireless communication to the device. In
general, server 110 may produce programming instructions, files, or
data that may be transmitted over network 120 to operate devices
130. In addition, network 120 may also provide access to
instructional material which can be retrieved and displayed on
device 130.
[0054] In general, a user of device 130 may communicate over
network 120 to server 110, which includes programming to receive
and transmit information with devices 130. FIG. 2 illustrates one
embodiment of platform 100 programmed as a system of the present
invention.
[0055] Server 110 is a computer, a computer system, or network of
computers or systems that may include a network interface 111, a
memory 113, and a processor 115. Is to be understood that network
interface 111, memory 113, and processor 115 are configured such
that a program stored in the memory may be executed by the
processor to accept input and/or provide output through network
interface 111 over network 120 to devices 130.
[0056] Devices 130 may be, for example and without limitation, a
desktop or portable computer or a cellular telephone, tablet
computer, or a portable digital assistant, and includes a network
interface 131, a memory 133, a processor 135, a display 137, and an
input device 139. Network interface 131 is used by device 130 to
communication over a wireless network, such as a cellular telephone
or Wi-Fi network, and then to other telephones through a public
switched telephone network (PSTN) or to a satellite, or over the
Internet. Memory 133 includes programming required to operate
device 130 (such as an operating system or virtual machine
instructions) and may include portions that store information or
programming instructions obtained over network interface 131, or
that are input by the user (such as telephone numbers or images
from a device camera (not shown). In one embodiment display 137 is
a touch screen, providing the functions of the screen and input
device 139. Input device 139 may be a keyboard, a touchscreen, a
trackball, a mouse, a microphone or a camera which generates a
video feed and a microphone which generates an audio feed.
[0057] One use of platform 100 for facilitating of an on-line
college platform will now be described. In general, there are at
least two types of users of platform 100--instructors and students.
Thus, for example, memory 113 and or 133 is provided with software
that enables instructors to provide students with instruction, such
as lectures and quizzes, allows students to communicate with
instructors, such as by answering quizzes or asking questions, and
enable students to communicate with each other during breakout
sections and have the instructor monitor their progress.
[0058] In addition to instructors and students, platform 100 is
alternatively programmed with other interfaces, such as: a tech
support interface that may be used by technical support ("tech
support") staff to attend to technical issues during class, a
recorder interface that allows observation of a lesson without
interacting or interfering with a class; a dashboard interface for
use by coaches, admissions, tech support, instructors, and other to
check on the status of students enrolled in courses, an operations
interface for use by software developers and dev/ops personnel to
perform maintenance and upgrades, an instruction design interface
for use by instructional designers to design lesson plans, enter
quiz, midterms, final exams and poll questions, or otherwise
provide classroom content, and a school administrator interface for
use by school administrators to edit the academic calendar, enroll
students, run reports, and perform other school administrative
tasks.
[0059] The following discussion presents embodiments of platform
100 wherein server 110 and devices 130 include programs stored in
memory 113 and 133, respectively, which instruct processor 115 and
135, respectively, to communicate over the network 120, including
retrieving data stored in memory 113, and provide output on
displays, such as display 137.
System Architecture
[0060] FIG. 3 is a high-level architecture diagram of the front
end/web app detail of platform 100, which shows interactions
between one of a plurality of browser web apps 300, each operating
on one of a plurality of devices 130 and server 110. The elements
of browser web app 300 include components that permit the browser
web app to communicate over network 120, and may include, for
example and without limitation, a reactJS/Single-page-app 303, a
publish/subscribe client 304, a peer-peer video client 305, chat
306, live tech support 307, remote control 308, share
whiteboard/docs 309, screen sharing 310, metrics and monitoring
311, and a classroom videos grid 312.
[0061] Using these components, browser web app 300 accesses Content
Management System (CMS) 320, Content Delivery Network (CDN) 330,
Third-Party API Servers/Services 340, College API Servers 350,
Databases 360, Cloud Hosting Services 370,
Authentication/Authorization 380, Publish/Subscribe Service 390,
and Video Service 392, as well as other browser web apps 300 and
other computers on network 120. Unless stated explicitly herein as
being provided over the Internet or by third parties, it is
understood that the programming and databases described herein
reside on servers 110.
[0062] College API servers 350 are "stateless" servers that access
data by retrieving data in College Database 810 or by retrieving
data from calls to other services, as discussed subsequently.
[0063] CMS 320 includes College API Servers 350 and CMS Data 410
and Lesson Plans 420 which are stored in Database 360. In certain
embodiments, an API call from browser web app 300 of one device 130
are directed to a College API Servers 350 configured to retrieve
lesson content from Database 360 that is locally stored on platform
100, or may further call Third-Party API Servers/Services 340 or
Cloud Hosting Services 370 to retrieve lesson content stored
external to platform 100, such as on the Internet.
[0064] Authentication/Authorization 380 is a type of Third-Party
API Service 340 that may be used to authenticate and log users onto
platform 100.
[0065] Publish/Subscribe Service 390 is a Third-Party API Service
340 that is used for a many purposes, such as for chat sessions
between browser web apps 300, including classroom chat, BOG chat,
person to person chat, or tech support chat.
[0066] Publish/Subscribe Service 390 is also used to send commands
from a first browser web app 300 to a second browser web app. Thus,
for example and described subsequently, an Instructor Browser web
app 300A may send commands via Publish/Subscribe Service 390 to one
or more Student Browser web apps 300B that "remote control" the
Student Browser web apps and/or the device on which the browser is
running. Remote control actions include, but are not limited to one
of the following: send "current lesson step" meta data and data to
display lessons on each student's Student Browser web apps; send
"flash messages" to one or more students' Student Browser web app
to display a bright popup that covers the rest of the content on
the browser and is a "high priority" notification to the student;
refresh the Student Browser web app video streams in the event that
the student has contacted tech support with problems regarding
video streams; "refresh the user's browser," which reloads browser
web apps 300 and to clear intermittent bugs; turn students'
microphones and/or videos off for the device 130 running their
browser web app; enable/disable students' ability to turn their own
microphones on or off for the device running their browser web app;
enable/disable students' ability to "raise their hands" using the
button along the bottom of their web apps; enable/disable students'
ability to "open and use classroom chat" using the button along the
bottom of their web apps; transition the students' web apps into
different "views" and "states". For instance, at the end of class,
transition students into a quiz--where they see a UI that allows
them to answer questions; move students between BOGs; transition
students' browsers into a view/state where they see the other
members of their BOG and see the shared content their BOG is
working on; "spotlight" a BOG. This transitions everyone`s` web
apps into a state where they see a selected BOG, its content, and a
grid of the entire classroom; or spotlight a specific student.
[0067] Video Service 392 is a Third-Party API Service that provides
video chatting between different devices 130. Video Service 392 is
used to: allow the instructor and students to "enter into" and
"exit from" a virtual meeting space where each attendant is
streaming audio and video from their computing device and where
others in the space have access (can view and hear) the video and
audio streams of all others in that same meeting space; allows
platform 100 to turn individual students' audio and/or video
streams on and off; present video/audio of the instructor to all
students attending class; present video/audio of all other students
attending the class; and store and forward student metrics such as:
amount of talking, if student turned their video or audio on or
off, who is currently "connected" into the shared meeting
space.
[0068] In addition to retrieving lesson content, College API
Servers 350 may also be used to perform one or more of the
following tasks: create/drop/add users; enroll/unenroll students;
administer academic calendar, instructors; edit courses; generate
quizzes, midterms, final exams; generate BOGs content; group
students into BOGs during instruction; show user polls and how
responses during instruction; monitor student interaction with
content during BOGs during instruction; monitor student interaction
with chat during BOGs during instruction; monitor student
interaction with video/audio during BOGs during instruction;
generate reports; provide infrastructure for "operations" of
classes during classes; provide storage for all meta data used to
describe academic calendar, curriculum, roles, students, etc.;
store and report system-level (infrastructure) metrics; and store
and report user-level (school, students, instructors, quizzes,
etc.) metrics.
[0069] Cloud Hosting Services 370 may be used as follows: S3
Content Delivery Network; Deliver Web App to Browsers; Horizontal
Scalability; Vertical Scalability; Continuous Integration;
Continuous Deployment; Software Versioning; Virtual Servers;
Firewalls & Routers; Virtual Private Networks;
FERPA-Compliance; Fault Tolerance; Serverless (Functions As A
Service); SSL Certificates; Relational Databases; NO SQL Databases;
Automated Backup/Restore; Audit Trails; ElasticSearch Log
Management; and/or Logging/Monitoring/Metrics.
Browser Configurations
[0070] As noted above, platform 100 includes interfaces for
different purposes and which provide different functionalities. In
certain embodiments, platform 100 includes a number of different
browser web apps 300 that provide these interfaces and which
interact differently with one or more of Content Management System
(CMS) 320, Content Delivery Network 330, Third-Party API
Servers/Services 340, College API Servers 350, Databases 360, Cloud
Hosting Services 370, Authentication/Authorization 380,
Publish/Subscribe Service 390, and Video Service 392 differently,
as discussed subsequently.
[0071] Certain browser web apps 300 are configured to present the
classroom videos grid 312. Classroom videos grid 312 includes the
student's video feed of the students in the class--that is, the
video feed of device 130 for student's Student Browser web app
300B. In certain embodiments, classroom videos grid 312 is sized to
show a portion of all the student's video feeds at a time and
scrolls the video feeds through the classroom videos grid. If a
student's video feed cannot be displayed for some reason, such as
due to network bandwidth limitation, the space intended for a
student's video feed may be replaced with an icon or image, such as
a photograph of the student.
[0072] In certain embodiments, Browser web apps 300 may include but
is not limited to: [0073] an Instructor Browser web app 300A
intended for use by the instructor of the class and used, for
example and without limitation, to control, communicate, and
monitor students during lectures or BOGs through their Student
Browser web apps 300B. Instructor Browser web app 300A may present,
for example and without limitation, lesson steps defined by
Instruction Design web app 300G, including lecture material, BOGs,
and quizzes, classroom videos grid 312 which presents, preferably,
video feeds of each student who is logged in to their Student
Browser web app 300B, an image or video of the user of Instructor
Browser web app 300A, the ability to communication with individual
or all Student Browser web apps 300B, and other classroom widgets;
[0074] a Student Browser web app 300B intended for use by a student
to access a course both before, during, and after class. During the
presentation of a lesson plan, Student Browser web app 300B accepts
commands from Instructor Browser web app 300A that may control the
presentation lesson plan steps, and provides communications with
the Instructor Browser web app 300A and other Student Browser web
apps 300B, an image or video of the user of Instructor Browser web
app 300A, and classroom videos grid 312; [0075] a Tech
Support/Observer Browser web app 300C intended for use by tech
support, course observers, or the instructor to answer students'
technical questions during class, to move individual students
between BOGS, and to control Instructor Brower web app 300A and
Student Browser web app 300B during class; [0076] a Recorder
Browser web app 300D presents a view of the class from the
student's perspective, as in Student Browser web app 300B. Recorder
Browser web app 300D can also be used to record the class for later
viewing. [0077] a Dashboard Browser web app 300E intended for use
by coaches, admissions, tech support, instructors, and other to
check on the status of students enrolled in courses; [0078] an
Operations Browser web app 300F intended for use by software
developers and dev/ops personnel to perform maintenance and
upgrades to platform 100; [0079] an Instruction Design web app 300G
is intended for use by instructional designers to design lesson
plans 420 for each lesson, enter quiz, midterms, final exams and
poll questions, and to interact with the CMS that drives classroom
content; and [0080] a School Admin Browser web app 300H intended
for use by school administrators to "open class," which prepares a
lesson plan for teaching, edit the academic calendar, enroll
students, run reports, and perform other school administrative
tasks.
Instructor View States
[0081] Instructor Brower web app 300A provides screen 137 of device
130 with views (also referred to herein as "instructor view
states") that may be used by the instructor to run the class, and
may include, for example and without limitation, to control the
presentation of class material, to interact with students, to
monitor student progress, and to use various technologies such as
publish and subscribe technology for interaction with the students
having their own devices 130B, including remotely controlling the
student's devices.
[0082] The following discussion describes various instructor view
states, followed a list of UI widgets which may be used to generate
the various instructor view states.
[0083] Instructor view states on Instructor Brower web app 300A may
include, but is not limited to the following instructor view
states:
[0084] Before Class has Started Instructor View State: This view
states presents an icon for each of up to 200 students as each
student, through their Student Brower web app 300B, "arrive" to
class. In the lower right-hand corner of the screen the first step
of the lesson plan is displayed. Once the instructor logs in to
system 100 and "starts" class, the instructor view state changes to
the Everyone In Class (Instructor View) Instructor View State, and
the student view state (that is, the view provided on screen 127 of
the device 130 running Student Brower web app 300B) changes to the
Everyone In Class (Students View) Student View State, both of which
are discussed subsequently.
[0085] Everyone In Class (Instructor View) Instructor View State:
This view state may present, for example and without limitation: a
lesson step selector along the left side of the screen; the current
lesson step's content in the main content area; the instructor's
video feed (that is, the video feed device 130 running Instructor
Browser web app 300A) in the upper-right portion of the page; the
classroom videos grid 312 along the lower right portion of the
page; a top navigation bar along the top of the page; and a page
footer along the bottom of the page. FIG. 21 is a screenshot 2100
illustrating one embodiment of the Everyone In Class (Instructor
View) Instructor View State;
[0086] Everyone in Class With One Student Spotlit Instructor View
State: This view state is similar to the Everyone in Class view
state, except that, instead of the classroom videos grid 312, the
single student video stream for the "currently spotlit" student is
shown. Thus, for example and without limitation, an individual
student may be selected by clicking on their icon, image, or video
feed from classroom videos grid 312, resulting in the spotlit
student's video feed expanding to fill the entire space previously
occupied by classroom videos grid 312. Clicking on the spotlit
student's video feed reverses the process, reinstating the
classroom videos grid 312.
[0087] BOG Activity In Progress Instructor View State: This view
state may present, for example and without limitation: a BOGs Heat
Map widget, a Move Students Between BOGs widget, and an Embedded
BOG Widget, which shows the instructor the currently selected BOG.
FIG. 22 as a screenshot 2200 illustrating one embodiment of the BOG
Activity In Progress Instructor View State;
[0088] BOG Activity In Progress Instructor "Visiting" One Group I
Instructor View State: This view state is similar to the BOG
Activity In Progress View State and also shows the instructor's
video feed above the embedded BOG view, and to Student Browser web
apps 300B of each student of the currently selected BOG, and
provides for the instructor to speak, through their respective
devices 130, directly and privately to only the members of that
BOG.
[0089] BOG Activity In Progress Instructor Broadcasting/Intercom To
All Groups Instructor View State: This view state is similar to the
BOG Activity In Progress View State and also shows the instructor's
video feed above the embedded BOG view, which is also seen by all
the members of all BOGs, and provides for the instructor to
broadcast a message to all BOGs at the same time through their
respective devices 130.
[0090] "Spotlit" BOG Instructor View State: This view state is
displayed when a single BOG has been "spotlit" and asked to present
their work to the class, and presents the same content as the BOG
Activity In Progress View State and the classroom videos grid 312.
FIG. 28 as a screenshot 2800 illustrating one embodiment of the
"Spotlit" BOG Instructor View State;
[0091] Poll in Progress Instructor View State: This view state
differs from the Everyone In Class View State in that the main
content area shows a poll question that the instructor broadcasts
to Student Browser web apps 300B.
[0092] Showing Poll Results Instructor View State: This view state
differs from the Everyone In Class View State in that the main
content area shows the results of a poll question that all the
students have answered. Poll results can be shown in real time,
changing as more students respond.
[0093] Quiz In Progress Instructor View State This view state may
present, for example and without limitation: a message that a quiz
is in progress, the instructor's video feed and the classroom
videos grid 312, and allows the instructor to watch all the
students during the quiz to ensure that the students are not
cheating.
[0094] UI widgets which may be used to generate the various
instructor view states include, but are is not limited to the
following UI widgets:
[0095] Lesson Step Selector Widget: This widget runs the height of
the browser web app 300 and is docked along the left side. It
allows the instructor to control what image or video or poll or BOG
or other content the students are presented on Student Browser web
apps 300B. The instructor may operate Instructor Browser web app
300A to advance the class step-to-step or to a specific lesson plan
step using a "lesson step selector" widget along the left-hand side
of the screen.
[0096] Main Content Widget. This widget is in the center of the
screen and displays the currently selected lesson step's content,
which may be, for example and without limitation, an image, a
video, a PDF file, an editable shared document, or HTML/text.
[0097] Instructor Video Feed Widget: This widget is a large, square
region of the screen where the instructor's video feed is
displayed. This feed is provided to Instructor Browser web app 300A
and Student Browser web app 300B so that it may seen by both the
instructor and by students.
[0098] Classroom Videos Grid Widget: This widget is a grid
(1.times.1, 2.times.2, 3.times.3, or 4.times.4) of video feeds of
all the students currently attending class. The video feeds scroll
horizontally (either right to left or right to left) so that even
if there are as many as 200 students currently logged in, only 16
video feeds (at most) can be seen at the same time. The student can
adjust the size of the matrix, which allows them to cope with poor
bandwidth or using an older, slower computer.
[0099] User Poll (Instructor View) Widget. This widget presents a
view of the poll question that the instructor has asked all
students to answer.
[0100] User Poll Results Widget: This widget is a bar chart showing
an aggregated view of the answers that, by the operation of the
Student Browser web app 300B, the students have provided to a poll.
This view can change in real time as more students respond
[0101] "Remote Control" Video (Instructor View) Widget: This widget
allows the instructor to control the play, pause, seek of a video.
When the instructor uses this widget to start a video or pause a
video or seek to a specific play position within a video, the
changes the instructor makes are broadcast to all the students'
views of the video. This allows the instructor to control the
playback of the video.
[0102] Embedded Web Content Widget: This widget shows an HTML
"embed" of content using an HTML "iframe." This widget is used to
display videos, web pages, audio streams and other types of content
that can be embedded into a lesson step.
[0103] Image View Widget: This widget displays an image that is
hosted on a CDN such as: Amazon Cloud Front/S3, Akami, CloudFlare,
or any other web-based content delivery network.
[0104] Embedded BOG View Widget: This widget displays the
video/audio streams and the shared document/whiteboard of the
students in a single BOG. This is used during BOGs as well as
during "BOG spotlight"--when a single BOG is presenting their work
to the entire class.
[0105] BOGs Heat Map Widget: This widget is displayed when the
system is in "breakout activity" mode and all of the students are
grouped into small, private "BOGs" where they can collaborate. The
BOGs Heat Map Widget displays a visual display of the status of
each group (different colored rectangles show groups that are
highly active, not sufficiently active, too active), etc. The
specific measures of "activity" in each BOG include, but are not
limited to, the following measures of activity as determined from
video, audio, and typing from each of the Student Browser web app
300B in the BOG: how much talking is taking place; how much
activity is taking place in the shared work product; how much use
is being made of Group Chat; the average "initiative" level
(determined by how many times the members have previously raised
their hands during class); the average amount of talking; the
variance in talking, indicating how evenly talking is distributed
across group members; the variance in amount of times cursors are
engaged in the shared work product, indicating how evenly work
production is distributed across group members; ratios consisting
of the person with the greatest activity being divided by the
average of the activity of all other members (which provides a
measure of whether one person is dominating). The visual display in
the BOGs Heat Map Widget of the status of each group is a geometric
shape that is color coded according to the activity in the BOG.
Examples of the visual display, which are not meant to limit the
scope of this patent, are discussed for example and without
limitation, in the discussion of FIG. 15, and are shown in FIG.
22.
[0106] The heat map is interactive and operates as a "channel
changer." Thus, for example, when the instructor clicks on the
geometric shape corresponding to a specific BOG, the instructor
will enter the BOG (and then clicks again to leave that group, at
will). Further, when the instructor joins a BOG, the instructor may
participate as a full member of the group--conversing with students
and having access to any shared document/whiteboard. The instructor
may also operate Instructor Browser web app 300A to "pin" a group,
allowing the instructor to revisit the group at a later time.
[0107] Move Students Between BOGs Widget: This widget is displayed
at the same time that the BOGs Heat Map is displayed. This widget
allows the instructor to move students between BOGs after those
BOGs have been created and are in progress.
[0108] Students Tool Widget: This widget is a popup window that
allows the instructor to see a list of all students who have logged
into class, the status of their video and audio (did the student
turn these off?), how long the student has been attending class,
etc. The instructor can use this tool to turn off a single
student's video and/or audio, refresh their browser (this sometimes
helps when the student encounters a bug with Student Browser web
app 300B), send the student a private "flash message" that shows up
in front of all of the content on their screen, and remotely log a
student out if that student is being unruly in class.
[0109] Top Navigation Widget: This widget is a bar along the top of
the screen. It contains the name of the currently taught class, the
current step number, and a drop-down menu full of tools the
instructor can use to control certain aspects of class.
[0110] Page Footer Widget: This widget resides along the bottom of
the web application. It allows the instructor to turn on and off
their own camera/feed and/or microphone/audio. It also allows the
instructor to turn on or off all students' microphones, turn on or
off all students' videos, lower all students' hands, enable/disable
students' ability to raise their hands, and enable/disable
students' ability to access the classroom chat stream.
[0111] Quiz In Progress View Widget: This widget is a status
message view that lets the instructor know that a quiz is in
progress and all the students are currently answering the quiz.
[0112] The instructor may operate Instructor Browser web app 300A
change/select view states on the Instructor Browser web app 300A
and on one or more Student Browser web app 300B by clicking on
various pieces of the Instructor Browser web app 300A user
interface, such as:
[0113] Clicking on the "start BOGs" button in the lesson step
chooser changes the instructor's view state to one that shows the
embedded/selected BOG, the students mover tool, and the heat map.
In addition, this "remotely" causes all students' browsers to be
transitioned to a view state that shows them in a BOG.
[0114] Clicking a "poll" button in the lesson step chooser changes
the instructor's view state to one that shows a poll question that
the students will answer. In addition, this "remotely" causes all
students' browsers to be transitioned to a view state that shows a
question that they can answer.
[0115] Clicking a "quiz" button in the lesson step chooser changes
the instructor's view state to one that shows a "quiz in progress"
message as well as the classroom videos grid 312 of students taking
the quiz. In addition, this "remotely" causes all students'
browsers to be transitioned to a view state that shows a quiz
view--allowing them to take a quiz.
[0116] Clicking a lesson step from the lesson step chooser along
the side of the UI shows a "viewer" for that type of lesson step
(for instance, an image or a video or an embedded web site or some
text, etc.) In addition, this "remotely" causes all students'
browsers to be transitioned to a view state that that lesson step
in the appropriate type of viewer.
Student View States
[0117] Student Brower web app 300B provides screen 137 of device
130 with views (also referred to herein as "student view states")
that may be used by the students before, during, or after a lesson.
The following discussion describes various student view states,
followed a list of UI widgets which may be used to generate the
various student view states.
[0118] Student view states may include, but is not limited to the
following student view states:
[0119] Before Class has Started Student View State: This view state
shows an icon for each student as they "arrive" to class. The
student icons are randomly distributed around the screen. In the
lower right-hand corner of the screen the first step in the lesson
plan is displayed. Once the instructor logs in and "starts" class,
the instructor view state and student view state transitions to the
Everyone In Class view state.
[0120] Everyone In Class (Students View) Student View State: This
view state shows: the current lesson step's content in the main
content area; the instructor's video feed in a square in the
upper-right portion of the page; the classroom videos grid 312
along the lower right portion of the page; a top navigation bar
along the top of the page; and a page footer along the bottom of
the page. FIG. 23 is a screenshot 2300 illustrating one embodiment
of the Everyone In Class (Students View) Student View State, and
FIG. 24 is a screenshot 2400 illustrating one embodiment of the
Everyone In Class (Students View) Student View State during a
classroom discussion;
[0121] Everyone in Class With One Student Spotlit Student View
State: This view state is the same as the Everyone in Class state
except that instead of an entire grid of many students' videos
showing in the lower right area, the single video stream for the
"currently spotlit" student is showing.
[0122] BOG Activity In Progress Student View State: This view state
shows all of the video feeds of the other members of the BOG in
which the student has been grouped. He/she can also see one or more
tabs of content that are either read-only or editable. These tabs
are what allow the members of the BOG to access the shared
content/whiteboard that is an important feature of the BOG
activity.
[0123] BOG Activity In Progress Instructor "Visiting" One Group
Student View State: This view state is similar to the BOG Activity
In Progress View State but also shows the instructor's video feed
above the embedded BOG view. This view state also allows the
instructor to speak directly and privately to only the members of
that BOG through their respective browser web apps 300.
[0124] BOG Activity In Progress Instructor Broadcasting/Intercom To
All Groups Student View State: This view state is similar to the
BOG Activity In Progress View State, and also shows the
instructor's video feed in a circle that sits above the embedded
BOG view. This view state also allows the instructor to broadcast a
message to all BOGs at the same time. FIG. 26 is a screenshot 2600
illustrating one embodiment of BOG Activity In Progress Instructor
Broadcasting/Intercom To All Groups Student View State;
[0125] "Spotlit" BOG Student View State: This view state is similar
to the BOG Activity In Progress View State, and also shows the
entire classroom videos grid 312. This view state is displayed when
a single BOG has been "spotlit" and asked to present their work to
the class. FIG. 27 is a screenshot 2700 illustrating one embodiment
of the "Spotlit" BOG Student View State;
[0126] Poll in Progress Student View State: This view state differs
from the Everyone In Class View State in that in that the main
content area shows a poll question that the instructor wants all
the students to answer.
[0127] Showing Poll Results Student View State: This view state
differs from the Everyone In Class View State in that the main
content area shows the results of a poll question that all the
students have answered. Poll results can be published in real time,
changing as more students respond. FIG. 25 is a screenshot 2500
illustrating one embodiment of the Showing Poll Results Student
View State;
[0128] Quiz In Progress Student View State: This view state shows
the student the Quiz Widget and nothing else. All of the other
pieces of the UI that the student saw during class (such as the
instructor video, the classroom videos grid, and the main content)
are no longer visible on the screen.
[0129] UI widgets which may be used to generate the various student
view states include, but are is not limited to the following UI
widgets:
[0130] Main Content Widget: This widget is described above with
reference to the instructor view state widgets.
[0131] Instructor Video Feed Widget: This widget is a large, square
region of the screen where the instructor's video feed is
displayed.
[0132] Classroom Videos Grid Widget: This widget is described above
with reference to the instructor view state widgets.
[0133] User Poll (Student View) Widget: This widget is a view of
the poll question that the instructor has asked all students to
answer. By the operation of the Student Browser web app 300B,
students can select and then submit their answers to a poll using
this widget.
[0134] User Poll Results Widget: This widget is described above
with reference to the instructor view state widgets.
[0135] "Remote Control" Video (Student View) Widget: This widget
that allows the instructor to control the play, pause, seek of a
video. When the instructor uses this widget to start a video or
pause a video or seek to a specific play position within a video,
the changes the instructor makes are broadcast to all the students'
views of the video, allowing the instructor to control the playback
of the video. The students can watch the video via this widget but
cannot control its playback themselves.
[0136] Embedded Web Content Widget: This widget is described above
with reference to the instructor view state widgets.
[0137] Image View Widget: This widget is described above with
reference to the instructor view state widgets.
[0138] Embedded BOG View Widget: This widget is described above
with reference to the instructor view state widgets.
[0139] Top Navigation Widget: This widget presents a bar along the
top of the screen. It contains the name of the currently taught
class, the current step number, and a drop-down menu full of tools
the student can use to fine-tune the way their UI looks during
class.
[0140] Page Footer Widget: This widget allows the student to turn
their microphone and video on or off. There are times during class
when the instructor disables these buttons. There are other times
during class when the instructor "turns off" the students' videos
and/or audio feeds. This widget also has a "raise/lower" hand tool.
When a student raises their hand using this tool, it causes their
classroom videos grid 312 as well as the classroom videos grids of
all other students and the instructor to update and show a "hand"
in front of the student's video along with an orange border
surrounding their icon. This allows everyone to know that the
student is raising their hand. The instructor can then "call on"
this student by clicking on them in the classroom videos grid, at
which point their image expands to fill the area previously
occupied by the entire grid. The student also uses the page footer
widget to open and close the classroom chat, BOG chat, and
technical support chat.
[0141] Quiz Widget: This widget allows the student to answer the 6
or more questions in a quiz. Each question is displayed on a page
of the screen. Once the student answers the question, they
immediately receive the correct answer if they make a mistake,
along with a brief explanation as to why that is the correct
answer. They then are transitioned to the next question in the
quiz. Once the student has answered all questions, they are
transitioned to a final page that shows their total quiz
results.
[0142] The student may changes/selects view states on their Student
Browser web app 300B by clicking on various pieces of the Student
Browser web app in a manner similar to that described regarding the
Instructor Browser web app 300A.
Technical Support View States
[0143] Technical Support Brower web app 300C provides screen 137 of
device 130 with views (also referred to herein as "technical
support view states") that may be used by the IT personnel to
provide technical support. Technical support view states are
identical to the instructor view states, except that the "lesson
step selector" in the technical support view is "read only"--it
shows what the instructor has selected but does not allow the
technical support user to choose which step of a lesson all of the
students are viewing.
Recorder View States
[0144] Recorder Brower web app 300E provides screen 137 of device
130 with views (also referred to herein as "recorder view states")
that may be used to "observe" a class. The recorder view states are
identical to the student view state, with the exception that a
non-student can log into device 130A to the recorder view in order
to "observe" the class from a student's perspective. The recorder
view is used by platform 100 to record classes. The recording
software brings up a browser with the recorder view and then
"records" all screen and the audio during a class. This results in
a recording of an entire class from the student's perspective. This
recording can then be provided to students--allowing them to review
classes they missed or that they had attended previously.
Instruction Design View States
[0145] Instruction Design web app 300G provides screen 137 of
device 130 with views (also referred to herein as "instructor view
states") that may be used to design lesson plans 420 for each
lesson, enter quiz, midterms, final exams and poll questions, and
to interact with the CMS that drives classroom content.
[0146] Instruction Design web app 300G allows an instructional
designer to choose a course and a specific lesson plan by
viewing/editing/creating/deleting lesson plan steps that compose
the lesson plan using the Instruction Design web app. The following
discussion presents examples of the functioning of the Instruction
Design web app 300G.
[0147] Each lesson plan step has a lesson step type. When a lesson
plan step is added to a lesson plan in the Instruction Design web
app 300G, the UI prompts the user as to the lesson step type, such
as an image, a video, a BOG, a poll, etc., and presents the
appropriate type of "lesson plan step editor" based on the lesson
step type in Instruction Design web app 300G.
[0148] Instruction Design web app 300G permits the user to:
drag-and-drop lesson plan steps to rearrange where/when they occur
during a lesson; enter all of the questions for a quiz, mid term,
or final exam; edit the properties of a BOG activity, where the BOG
activity properties may include, but are not limited to: the number
of students to be included in the BOG; the time allocated for the
group; assign (or not) a role to each BOG user; rules on how to
order or group students so that they are grouped with the
appropriate members; the number and types of groups will be formed
for the BOG activity (See, for example, FIGS. 14 and 15); specify
the shared editable documents and/or read-only documents, images,
and videos are to be used by the BOG, including which documents may
be used by students depending on their role.
Generation, Storage, and Display of Lesson Plans
[0149] The generation of lesson content on platform 100 and the
presentation of lesson content to Student Browser web apps 300B is
illustrated in FIGS. 4-7, where FIG. 4 is a general schematic of
Content Management System (CMS) 320, FIG. 5 is a schematic of an
instructional design view of the CMS, FIG. 6 illustrates a live
classroom view of the CMS for presenting lesson content, and FIG. 7
is a schematic of College API Servers 350.
[0150] In certain embodiments, an instructor produces
specifications for the lesson content, an instructional designer
uses Instruction Design web app 300G to generate or locate the
content for use by platform 100, and the lesson plan may then be
accesses by an instructor using Instructor Brower web app 300A and
each students using Student Browser web app 300B.
[0151] FIG. 4 illustrates that a lesson designer may use
Instruction Design web app 300G to store documents, videos, and
images comprising CMS Data 410 and Lesson Plans 420 in Database
360. Specifically, the lesson content stored in CMS Data 410
includes text, graphics, or videos to be presented, which may be
the actual contents or a link to content accessible over and Lesson
Plans 420 includes specifications of each of the lesson steps,
including metadata of data stored as CMS Data 410.
[0152] In certain embodiments, Instruction Design web app 300G is
programmed to accept an input file and to identify and work with
four different types of events: 1) content shown directly to the
students (e.g., slides in a standard PPT format or videos); 2)
materials sent to BOGs (with different materials being sent to
different groups, if so specified); 3) polls (which can be
pre-determined or configured on-the-fly, with the results shown to
the students as a bar graph); and 4) Quiz questions which may be,
but are not limited to: multiple choice (including simple
True/False); "Mark all that apply" multiple answers; instructions
to match entries in one column with those in another column.
[0153] FIG. 8 is a schematic of the structure of Database 360,
which includes data accessible from a server platform 100 and data
accessible over an external network, such as the Internet. Data on
platform 100 includes CMS Data 410 and Database 810 which may
include User Roles 811 (such as student, instructor, etc.), Lesson
Plans 420, an academic calendar 813, Questions, Quizzes, and Polls
and Results 550, a List 815 of all users such as students,
instructors, admins, designers, etc., BOG Configurations 540,
Student Metrics 817, and Course Attendance and Online Status for
Each Student 819. Data accessible over an external network includes
Live Video Sessions 801, Live Sessions and Messages 803, Shared
Whiteboard/Documents, Users and Metrics 805, and Authentication
Data 710.
[0154] As illustrated in FIG. 8, Databases 360 are accessed by
calls to College API Servers 350, which access data stored on
platform 100, and which further calls Publish/Subscribe Services
390 for accessing Live Sessions and Messages 803, Video Services
392 for accessing Live Video Sessions 801, and Cloud Hosting
Services 370 to access Shared Whiteboard/Documents 805, when
needed.
[0155] With CMS Data 410 and Lesson Plans 420 stored in Database
360, users of Instructor Browser web app 300A and Student Browser
web app 300B may access the lesson content via College API Servers
350. Thus, for example and as noted above, Instructor Brower web
app 300A includes a lesson step selector which allows the
instructor to control the presentation of lesson content to the
Student Browser web apps 300B.
[0156] In addition to CMS Data 410, which generally includes
documents, videos, or images that are stored locally on platform
100, lesson content may include more complex and/or interactive
content such as learning activity specifications or content,
defined for example, in BOG Definitions, and quiz, poll, or tests
and results, which are also stored on platform 100, and external
media, such as text, images, or videos that are stored externally,
such as on the Internet.
[0157] FIG. 5 illustrates one embodiment of the generation of
lesson content, which includes CMS Data 410 and Lesson Plans 420,
Shared Whiteboard/Documents, Users and Metrics 805, Embed codes
520, web content embed codes and links 540, BOG Definitions 540 and
Questions, Quizzes, and Polls 560, which are stored in Database 360
as shown in FIG. 8. In addition, pointers to the Shared
Whiteboard/Documents are stored as Whiteboard/Documents in 510 of
College Database 810.
[0158] Thus, for example, the instructional designer operates
Instruction Design web app 300G to define the lesson steps
comprising Lesson Plans 420, locate or input and upload lesson
content of each lesson step, which is stored as CMS Data 410,
configure and store Shared Whiteboard/Documents, Users and Metrics
805, obtain and store embed codes 530 for videos hosted on third
party sites, obtain and store web content embed codes and links
530, generate and store BOG Definitions 540 and generate and store
Questions, Quizzes, and Polls 560, both of which are stored in
College Database 810. During the storing of CMS Data 410,
Instruction Design web app 300G stores CMS Data metadata for each
lesson step in Lesson Plans 420.
[0159] BOG Definitions 540 may include, but are not limited to: how
many students will be first be included in the groups, how much
time is allotted, assigned roles (or not) to each BOG user, rules
on how to order or group students so that they are grouped with the
appropriate members, how many steps there are in the breakout
activity, and the rules for forming groups for those steps; for
each step in the breakout activity, a specification of shared
editable documents and/or read-only documents, images, videos, they
should see during the group, and allotted time for each step; and
specification of which students, based on their role, can access
which documents.
[0160] Shared Whiteboard/Documents, Users and Metrics 805 includes
template and cloned copies of Whiteboard/Documents for the students
to share, permissions for students to access the
Whiteboard/Documents, and metrics of student use of the shared
Whiteboard/Document.
[0161] The presentation of lesson content to Student Browser web
apps 300B is illustrated with reference to FIG. 6. During the
presentation of a lesson plan, the instructor, through Instructor
Browser web app 300A sequences through the lesson step. For
example, as described above as one embodiment, above, the Everyone
In Class (Instructor View) View State of Instructor Browser web app
300A includes the lesson step selector. This selector allows the
instructor to sequence through the lesson steps of Lesson Plans
420. Lesson Plans 420 includes the sequence of lesson steps and
metadata for the content of each step.
[0162] When sequencing to the next step, platform 100 reads the
specification of the next lesson step from Lesson Plans 420 and,
based on the specification, such as metadata, calls College API
Servers 350 to retrieve the required content from CMS Data 420, to
retrieve interactive Shared Whiteboard/Documents, Users and Metrics
805, to retrieve BOG Definitions 540, and/or to retrieve Questions,
Quizzes, and Polls 560, and by calling Third-Party API
Servers/Services to retrieve content from embed codes 520, and to
retrieve content from web content embed codes and links 530, when
required. College API Servers 350 then call Publish/Subscribe
Service 390, which sends the retrieved content to the Instructor
Browser web app 300A and each Student Browser web app 300B, and
instructs the browser web apps to change their view state to show
the retrieved content.
[0163] FIG. 7 is a schematic that illustrates the use of College
API Server 350 and a Load Balancer 701. College API Server 350 may
consist of several identical codes running on one of multiple API
servers. Load balancer 701 intercepts calls from browser web apps
300 to the College Server API and determines which specific College
API Server 350 will handle the call. This ensures that each server
does a portion of the work and no single server is doing all the
work of all the College API calls. Load balancer 701 also tracks
which API servers are operating, thus ensuring that platform 100
will continue to operate if a server crashes.
Student Metrics
[0164] One important aspect of platform 100 is tracking student
metrics, which are stored as Student Metrics 817. Student metrics
related to whiteboard use are stored in Shared
Whiteboard/Documents, Users and Metrics 805.
[0165] Student metrics may be determined from the student's current
or previous use of their Student Browser web app 300B, examples of
which include, but are not limited to: [0166] the student's answer
to a question or set of related questions prior to forming one or
more BOGs, such as the answer to a quiz question or set of related
questions about a lecture that was just delivered; answers either
to a very specific quiz administered immediately before the BOG or
to a set of previous quizzes, where the answers are coded by
learning outcome and stored in a data base; responses to poll
questions; and chat entries; [0167] the student's interaction
activity with the on-line class prior to forming the one or more
BOGs, as determined by their asking questions during a lecture; the
amount of keyboard or microphone activity; responses to polls. and
chat entries; [0168] a measure of the student's interaction with
other students of the plurality of students determined from one or
more previous breakout sections, where the measure is: obtained
from audio interactions with other members of the previous BOG or
groups; determined from interactions in a written document with
other members of the previous BOG or groups; or is determined from
what specific combinations of students in previous BOGs resulted in
a large amount of group interaction; [0169] one or more performance
capabilities assigned to each student based on prior performance on
specific quiz items or other assessment measures that are coded to
assess a specific capability (i.e., skill or aspect of knowledge),
assign a measure of one or more capabilities to each student.
Performance capabilities may include, but are not limited to,
specific aspects of: analytical skills; critical thinking skills;
self-management skills; personal interaction skills; learning
skills; problem solving skills; and/or communication skills; [0170]
the student's interaction activity with the on-line class after
forming the one or more BOGs as determined from the student's
microphone activity; whiteboard activity; or cursor activity in
shared documents; and [0171] the roles played by students in
BOGs.
Use of Publish/Subscribe
[0172] Publish/subscribe services 390 may be used in platform 100
to perform a variety of tasks, such using an Instructor Browser web
app 300A to direct lesson content to Student Browser web app 300B,
to permit video or text chat between various browser web apps 300,
for conducting interactive learning activities, for controlling
another device 130 remotely, as described above in the discussion
of FIG. 3.
[0173] FIG. 9A is a schematic that illustrates how
publish/subscribe services are used in the classroom setting.
First, an instructor sequences to the next lesson step in Lesson
Plan 420 using the lesson step selector on Instructor Browser web
app 300A. This results in Instructor Browser web app 300A calling
College API Server 350, which retrieves the content of the next
lesson step and a list of students in the class stored in Course
Attendance and Online Status for Each Student 819.
[0174] In the next step, College API Server 350 calls
Publish/Subscribe Service 390 and provides the service with the
retrieved lesson content and list of students.
[0175] In next, Publish/Subscribe Service 390 provides the Student
Browser web app 300B for each student in the retrieved list of
students with the content of the next lesson step.
[0176] FIG. 9B is a schematic that illustrates how
publish/subscribe services control student's behavior and views.
First, using Tech Support Browser web app 300C, tech support
personnel send a call to College API Server 350, examples of which
include but are not limited to: refresh browsers; flash a message
to one student or an entire class; move a student between BOGs; set
microphones on or off; turn videos off; or force the log off of a
students.
[0177] In the next step, depending on the call, College API Server
350 updates information in College Database 810, such as lesson and
student attendance 819 or BOG configuration 540.
[0178] Next, College API Server 350 calls Publish/Subscribe Service
390, which then sends the appropriate message to Student Browser
web apps 300B which may be, for the above examples: refreshing
browsers; flashing a message to one student or to the entire class;
move students between BOGs; turn microphones on or off; videos off,
or force a log out.
[0179] In the final step, Publish/Subscribe Service 390 provides
the Student Browser web app 300B for each student in the retrieved
list of students with the content of the next lesson step.
[0180] Platform 100 also provides for communication between those
using the platform is facilitated by having several types of chat.
Classroom chat occurs between Instructor Browser web app 300A and
Student Browser web app 300B. Classroom chat is initiated from
either Instructor Browser web app 300A or Student Browser web app
300B, and can be viewed from the Instructor Browser web app and all
Student Browser web apps. Classroom chat stream is automatically
organized thematically, with comments that are responding to the
same message are grouped together.
[0181] Help chat occurs between Tech Support Browser web app 300C
and Instructor Browser web app 300A can only be seen by help desk
personnel and teaching assistants. Help chat also includes controls
for the viewing mode, including but not limited to: the size of the
student matrix and whether scrolling is enabled.
[0182] BOG chat occurs between the students using Student Browser
web app 300B that are within a BOG, and can be monitored by the
instructor with Instructor Browser web app 300A.
[0183] Person to person chat occurs between Tech support and the
student, between Tech support and the instructor, or between the
Instructor and Student.
[0184] Tech support chat may be used by Instructor Browser web app
300A to "remote control" one or more Student Browser web apps 300B.
Remote control actions include, but are not limited to one of the
following: send "current lesson step" meta data and data to each
student; send "flash messages" to one or more students--this is a
bright popup that covers the rest of the content and is a "high
priority" notification to the user; refresh the user's video
streams in the event that the user has contacted tech support with
problems regarding video streams; "refresh the user's
browser"--which reloads the browser web app 300 and can clear away
intermittent bugs; turn students' microphones and/or videos off;
enable/disable students' ability to turn their own microphones on
or off; enable/disable students' ability to "raise their hands"
using the button along the bottom of their web apps; enable/disable
students' ability to "open and use classroom chat" using the button
along the bottom of their web apps; transition the students' web
apps into different "views" and "states". For instance, at the end
of class, transition students into a quiz--where they see a UI that
allows them to answer questions; move students between BOGs;
transition students' browsers into a view/state where they see the
other members of their BOG and see the shared content their BOG is
working on; "spotlight" a BOG. This transitions everyone`s` web
apps into a state where they see a selected BOG, its content, and a
grid of the entire classroom; or spotlight a specific student.
[0185] FIG. 10 is a schematic that illustrates how
publish/subscribe services are used for chatting. First, a student
interacts with a chat widget in Student Browser web app 300B to
pose a question to the class. This results in Student Browser web
app 300B sending the text to College API Server 350, which stores
the messages in Database 810 and retrieves the list of students
stored in Course Attendance and Online Status for Each Student 819
and chat message for each lesson 1001 from Database 810.
[0186] In the next step, College API Server 350 calls
Publish/Subscribe Service 390 and provides the service with the
retrieved lesson content and list of students. Publish/Subscribe
Service 390 stores the message in Send/Receive Messages 803.
[0187] In the final step, Publish/Subscribe Service 390 provides
the Instructor Browser web app 300A and Student Browser web app
300B for each student in in the retrieved list of students with the
messages, which is displayed within the web app.
Instruction Design
[0188] An instructional designer operates Instruction Design web
app 300G to design Lesson Plan 420 as described above with
reference to FIG. 5. Designing a lesson plan may include, for
example and without limitation, uploading slide images for a lesson
step, designing polls and quizzes, editing the
description/meta-data of one more BOG activities, where each
activity may have one or more steps, and making a reference to a
URL on the internet that points to: a static image, a static PDF, a
video, or an interactive shared whiteboard/document), and creating
the templates and rules for shared BOG workspaces. To design a
poll, a "poll designer" page is used to design a single-question
poll. To design a quiz, Instruction Design web app 300G presents a
"quiz designer" page that is used to create the quiz. For both
quizzes and polls, a "question editor" widget for each question is
used to create and/or edit the question.
[0189] FIG. 11 is a schematic that illustrates how
publish/subscribe services are used for learning activities. First,
an instructional designer operates Instruction Design web app 300G
to define a BOG of Lesson Plans 420 by configuring templates of
whiteboard/documents and user permissions for a BOG. The
whiteboard/documents templates are stored in Shared
Whiteboard/Documents 805. The rules for forming BOGs are stored in
BOG Definitions 540 and pointers to the Shared Whiteboard/Documents
are stored in Whiteboard Templates 510. During class the
whiteboard/documents templates are cloned and stored as Cloned
Whiteboard/Documents in Shared Whiteboard/Documents 805, with
pointer stored in in Shared Whiteboard/Documents 110 of College
Database 810. Using Student Browser web app 300B, and according to
the BOG definitions, students are directed to their BOG's Cloned
Whiteboard/Documents in Shared Whiteboard/Documents 805.
Performance Assessment
[0190] Active learning is enhanced in platform 100 by determining
how well the students comprehend and remember what is being taught
in each lesson. Performance assessments may include, but are not
limited to, polls conducted at regular intervals (such as one every
15 minutes or less), polls conducted at key moments (e.g., after a
specific concept has been conveyed in lecture), by quizzes (both
embedded midstream during a class and at the end of class),
midterms, and final exams. Quizzes, polls, midterms and final exams
are each a lesson step of Lesson Plan 420.
[0191] In certain embodiments, each student must have perfect quiz
scores, after taking a version of the quiz (each with different
specific questions) up to a criterion number of times (e.g., six).
To facilitate this, system 100 monitors the student's responses and
forms one or more questions in the quiz to help achieve this goal.
Thus, for example and without limitation, during a quiz lesson
step, system 100 presents a mixture of questions that are
previously unseen by the student and questions that were previously
answered incorrectly.
[0192] FIG. 12 is a schematic illustrating the creation of quizzes,
midterms, or final exams (referred to herein and without limitation
as "quizzes"), and showing a quiz that is presented in Student
Browser web app 300B, Quiz Creation Logic 1200, and, as stored in
Questions, Quizzes, and Polls and Results 550 of Database 810, data
including a Per-Course Quiz Question Bank and Previously Taken
Quizzes and Questions for Each Student, and, for each question, one
or more associated lesson plans 420 and one or more associated Nano
competency/subject area.
[0193] The order of preference in selecting questions by Quiz
Creation Logic 1200 is, first, questions that have not been
presented to the student before, followed by questions previously
answered incorrectly by the student, followed by questions
previously answered correctly by the student.
[0194] Quiz Creation Logic, which is programmed into platform 100,
includes 3 steps. In the first step, In Step 1201 database 550 is
queried to determine the type of quiz: an end-of-lesson quiz, a
midterm, or a final exam and how many questions should exist in the
quiz based on each Nano competency/subject area being tested by the
quiz.
[0195] In Step 1203, database 819 is queried to determine whether a
student is currently enrolled in the class.
[0196] In Step 1205, questions are selected for each student based
on the student's performance and the type of quiz. The questions
are selected from a question bank of questions from the current
lesson plan 420 and prior lesson plans, as stored in database 550,
as follows: [0197] If the "quiz" is an end-of-lesson quiz, then 5
questions are selected corresponding to the current lesson plan and
1 question is chosen randomly from any prior lesson plan. [0198] If
the "quiz" is a midterm, then 2 questions are selected for each
Nano competency/subject area of the lesson plan. [0199] If the
"quiz" is a makeup midterm, then the selected questions are all
questions that the student answered incorrectly in the previously
taken midterm. [0200] If the "quiz" is a final exam, then the
selected questions are 2 questions for each Nano competency/subject
area of the lesson plan and questions in the Nano
competency/subject area of the lesson plan that the student
answered incorrectly in the previously taken midterm [0201] If the
"quiz" is a "makeup" final exam, then then the selected questions
are all questions that the student answered incorrectly in the
previously taken final exam.
[0202] With each student's quiz questions 1220 thus determined, CMS
320 provides the determined quiz questions to the each students
Student Browser web app 300B.
[0203] After the student as answered the quiz questions, platform
100 grades the quizzes and stores each student's results in
Questions, Quizzes, and Polls and Results 550.
Learning Activity Groups
[0204] In certain embodiments, a lesson step includes forming
groups of students that work together on an activity. One such
learning activity is a BOG, in which each student is provided with
instructions for the learning activity and shared space, such as a
document, to which each student may contribute.
[0205] FIG. 13 is a schematic which illustrates one embodiment for
how system 100 forms BOGs during a lesson, and based on the BOG
definitions using Instruction Design web app 300G. BOGs are formed
during while executing the lesson plan according to Group Creation
Logic 1300, which has access to a list of students 1320 in the
course, which is obtained from Course Attendance and Online Status
for Each Student 819, shared Whiteboard/Documents templates 805,
student quiz scores 550 and the BOG configurations 540.
[0206] In certain embodiments, Group Creation Logic 1300 includes
step 1301, where the type of group to be created is determined.
Examples of how group are arranged may include, but is not limited
to, grouping by adjacent ranking, scores, or performance
capabilities. In addition, the type of BOG may have predetermined
performance capability or capabilities or roles for participants in
the BOG for the task to be used in the BOG to which the student is
assigned, which is also stored in BOG configurations 540, and the
grouping takes the BOG requirements into account to achieve a
balance of students in each group.
[0207] Next, in Step 1303, students 1320 are ranked, sorted, or
ordered according to the type of groups. Thus, for example, there
may be no predetermined or preferred ordering of students, and the
students may be "ranked" by a random order or in alphabetical
order. Alternatively, the type of group may require ranking or
sorting according to Student Metrics 817 which may include but is
not limited to student's answers to questions, interaction activity
prior or during class, interaction with other students, performance
capabilities, and/or student roles.
[0208] In certain embodiments, groups may be formed of students
automatically based on their prior performance on the capabilities
required for the task. Thus, for tasks where at least one person
must have previously mastered at least one relevant capability,
students may be assigned so that at least one such person is
present in every BOG. For tasks where the group is engaged in tasks
(e.g., problem solving, role playing, debate) where comparable
levels of the relevant capabilities are desirable, so that students
are not intimidated or looking down at other members of the group,
students are assigned with comparable levels of prior performance
of the task-relevant capabilities.
[0209] Next, in Step 1305, BOGs 1310 are formed into individual
BOGs 1311, 1312, 1313 . . . based on the creation criteria and the
ranking of students. And lastly, in Step 1307, each student within
each group is provided, through their Student Browser web app 300B,
with a shared work environment for their group, which may include a
shared whiteboard, as discussed herein.
[0210] FIG. 14 illustrates forming BOGs with students having
similar ranking, such as by the student metric of quiz scores.
Students 1320 are arranged in order of increasing quiz scores, and
are formed into BOGs 1410 comprising individual groups, shown
illustratively as BOGs 1411, 1413, 1415, and 1417. At future time,
the BOGs may be formed into larger BOGs 1420, as a BOG 1421
consisting of the students of BOG 1411 and 1413, and a BOG 1423
consisting of the students of BOG 1415 and 1417.
[0211] In an alternative embodiment, BOGs 1420 are formed from BOGs
1410 by providing BOGs with instructions to perform some action or
have a discussion. Each group spends a few minutes reading and
attempting to understand the instructions. BOGs 1420 are formed
from pairs of groups from BOGS 1410, where the pairing is random
within the role or category of the group. For example, if there are
two roles (A and B), only two groups with the same roles will be
paired.
[0212] The members of each BOG 1420 then attempt to explain the
instructions to each other. If any "paired" groups can't come to a
consensus/understanding of the instructions, they can "raise their
hands"--alerting the instructor to come "visit" their group and
explain the instructions in more depth. Or if the instructions are
really not being understood, the instructor can do a "broadcast"
audio/video chat to the entire class--clarifying for all of them
the meaning of the instructions.
[0213] FIG. 15 illustrates forming BOGs with students based on more
than one student metric. By way of example, students 1320 are
arranged in order of increasing quiz scores and identified by their
role, indicated as a Role A, B, C, or D. Students 1320 are formed
into BOGs 1510 comprising individual groups each having students
with the same role. Thus, as illustratively, BOGs 1511 includes
students of role A and adjacent quiz scores, 1513 includes students
of role B and adjacent quiz scores, 1515 includes students of role
C and adjacent quiz scores, and 1517 includes students of role D
and adjacent quiz scores, At a future time, follow-on BOGs may be
formed into BOGs 1520 from student having different roles and
adjacent score of students with those scores. Thus, for example,
BOG 1521 consists of one student with each role, such as the A role
student with the highest score, the B role student with the highest
score, the C role student with the highest score, and the D role
student with the highest score. BOGs 1523/1525 consist,
respectively, of the A role student with the second/third highest
scores, the B role student with the second/third highest scores,
the C role student with the second/third highest scores, and the D
role student with the second/third highest scores. In general,
sequences of BOGs may be created where members of each group each
are assigned to a subsequent group that will have one member from
each of the previous groups. In addition, early groups may be
reconstituted at any time, and students returned to those groups.
As many as five groups in sequence may be formed, each created
automatically based on criteria specified in the lesson plan.
[0214] An example of sequential BOGS will now be discussed with
reference to a lesson to teach students negotiating skills, in
particular the technique known as Best Alternative To a Negotiated
Agreement (BATNA). The purpose of BATNA is to find an advantageous
alternative that a negotiating party can take if negotiations fail
and an agreement is not reached.
[0215] In this example, the students are automatically put into
several groups that role play negotiating water rights, where each
student is assigned a stakeholder role, such as a farmer, an
environmentalist, farmers, home owners, engineers and
environmentalists. This example will have 3 different sequential
BOGs, Phase I (or BOG 1), followed by Phase II (or BOG 2), followed
by Phase III (or BOG 3).
[0216] For BOG 1, 4 stakeholder groups are formed--that is, each
group consists of students playing the same stakeholder role.
During BOG 1, the students are instructed to devise a strategy for
negotiating, including a BATNA. During BOG 1, the students know
that BOG 2 and BOG 3 will follow.
[0217] After 10 minutes, new groups (BOG 2) are formed, which each
include one representative from each of the previous groups, such
that each student is the sole representative of their stakeholder.
The students then simulate negotiations and attempt to discover the
BATNAs of other stakeholders.
[0218] After another 10 minutes, the original group are reformed in
BOG 3. The students report inferences about the BATNAs for each of
the other stakeholders.
[0219] Lastly, all students return to class. Each of the groups
forming BOG 1 summarize one BATNA they inferred, and that group
then verifies their inferences.
[0220] As this example illustrates, we provide incentives and
consequences for each student at every phase, since their groups
are relying on them to learn the material. System 100 may also
administer a quiz right after the introductory lecture whose
answers can be used to automatically compose BOGs so that there is
a range of competency.
BOG Definitions
[0221] An example of the use of Instruction Design web app 300G to
define BOGs is illustrated in the screenshots of FIGS. 20A-20F.
[0222] FIG. 20A shows a screen 2000A which provides accesses tools
to set up rules for forming a BOG. Screen 2001 includes a viewing
area for the steps of the lesson plan and the current lesson plan
step being edited as step 2002. Instruction Design web app 300G
includes a BOG Widget 2004 which, when clicked, provides possible
applications to the BOG, such as how to organize the BOG, which may
be for example and without limitation, on how to pair up students,
or an activity, which may be for example and without limitation, to
view a document, edit a document, or view an image, or to
paraphrase provided instructions with another BOG.
[0223] Thus, for example, in screen 2001 the BOG Widget was
previously selected to produce an application 2011, labeled
"Organize, Create BOGS with 2 Students each "Two Roles, Groups of
2," an application 2012 labeled "Activity" with an indication 2013
that the embedded documents are for the "A" role students, an
application 2014 labeled "Activity" with an indication 2015 that
the embedded documents are for the "B" role students, an labeled
2016 "Activity" with an indication 2017 that the embedded documents
are for all students in the group, an application 2018, labeled
"Pair Up, Pairs of BOGs work Together" with an indication 2010 that
this activity is for all students in the group, and an application
2020, labeled "Activity" with an indication 2010 that this activity
is for all students in the group
[0224] From the Organize application, the user may define the
composition of BOGs, such as if the students have roles (such as
none, A, B, C, etc.), the composition of the group, which may be,
for example and without limitation, homogeneous, where the roles
are in the same group, such that six students might divide into AAA
BBB, AA BB CC, etc., heterogeneous, where the roles are in
different groups, such that six students might divide into AAB ABB,
ABC, etc., and group size (such as an ideal number of students, a
min number of students, or a max number of students).
[0225] Another way to organize students is to "pairify," in which
group are initially formed from an even number of groups which may
be given activities individually or in pairs of groups, as
illustrated in FIG. 14 or FIG. 15.
[0226] FIG. 20B shows a screen 2000B which being used to organize
the BOG, where the group name is provided in portion 2003, the
group is defined a having 2 roles (A and B) in portion 2004, the
group is defined as having a homogeneous composition in portion
2005, and the group sizes are defined in portions 2006 and
2007.
[0227] FIG. 20C shows a screen 2000C which being used to provide a
shared whiteboard for the group. A link to the shared documents is
provided in portion 2008, and an image of the document to be
embedded is shown in portion 2009.
Instruction BOG View
[0228] As discussed above, BOG Activity In Progress View State
presents a BOGs Heat Map Widget, a Move Students Between BOGs
Widget, and an Embedded BOG Widget, which shows the instructor the
currently selected BOG.
[0229] FIG. 16 illustrates the generation of the BOGs Heat Map
Widget 1600 of Instructor Browser web app 300A. The BOGs Heat Map
Widget 1600 presents a graphic representation of activity in BOGs,
shown illustratively and without limitation as BOGs 1410. During
the breakout session, each BOG interacts with shared
whiteboard/documents though Cloud Hosting Service 370 and with each
other through Video Services 392 and a chat service provided by
Publish/Subscribe Service 390. Instructor Browser web app 300A
places calls to College API Servers to extract BOG metrics, which
may, for example and without limitation, be metrics of in-group
video/audio interactions between students, in-group chat message
between students, and in-group document interactions. Specific
examples of BOG metrics may include, but are not limited to, how
much talking is taking place; how much activity is taking place in
the shared work product; how much use is being made of Group Chat;
the average "level" (determined by quiz performance) of the group;
the average "initiative" level (determined by how many times the
members have previously raised their hands during class); the
average amount of talking; the variance in talking, indicating how
evenly talking is distributed across group members; the variance in
amount of time cursors are engaged in the shared work product,
indicating how evenly work production is distributed across group
members; ratios consisting of the person with the greatest activity
(e.g., amount of talking or cursor activity) being divided by the
average of the activity (e.g., amount of talking or cursor
activity) of all other members (which provides a measure of whether
one person is dominating). The BOG metrics are directed, by a call
from though Publish/Subscribe Service 390, into BOGs Heat Map
Widget 1600. being divided by the average of the activity of all
other members (which provides a measure of whether one person is
dominating). The BOG metrics are directed, by a call from though
Publish/Subscribe Service 390, into BOGs Heat Map Widget 1600.
[0230] The BOGs Heat Map presents each BOG as a similarly shaped
object, such a rectangle, that is labeled, for example, with the
names of students in the group. Thus, for illustrative purposes,
BOG 1411 is represented by a rectangle 1601, BOG 1413 is
represented by a rectangle 1603, and BOG 1415 is represented by a
rectangle 1605,
[0231] In one embodiment, the BOG metrics for each BOG are
converted into a fill color for the corresponding rectangle of the
BOGs Heat Map Widget. In one embodiment, the color scheme is chosen
to permit a quick visual determination of any outliers among the
different groups.
[0232] Thus, for example, the color corresponding to the metric of
BOGs 1411/1413/1415 fills the interior of rectangles
1601/1603/1604, respectively. A visual inspection of rectangles
1601/1603/1604 provides a quick way of determining the relative
performance of groups, and may provide the basis for the instructor
to "drop in" on specific BOGs.
[0233] If, for example, the BOG metric is the amount of in-group
interactions, the instructor can quickly determine groups that may
be having problems with the activity and may switch to the BOG
Activity In Progress Instructor "Visiting" One Group View State,
which permits the instructor to communicate with members of the
selected BOG.
Moving Breakout Whiteboards to the Classroom View
[0234] In certain embodiments, platform 100 enables BOG shared
whiteboards to be shared with the entire class, as will be
discussed with reference to FIG. 28 and FIG. 29 which shows one
embodiment of a screenshot 2900 of a document from a breakout
session; and FIG. 30 which shows one embodiment of a screenshot
3000 of a document from a breakout session imported into the
Everyone In Class (Students View) Student View State. FIGS. 29 and
30 illustrate the moving of breakout whiteboard to the classroom
view, where FIG. 29 is a screenshot of a document from a breakout
session; and FIG. 30 is a screenshot of a document from a breakout
session imported into the Everyone In Class (Students View) Student
View State.
[0235] Thus, for example FIG. 28 is a screenshot of a classroom in
"BOG Spotlight" mode. In this mode a single BOG is sharing the
document they have been working on. To share the shared document
with the class, the instructor clicks the "Open" button in the
upper right corner of the document. This opens a new browser tab
with the shared document, as shown in FIG. 29. The instructor then
creates a new "Ad Hoc Page" by clicking a button available in the
instructor application's footer area. The instructor pastes the URL
of the document from the new browser tab into the editor for the Ad
Hoc Page.
[0236] Lastly, the instructor clicks the lesson step widget that is
along the side of her screen and represents the "Ad Hoc Page" to
the class. The instructor and all of the students in the class are
viewing the document that was originally edited by the BOG, as
shown in FIG. 30.
Classroom Videos Grid
[0237] Each Student Browser web app 300B generates a video feed
from the camera of the student's device 130. To provide the
appearance of a classroom of students, it is desirable for the
students and the teacher to view as many of these video feeds as
possible. FIG. 17 illustrates a Classroom Videos Grid 1700, which
is generally similar to classroom videos grid 312, except as
explicitly stated, and which is presented on the Instructor Browser
web app 300A, any of the Student Browser web apps 300B, or on other
browser web apps 300.
[0238] FIG. 17A illustrates the display of a plurality of video
feeds 1701 in the Classroom Videos Grid 1700. In certain
embodiments, each video feed of the plurality student's video feeds
is obtained from one of the plurality of Student Browser web apps
300B and are arranged on a rectangular grid for display within an
area 1703 of the Classroom Videos Grid. Thus, for example and
without limitation, video feeds 1701 in FIG. 17A are shown, for
example and without limitation, as being arranged in columns for 4
video feeds and extending as far horizontally as is necessary to
fit all of the video feeds, and will repeat to form an endless loop
of videos. Of video feeds 1701, video feeds 1720 are at least
partially within area 1703, video feeds 1710 are just about to
scroll into area 1703, and video feeds 1730 have just previously
scrolled through area 1703.
[0239] Classroom Videos Grid 1700 scrolls all of the video feeds
horizontally, so that a portion of all of the video feeds are
visible at one time, such that each video is scrolled across area
1703 within some reasonable time, such as 10 seconds, in an endless
loop. Over the course of a specific amount of time, such as two
minutes, all of the video feeds will scroll through area 1703. When
it is determined that a video feed is to be displayed in Classroom
Videos Grid 1700, video service API call is made to retrieve the
video feed for display. In various embodiments, area 1703 and video
feeds 1701 are arranged to display, for example, 1.times.1,
2.times.2, 3.times.3, or 4.times.4 of video feeds simultaneously.
In certain embodiments, the video feeds are turned on just prior to
moving into area 1703 so that the effect is to view a continuous
movement of video feeds.
[0240] In certain embodiments, the plurality student's video feeds
are arranged, for example and without limitation, by virtually
arranging the video feeds on a rectangular grid having the height
of area 1703 and moving the virtual arranged video feeds across a
rectangular area that corresponds to area 1703. Thus, for example,
if area 1703 has the height to accommodate 4 video feeds, then the
grid will have a height of 4 video feeds. The movement of the video
feed across area 1703 is incremental--that is they move
incrementally by a distance less that is less than width of the
video feed, and thus appear to crawl across the area. As the video
feeds move, the video feeds that are streamed to the browser are
only those video feeds that: are at least partially in area 1703;
or will be at least partially in area 1703 at the next increment of
time are provided to the Classroom Videos Grid 1700 for display.
Thus, all of the video feeds will appear to move smoothly across
area 1703 (moving from left to right or from right to left) while
minimizing the number of video feeds sent to the browser for
viewing.
[0241] In certain embodiments, the virtual arrangement of videos on
the grid is fixed as the videos scroll. In certain other
embodiments, the arrangement of videos is randomized on the grid.
In other embodiments, copies of one or more video feeds are
duplicated and placed on the grid, in a fixed or randomized
arrangement and the copies may be replaced after moving off of area
1703. The placement of video feed copies on the grid has the effect
of the student's not being able to predict when their video feed
will be presented.
[0242] FIGS. 17B, 17C, and 17D illustrate three sequential views of
the Classroom Videos Grid 1700, with adjacent video feed columns
1741, 1742, 1743, 1744, 1745, 1746, and 1747 as they scroll past
within area 1703.
[0243] FIG. 17B shows video feeds 1701 at a first time, where at
least partially viewable video feeds 1720 includes video feed
columns 1742, 1743, 1744, 1745, and 1746, where video feeds 1710
includes video feed column 1741, and where video feeds 1730
includes video feed column 1747.
[0244] In certain embodiments, the horizontal scrolling of video
feeds 1701 is controlled by JavaScript code within the browser web
app 300. The JavaScript code operates as a "timer" that is
continuously reset after some time interval, at which time the
JavaScript code moves the plurality of video feeds 1701 a distance
to the left.
[0245] FIGS. 17C, and 17D illustrate additional sequential views of
the Classroom Videos Grid 1700 after the view of FIG. 17B. When a
video feed column has moved so that it is no longer at least
partially within area 1703, all of the video streams in that column
are shut down. Thus, for example, in going from the view of FIG.
17B to that of FIG. 17C, video feed column 1746 has moved out of
area 1703, and thus the video feeds comprising video feed column
1746 are turned off, and in going from the view of FIG. 17C to that
of FIG. 17D, no video feed columns have moved out of area 1703, and
thus no video feeds are turned off
[0246] When a video feed column is about to move into within area
1703, all of the video streams in that column are turned on. Thus,
for example, in going from the view of FIG. 17B to that of FIG.
17C, video feed column 1741 is about to move into area 1703, and
thus the video feeds comprising video feed column 1741 are turned
on, and in going from the view of FIG. 17C to that of FIG. 17D, no
new video feeds are about to move into area 1703, and thus no video
feeds are turned on.
[0247] During a class a student may "raise their hand" by
interacting with the Page Footer Widget of the Student Browser web
app 300B, as described above. When the student raises their hand
using the "raise/lower" hand tool, the Classroom Videos Grid 1700
brings that student's video feed into the Classroom Videos Grid
1700.
[0248] FIGS. 18A and 18B illustrate the effect of the "raise/lower"
hand tool on the Classroom Videos Grid 1700. In the example of FIG.
18A, a student whose video feed is not within area 1703 has
activate the "raise/lower" hand tool on their Student Browser web
app 300B. As illustrated in FIG. 18B, Classroom Videos Grid 1700
responds to this activation by scrolling the video feeds so that
that student's video feed is within area 1703, and also
superimposes a small raised hand image over the student's video
feed and surrounds the icon with an orange border so that it is
clear which student is requesting attention. The icons for students
with raised hands are not scrolled; they are pinned in the matrix
so that they are visible until the instruction either clicks on
them or dismisses them.
[0249] In one embodiment, Student Browser web app 300B presents a
"raise hand" tool in a Page Footer Widget, as discussed above. When
a student clicks the "raise hand" tool, a call is placed to College
API Servers 350 and then to Publish/Subscribe Service 390, as
discussed above with reference to FIGS. 7 and 8. The identity of
that Student Browser web app 300B is stored in database 803, and
the Publish/Subscribe Service 390 sends a message out to all
Instructor Browser web app 300A, Student Browser web apps 300B,
Tech Support/Observer Browser web apps 300C and Recorder Browser
web apps 300D that are running. When each browser web apps 300
receives this message, the scrolling of Classroom Videos Grid 1700
is stopped, each browser web app determines if the video feed of
the Student Browser web app 300B having clicked the "raise hand"
tool is within area 1703 and arranges Classroom Videos Grid 1700
such that that video feed is within area 1703. and an orange
rectangle and a small orange hand is displayed along the edge of
that video feed.
[0250] If the instructor operating Instructor Browser web app 300A
wants to "call on" the student with the raised hand and "put them
in the spotlight," they click on the student image in the matrix of
videos on the Instructor Browser web app. A call is then placed to
College API Servers 350 which then calls Publish/Subscribe Service
390, sending instructions to change the Classroom Videos Grid 1700
in each browser web app to replace all of the other videos feeds in
Classroom Videos Grid 1700 to display only the student's video feed
of the called on student. In addition, the Instructor Browser web
app 300A makes a call to the College API Servers 350 telling it
that the clicked on student is "in the spotlight." The server then
uses Publish/Subscribe Service 390 to send the "student is spotlit"
message out to all of the Student Browser web apps 300B, Tech
Support/Observer Browser web apps 300C and Recorder Browser web
apps 300D that are running. The browser web apps 300 receive a
"student is in the spotlight" and the browser web apps switch to a
spotlit view. Thus, for example, the Instructor Browser web app
300A displays the Everyone in Class With One Student Spotlit
Instructor View State, and all Student Browser web apps 300B
displays the Everyone in Class With One Student Spotlit Student
View State.
[0251] FIGS. 19A and 19B illustrates the effect of an instructor
spotlighting a student from Classroom Videos Grid 1700. During a
class the instructor may wish to "spotlight" one student and only
show the spotlighted student's video feed. As illustrated in FIG.
19A, the instructor operating Instructor Browser web app 300A may
spotlight a student by clicking on the student's video feed 1901
from within area 1703. As illustrated in FIG. 19B, Classroom Videos
Grid 1700 responds to this selection by switching to a 1.times.1
view of that video feed, effectively viewing and broadcasting only
the spotlit student to browser web apps 300.
[0252] Once the instructor has called on a student and put them "in
the spotlight," they can then take them out of the spotlight. To do
this, the instructor operating Instructor Browser web app 300A
clicks on the spotlit student's video, resulting in a call being
placed to College API Servers 350 which then calls
Publish/Subscribe Service 390, which causes student's video to be
removed and then replaced with the matrix of the entire class'
videos as Classroom Videos Grid 1700. Specifically, messages are
sent to all Student Browser web apps 300B, Tech Support/Observer
Browser web apps 300C and Recorder Browser web apps 300D that are
running to resume showing the Classroom Videos Grid 1700.
Structure and Use of the Platform for Classroom Instruction
[0253] The following discussion is centered on the experience of
the users of platform 100, and it will be understood that the
actions of instructors are by way of interaction with Instructor
Browser web app 300A, the actions of students are through their
Student Browser web ap 300B, and the actions of instructional
designers using Instruction Design web app 300G.
[0254] In certain embodiments, the steps of using system 100 may
include: [0255] 1. an instructional designer uses Instruction
Design web app 300G to define lesson plan 420, as described for
example, in the discussion of FIG. 5; [0256] 2. a school
administrator "opens class" using School Admin Browser web app
300H, causing system 100 to prepare for the teaching of lesson plan
420. Thus, for example and without limitation, databases are
updated to reflect the upcoming teaching of lesson plan 420 to
enrolled students, and if required, clones copies of any shared
documents, as discussed above with reference to FIGS. 5 and 11 and
prepares quizzes, as discussed above with reference to FIG. 12.
[0257] 3. the instructor logs onto system 100 using Instructor
Browser web app 300A, which displays the Before Class has Started
Instructor View State and students log into the system using
Student Browser web app 300B, which displays the Before Class has
Started Student View State. [0258] 4. At the instructor's
discretion or at the appointed class start time, the instructor
transitions to the Everyone In Class (Instructor View) Instructor
View State, which sends publish/subscribe messages to the Student
Browser web apps 300B to transition to the Everyone In Class
(Students View) Student View State. [0259] 5. the instructor
sequences through the lesson plan steps and publish/subscribe
messages are sent to the Instructor Browser web app 300A and
Student Browser web apps 300B to transition the browsers to view
states defined by lesson plan 420. [0260] 6. when a quiz lesson
plan step is selected, system 100 sequences through steps described
above with reference to FIG. 12 to determine which question(s) are
presented to each student. [0261] 7. when a BOG lesson plan step is
selected, system 100 uses stored performance data to assign
students to specific BOGs, as discussed above with reference to
FIGS. 13, 14, and 15. Alternatively, in some embodiments students
are assigned to BOGs at random or the instructor manually assigns
students to BOGs. Following assignment, publish/subscribe messages
are sent to transition the instructor view to BOG Activity In
Progress Instructor View State and the student view state to BOG
Activity In Progress Student View State. The students may interact
with other students of their BOG and the instructor many monitor
BOG groups using the BOG heat map and visit individual BOGs, as
discussed above with reference to FIG. 16. In addition, the
instructor may operate, or request operation, of Tech
Support/Observer Browser web app 300C to move students between
BOGS, as discussed above, and bring content from the BOG to the
entire class, as discussed above regarding FIGS. 28-30.
[0262] Platform 100 includes many features which may be used to
provide an active-learning class of a course over a computer
network. These features enable active learning by empowering
teachers by giving them access to tools that promote active
learning; prepares students for active learning by engaging them
when they are exposed to new material and presenting
active-learning tasks; collecting and using data to optimize the
active learning environment, automatically tailoring class
instruction based on an assessment of at least one student's
performance during the course; creates and conducts BOGs partly on
the basis of data about previous student performance; and uses may
be used to adapt the system to improve the active learning
environment over time.
[0263] During lecture, students are given a poll, respond, and a
graph of the results is shown to the class. Several students are
then called on to explain their votes. Students often must answer a
query by typing into Group Chat. The instructor then selects some
of the responses and asks those students to expand further to the
class.
[0264] In certain embodiments, lesson plan 420 include a mixture of
lectures and BOGs in various proportions, Thus, for example a
primality lecture class may have a ratio of 75% lecture/25% BOG,
while a primality laboratory class may have a ratio of 25%
lecture/75% BOG.
[0265] Platform 100 provides multiple ways for students to interact
with instructor and class, including but not limited to: "raise
their hands," enter answers into chat and email/tech
support/etc.
[0266] Instructors are provided with dashboards/reports/very
detailed audit trail of: attendance in each lesson: number of
minutes attended each lesson; time "on camera" vs "off camera"
during lesson; quizzes attempted vs. quizzes passed; number of
interactions with "classroom chat" stream; number of interactions
with "BOG chat" stream and number of interactions with
documents/shared whiteboard during BOGs
[0267] Technical support tools allow instructors and/or technical
support to: send "flash" messages to individual students; turn
individual students' audio/video off if necessary; and
refresh/reset browser (web applications) of students who are
"stuck" in some way
[0268] Platform 100 may be configured to provide each student with
a survey at end of each class to gather students' experiences,
opinions, etc. These surveys are fed back to the admissions process
as well as the AI/ML system that helps allocate future students to
BOGs. This data also helps coaches/mentors to proactively help
students with issues/problems
[0269] In certain embodiments, platform 100 includes the following
structure or elements. [0270] 1. Platform 100 may accommodate up to
200 students in a combination of lecture and active-learning BOGs.
[0271] 2. Platform 100 has at least two types of chat: [0272] a.
Classroom Chat can be viewed by all students and all faculty. The
classroom chat stream is automatically organized thematically, so
that comments that are responding to the same thing are grouped
together. [0273] b. Help Chat can only be seen by help desk
personnel and teaching assistants. This chat also includes controls
for the viewing mode (i.e., the size of the student matrix and
whether scrolling is enabled; see below). [0274] 3. Platform 100
allows the user to define four different types of events as lesson
plans 420: [0275] a. Content provided from PowerPoint, Keyonte,
PDF, or any drawing program and which are shown directly to the
students (e.g., slides in a standard PPT format or videos). [0276]
b. Materials sent to BOGs (with different materials being sent to
different groups, if so specified) [0277] c. Polls (which can be
pre-determined or configured on-the-fly, with the results shown to
the students as a bar graph) [0278] d. Quiz questions; all answers
are machine graded. [0279] i. Multiple choice (including simple
True/False) [0280] ii. "Mark all that apply" multiple answers
[0281] iii. Match entries in Column A with those in Column B [0282]
4. The view states of Instructor Browser web app 300A includes:
[0283] a. A Lecture Mode, such as Everyone In Class (Instructor
View) Instructor View State, which includes a preview of the
sequence of upcoming slides on the left, the slide currently being
shown, the faculty member (typically on the upper right of the
display) and the classroom videos grid (or "Student Matrix"),
typically immediately below the instructor's icon), which is a set
of students randomly sampled to be shown in a matrix that is
continually changing: [0284] i. The default is a 4.times.4 matrix,
with icons of 16 students smoothly scrolling horizontally, as if a
camera were panning across the audience. [0285] ii. If the
student's computer or bandwidth are being taxed, the student can
select "Lite Mode" (the control is in Help Chat), where only a
2.times.2 matrix or even a single icon is shown and/or scrolling is
turned off [0286] iii. If a student raises his or her hand (by
clicking on a "raise hand" button at the bottom of the screen), a
hand symbol appears over his or her icon, and this icon is "pinned"
to the matrix--not being replaced. [0287] iv. The instructor can
call on a student by clicking on his or her icon in the Student
Matrix. At this point, the student's icon expands to fill the space
occupied by the entire matrix. When the instructor later clicks on
this enlarged icon, it shrinks back to its original size and the
Student Matrix reappears. [0288] b. A Breakout Mode, such as a BOG
Activity In Progress Instructor View State, which includes a
preview of the sequence of upcoming slides appears on the left, and
the rest of the screen contains a set of colored bars (one that
represents each BOG), which serve as heat maps (see below), and the
video feeds for students in one BOG and their work product (or
instructions). [0289] i. The instructor can click on the bar
representing a particular BOG and join that group, seeing those
students, their instructions, and any work product. The instructor
can then participate as a full member of that group. [0290] 5. The
view states of Student Browser web app 300B includes: [0291] a. A
classroom videos grid with a default matrix being 4.times.4 matrix,
with icons of 16 students smoothly scrolling horizontally, as if a
camera were panning across the audience. This grid produces a
"sense of belonging," which is known to increase student retention.
[0292] b. A Lecture Mode, such as an Everyone In Class (Students
View) Student View State, in which Student Browser web app 300B
displays the slide, video, or anything else that can be accessed
from the instructor's desktop or from the internet currently being
discussed, a large instructor's video feed, and the Student Matrix
under the instructor's video feed. [0293] c. A Breakout Mode, such
as a BOG Activity In Progress Student View State, in which Student
Browser web app 300B displays the other students in their group
plus any instructions and any materials (e.g., a Google Doc, Sheet,
or Form) that they will collaboratively use to produce a work
product. [0294] 6. The view states of Tech Support/Observer Browser
web app 300C is much like the view states of Instructor Browser web
app 300A, but neither the students nor instructor can see the
observer. The observer can not only view lectures, but also click
on the heat map to join specific BOGs. [0295] 7. The system
includes a student dashboard that indicates progress on mastering
the learning objectives. [0296] 8. All learning objectives are
defined in terms of "System 1" teaching (this is based on Daniel
Kahneman's distinction between System 1 and System 2) (see,
Kahneman, D. (2011). Thinking Fast and Slow. New York: Farrar,
Straus and Giroux.) [0297] a. We have a novel interpretation of
System 2: This is the system that loads heavily on IQ tests. System
1 operates by matching and/or filtering input, reflexively. [0298]
b. Our aim is to teach by drawing most heavily on System 1
processes, which do not involve IQ. [0299] c. We achieve this aim
by defining two types of learning objectives: [0300] i. "Hacks" are
templates that solve specific problems. For example, "Five
paragraph form" can be used whenever one is asked to write an
essay. [0301] ii. Heuristics" are processes that often (but not
always) will solve a problem. Our aim is to teach specific Hacks or
Heuristics so they are automatically triggered via System 1 in the
appropriate situation. [0302] iii. The teaching steps are [0303] 1.
Increase comprehension. This is represented by the top part of an
hourglass, which serves as a funnel of appropriate situations that
will trigger the learned Hack or Heuristic. [0304] 2. Enhance
memory [0305] 3. Illustrate application by the appropriate
applications for the Hack or Heuristic that is triggered. This is
represented by the bottom part of an hourglass, which indicates a
wide range of situations where the Hack of Heuristic can be
applied. [0306] 4. The conditions and actions of a Hack or
Heuristic are broad, allowing them to be used in a wide variety of
relevant situations. The use of Hacks and Heuristics solves some
aspects of the "problem of transfer"--ensuring that students use
material in new, unfamiliar situations.
[0307] In certain embodiments, the BOGs promote active learning, as
follows: [0308] 1. Rectangles represent each BOG on the instructor
screen (in Faculty Mode), with the color of each rectangle
functioning as a heat map that indicates what is going on in each
group: [0309] a. The information for the selected BOG is provided
in numerical form and/or graphical form. The graphical form may
include "heat maps" which use color coding to indicate quantity,
with the extremes being red or blue--which are "pop out" colors
that are immediately evident. Thus, the instructor can immediately
see which groups are behaving differently from the majority. [0310]
b. Maps can indicate many different quantities, including: [0311]
i. How much talking is taking place; [0312] ii. How much activity
is taking place in the associated Doc or other work product; [0313]
iii. How much use is being made of Group Chat; [0314] iv. The
average "level" (determined by quiz performance) of the group;
[0315] v. The average "initiative" level (determined by how many
times the members have previously raised their hands during class);
or [0316] vi. The variance in talking, indicating how evenly
talking is distributed across group members. [0317] vii. The extent
to which one person is dominating either the discussion or
production of the work product. [0318] c. The instructor can "pin"
given groups while they are taking place, allowing them later to be
revisited easily during post-BOG debrief sessions. [0319] 2. The
instructor can click on one of the rectangles representing a BOG
and immediately join that BOG. [0320] 3. Students can raise their
hands in BOGs, which is represented by a hand icon being pinned to
the corresponding rectangle (which is immediately evident to the
instructor). [0321] 4. Each student is associated with many "tags,"
indicating a wide range of characteristics (e.g., quiz performance
level, quiz performance level for specific learning outcomes,
region of residence, type of job, etc.). Any of these tags--or
combinations thereof--can be used to create BOGS. [0322] a. BOGs
can be set up according to these criteria (or randomly or with some
other criteria devised by the instructor) prior to class. [0323] b.
BOGS can be set up on the fly, using the same criteria that can be
used in advance (including based on results of a just-administered
quiz or poll). [0324] 5. The most direct use of tags concerns how
to use quiz scores to put people into BOGs automatically, based on
grouping them by "relevant" aspects of their performance.
"Relevance" requires matching their strengths to the requirements
of the activity in that particular BOG. This may be accomplished as
follows: [0325] a. Step 1. Empirically derive the dimensions that
characterize different types of quiz questions. [0326] i. First,
use Amazon's "Mechanical Turk" (mTurk) group-sourcing system
(hpps://www.mturk.com/) to generate many of different questions for
many lesson plans. [0327] ii. Next, for each separate lesson plan
420, construct a matrix where each item labels one of the rows and
each item--in the same order--also labels one of the columns
(ignore the diagonal, which would be each item against itself). Use
"the method of triads" to get humans raters to decide how similar
each pair of questions is, and fill in the corresponding cells of
this matrix. [0328] iii. Take the resulting similarity matrix
(which specifies a value for every possible pair) and submit it to
nonmetric multidimensional scaling, Principal Components Analysis,
and similar procedures. This will produce a geometric solution
where each question is a point in an n-dimensional space and will
specify dimensions in that space. [0329] iv. This in turn leads to
recovering the dimensions that underlie the space. Each dimension
in turn becomes a variable in a real-number vector. Each person
would have such a vector, which indicates his or her performance
levels on each dimension. These values define many of the "tags" we
have associated with each student. [0330] v. Additional tags are
specified (some binary) in terms of demographic variables (e.g.,
age, years in school). [0331] b. Step 2. Using the dimensions to
tag new quiz items automatically. [0332] i. Start by hand-coding a
set of questions, specifying each question with a vector where each
value indicates the question's location on one of the dimensions we
recovered above. [0333] ii. Go through a training set, training a
neural net or other machine learning (ML) technique to classify the
items appropriately. [0334] iii. Then give the neural net (or
another ML algorithm) new items and tune it up so that it
classifies them appropriately. This method will be used going
forward to classify every question. [0335] c. Step 3. Classifying
the activities [0336] i. Use the dimensions recovered in Step 1 to
classify each of the BOG activities, again specifying a vector
(where each value indicates the location on one of the dimensions).
[0337] ii. Repeat the procedure in Step 2, but now use descriptions
of the activities. [0338] d. Step 4: Matching [0339] i. For each
person, match the question vectors from previous classes to the
vector for the new BOG, choosing the question vectors that are
closest in the space (none should match completely, given that the
BOG is addressing a new learning outcome). [0340] ii. Using those
questions, compute the quiz scores (based only on the relevant
questions) for each student; group students based on similar quiz
scores. [0341] iii. This procedure will place students with
comparable scores on relevant previous measures into the same BOG.
It can also be used to ensure that there is a range of relevant
abilities in a BOG, as is appropriate for a specific learning
activity (see above). [0342] 6. BOGs can be terminated at a pre-set
time or by intervening. In both cases, a countdown clock appears
indicating that the group will end after a specified number of
seconds (with the default being 10 second). [0343] 7. BOGs can be
organized into a set of distinct "roles," each of which receives
instructions for just that role. [0344] 8. Instructions in BOGs are
written to be "multi-level": They can be approached at different
levels of sophistication. When BOGs are defined by performance
level on quizzes, this means that students of comparable ability
will "self titrate" and interpret the instructions
similarly--allowing them to set themselves at the Goldilocks spot:
Not too each, not too hard, but just challenging enough (which
respects the learning principle of "Desirable Difficulty"--Kosslyn,
S. M. (2017). The science of learning. In S. M. Kosslyn & B.
Nelson (Eds), Building the intentional university. Cambridge,
Mass.: MIT Press.). [0345] 9. The instructor can "broadcast" verbal
instructions (with his or her picture appearing simultaneously) to
all BOGs or just to the subset that corresponds to the set of
rectangles (in the heat map) that are clicked on. [0346] 10. In
order to scale to very large classes, we had to devise mechanisms
where students would not need the instructor to intervene. One
example is to ensure that students understand the instructions for
a BOG. Using the example noted earlier, the students might be role
playing different groups who have to negotiate water rights. There
might be 4 roles (farmers, home owners, engineers and
environmentalists). Groups that have the same roles will have the
same instructions (e.g., take the role of farmers when
negotiating). After they read them, they then join another group
with the same instructions and do the below. In this case: [0347]
a. Students read the instructions while in a BOG and push a key
when they are ready to paraphrase them to another group. [0348] b.
When another group that received the same instructions has pressed
the key, those two groups are temporarily joined. [0349] c. One
group is randomly chosen to paraphrase the instructions to the
other (which has received the identical instructions). [0350] d.
The other group is charged with vetting the paraphrase. If they
feel that the group paraphrasing has not understood the
instructions correctly, they correct that group. [0351] e. When the
two groups agree on the interpretation of the instructions, they
press a key. At that point, the activity begins. [0352] f. If the
groups cannot agree, they press a key to signal the instructor
(their rectangle in the heat map starts pulsing). [0353] g. If the
groups have not responded within a specified period, their
rectangle starts pulsing to signal the instructor. [0354] h. The
instructor intervenes by joining the group and clarifying the
problem. [0355] i. If it seems likely that other groups have had
the same problem, the instructor clarifies them to all groups
(using the "broadcast function" noted above). [0356] 11. After BOGs
end, during debrief a given BOG can be "spotlighted"--which means
that those students and their shared work products (e.g., Google
docs) are shown to the entire class. A set of BOGs can be specified
for spotlighting, and the instructor can control the order of
cycling through them as well as how much time is spent with each
one. [0357] 12. Sequences of BOGs can be specified in advance,
where each BOG assigns students from previous ones in specific
ways. For example, in a first set of BOGs the students in each BOG
could have received a different passage to analyze. The second set
of BOGs would then contain one person from each of the first set,
and they would be asked to compare/contrast the passages they
worked on previously. There could even be a third set of BOGs,
where new groups are composed with one student each from the second
set of BOGs, who now report what they found. And so on.
[0358] In certain embodiments, platform 100 provides synchronous
competency-based assessment of the student by ending every class
with a quiz which may consist of 6 questions. Five of the questions
assess the specific learning outcomes (Hacks or Heuristics) taught
in that class, where there is a minimum of 30 items for the Hacks
and Heuristics taught in each class. One of the questions is
randomly chosen from those for a previous class. The Quiz questions
are generated in part via mTurk: The content is presented with the
existing quiz question(s), and mTurkers are asked to generate
variants of the question. In alternative embodiments, quiz
responses are used to train machine learning algorithms, based on
latent semantic analysis of the training question, the new variant,
and ratings of how acceptable the new question is.
[0359] For each student's quiz, items are sampled randomly without
replacement--so no two students are likely to have the identical
quiz. Each quiz item is tagged by where it falls in a hierarchy
that specifies three levels of competency: Macro (the overarching
unit), Mini (a specific aspect of the unit), and Nano (an
actionable component of the Mini competency, which corresponds to a
"Hack" or "Heuristic").
[0360] Students must answer all questions on a quiz correctly
(100%) in order to pass. If they do not, they can retake the quiz
with a different set of items. If they fail a second time, they
then must watch a recording of the lecture before they can take it
twice more. If they fail on the fourth attempt, they must watch the
lecture again before they have two more tries. When students answer
a question incorrectly, they receive feedback that includes the
correct answer along with a brief explanation as to why that is the
correct answer.
[0361] Platform 100 tracks how many times the student required to
pass each quiz, and uses that information to place students in BOGs
(with other students at the same level). The platform also tracks
the student's Nano competency and if they seem problematic for a
given student. This allows tutors to know areas in which the
student needs help.
[0362] One embodiment of each of the methods described herein is in
the form of a computer program that executes on a processing
system, e.g., a one or more processors that are part of a computer
network. Thus, as will be appreciated by those skilled in the art,
embodiments of the present invention may be embodied as a method,
an apparatus such as a special purpose apparatus, an apparatus such
as a data processing system, or a carrier medium, e.g., a computer
program product. The carrier medium carries one or more computer
readable code segments for controlling a processing system to
implement a method. Accordingly, aspects of the present invention
may take the form of a method, an entirely hardware embodiment, an
entirely software embodiment or an embodiment combining software
and hardware aspects. Furthermore, the present invention may take
the form of carrier medium (e.g., a computer program product on a
computer-readable storage medium) carrying computer-readable
program code segments embodied in the medium. Any suitable computer
readable medium may be used including a magnetic storage device
such as a diskette or a hard disk, or an optical storage device
such as a CD-ROM.
[0363] It will be understood that the steps of methods discussed
are performed in one embodiment by an appropriate processor (or
processors) of a processing (i.e., computer) system executing
instructions (code segments) stored in storage. It will also be
understood that the invention is not limited to any particular
implementation or programming technique and that the invention may
be implemented using any appropriate techniques for implementing
the functionality described herein. The invention is not limited to
any particular programming language or operating system.
[0364] Reference throughout this specification to "one embodiment"
or "an embodiment" means that a particular feature, structure or
characteristic described in connection with the embodiment is
included in at least one embodiment of the present invention. Thus,
appearances of the phrases "in one embodiment" or "in an
embodiment" in various places throughout this specification are not
necessarily all referring to the same embodiment. Furthermore, the
particular features, structures or characteristics may be combined
in any suitable manner, as would be apparent to one of ordinary
skill in the art from this disclosure, in one or more
embodiments.
[0365] Similarly, it should be appreciated that in the above
description of exemplary embodiments of the invention, various
features of the invention are sometimes grouped together in a
single embodiment, figure, or description thereof for the purpose
of streamlining the disclosure and aiding in the understanding of
one or more of the various inventive aspects. This method of
disclosure, however, is not to be interpreted as reflecting an
intention that the claimed invention requires more features than
are expressly recited in each claim. Rather, as the following
claims reflect, inventive aspects lie in less than all features of
a single foregoing disclosed embodiment. Thus, the claims following
the Detailed Description are hereby expressly incorporated into
this Detailed Description, with each claim standing on its own as a
separate embodiment of this invention.
[0366] Thus, while there has been described what is believed to be
the preferred embodiments of the invention, those skilled in the
art will recognize that other and further modifications may be made
thereto without departing from the spirit of the invention, and it
is intended to claim all such changes and modifications as fall
within the scope of the invention. For example, any formulas given
above are merely representative of procedures that may be used.
Functionality may be added or deleted from the block diagrams and
operations may be interchanged among functional blocks. Steps may
be added or deleted to methods described within the scope of the
present invention.
* * * * *