U.S. patent application number 15/741457 was filed with the patent office on 2018-12-27 for a system and a method for monitoring progress of a learner through an experiential learning cycle.
The applicant listed for this patent is Intersective Pty Ltd. Invention is credited to Nicole James, Philipp Laufenberg, Beau Leese, William Emanuel Sonnenreich, Susannah Watson.
Application Number | 20180374374 15/741457 |
Document ID | / |
Family ID | 57684603 |
Filed Date | 2018-12-27 |
United States Patent
Application |
20180374374 |
Kind Code |
A1 |
Watson; Susannah ; et
al. |
December 27, 2018 |
A System and A Method for Monitoring Progress of a Learner Through
an Experiential Learning Cycle
Abstract
A computer system for monitoring progress of at least one
learner through an experiential learning cycle, the computer system
comprising: a computer server accessible through a communications
network, the computer server arranged to receive event data through
the communications network from a user computing device, the event
data being generated by the at least one learner progressing
through the experiential learning cycle; a memory for storing
empirical data indicative of the experiential learning cycle and
event data received at the computer server through the
communications network; and a processor configured to process the
received event data and comparing the processed event data with the
empirical data to determine the progress of the at least one
learner through the experiential learning cycle.
Inventors: |
Watson; Susannah; (Redfern,
AU) ; James; Nicole; (Sydney, AU) ;
Laufenberg; Philipp; (Sydney, AU) ; Sonnenreich;
William Emanuel; (Bondi, AU) ; Leese; Beau;
(Redfern, AU) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Intersective Pty Ltd |
Sydney, New South Wales |
|
AU |
|
|
Family ID: |
57684603 |
Appl. No.: |
15/741457 |
Filed: |
July 1, 2016 |
PCT Filed: |
July 1, 2016 |
PCT NO: |
PCT/AU2016/050582 |
371 Date: |
January 2, 2018 |
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G09B 5/14 20130101; H04L
67/22 20130101; G09B 7/077 20130101; G09B 5/12 20130101; G06Q 50/20
20130101; G06Q 50/205 20130101 |
International
Class: |
G09B 5/12 20060101
G09B005/12; G09B 5/14 20060101 G09B005/14; G09B 7/077 20060101
G09B007/077; H04L 29/08 20060101 H04L029/08 |
Foreign Application Data
Date |
Code |
Application Number |
Jul 3, 2015 |
AU |
2015902614 |
Claims
1. A computer system for monitoring progress of at least one
learner through an experiential learning cycle, the computer system
comprising: a computer server accessible through a communications
network, the computer server arranged to receive event data through
the communications network from a user computing device, the event
data being generated by the at least one learner progressing
through the experiential learning cycle; a memory for storing
empirical data indicative of the experiential learning cycle and
event data received at the computer server through the
communications network; and a processor configured to process the
received event data and comparing the processed event data with the
empirical data to determine the progress of the at least one
learner through the experiential learning cycle.
2. The computer system of claim 1, wherein the experiential
learning cycle comprises a plurality of learning phases and the
processor is configured to process the received event data to
determine at least one of the plurality of learning phases of the
experiential learning cycle in which the learner currently is.
3. The computer system of claim 2 wherein the computer server is
configured to process the received event data to determine a phase
probability that is indicative of a likelihood that the at least
one learner is currently in the at least one determined learning
phase.
4. The computer system of claim 1, 2 or 3, wherein if a result of
the comparison meets a predetermined criteria, the processor is
further configured to automatically trigger an intervention.
5. The computer system of claim 4, wherein the computer server is
configured to automatically generate information in relation to the
triggered intervention and to automatically make the information
available on one or more user computing devices.
6. The computer system of claim 1, wherein the processor is
configured to process the collected data to determine a phase
velocity of the at least one learner, the phase velocity being
indicative of a pace of the at least one learner progressing
through a plurality of learning phases of the experiential learning
cycle
7. The computer system of claim 6, wherein the processor is
configured to compare at least one of the determined phase and the
determined phase velocity of the at least one learner with the
empirical data to evaluate the progress of the at least one learner
through the experiential learning cycle.
8. The computer system of claim 7, wherein the processor is
configured to use the evaluation of the progress of the at least
one learner to determine a rating for the at least one learner.
9. The computer system of claim 1, wherein the processor is further
configured to process the received event data and compare the
processed event data with stored event data to determine prediction
data indicative of one or more future events, the stored event data
comprising historical event data generated by the at least one
learner and/or one or more other learners and/or one or more
educators.
10. The computer system of claim 1, wherein the processor is
configured to compare the determined progress of the at least one
learner with a predetermined outcome of the experiential learning
cycle and automatically trigger an intervention if a result of the
comparison meets predetermined criteria.
11. The computer system of claim 1, wherein the event data
comprises at least one of: a time stamp, a type of event, a virtual
location of the generated event, content information indicative of
the length of the event or sentiment and association
information.
12. A computer implemented method of monitoring progress of at
least one learner through an experiential learning cycle, the
method comprising: providing empirical data indicative of the
experiential learning cycle; facilitating access to the computer
through a communications network; receiving event data at the
computer through the communications network from a user computing
device, the event data being generated by the at least one learner
progressing through the experiential learning cycle; and processing
the received event data and comparing the processed event data with
the empirical data to determine the progress of the at least one
learner through the experiential learning cycle.
13. The method of claim 12, comprising a step of facilitating the
at least one learner to progress through the experiential learning
cycle by providing an education program, wherein information
indicative of the education program is made accessible through the
communications network to the at least one learner on the user
computer device.
14. Software, that when executed by a computer system causes the
computer system to perform the method of claim 12.
Description
TECHNICAL FIELD
[0001] The present disclosure generally relates to a computer
system, a computer implemented method and software for monitoring
progress of at least one learner through an experiential learning
cycle.
BACKGROUND
[0002] Experiential learning is "the process whereby knowledge is
created . . . from the combination of grasping and transforming
experience" (Kolb 1984, p. 41). Experiential learning cycles
describe the flow of grasping and transforming experiences--both
internally and externally. One example of an experiential learning
cycle is the Kolb cycle. According to the Kolb cycle, an external
concrete experience occurs, is internally reflected upon,
internally conceptualised in an abstract manner which leads to
external active experimentation. This triggers the next concrete
experience. As such, the environment external to a learner is as
important a factor in experiential learning as is the learner's
internal mental processes and state.
[0003] Conventional learning assessment and support processes do
not account for the interplay between the external environment and
the learner, which is innate to experiential programs. For example,
a student may present to an industry expert for assessment. The
expert feels the student has demonstrated great understanding and
gives positive feedback, but the student has performed poorly
against specific formal assessment criteria and receives a poor
mark. Resolving this situation from a support and learning outcomes
perspective requires greater context than the formal learning
assessment process collects.
SUMMARY
[0004] A computer system for monitoring progress of at least one
learner through an experiential learning cycle, the computer system
comprising:
[0005] a computer server accessible through a communications
network, the computer server arranged to receive event data through
the communications network from a user computing device, the event
data being generated by the at least one learner progressing
through the experiential learning cycle;
[0006] a memory for storing empirical data indicative of the
experiential learning cycle and event data received at the computer
server through the communications network; and
[0007] a processor configured to process the received event data
and comparing the processed event data with the empirical data to
determine the progress of the at least one learner through the
experiential learning cycle.
[0008] If a result of the comparison meets a predetermined
criteria, the processor may further be configured to automatically
trigger an intervention.
[0009] The computer server may be configured to automatically make
information in relation to the triggered intervention available on
one or more user computing devices through the communications
network. For example, the user computing device may be associated
with the at least one learner or one or more other users, such as
other learners and educators. When an intervention is triggered,
the computer server may automatically generate the information in
relation to the intervention that is communicated through the
communications network to one or more user computing devices.
[0010] The computer server may facilitate at least one learner to
progress through the experiential learning cycle by providing an
education program. In this regard, the computer server may be
configured to make information indicative of the education program
accessible through the communications network to the at least one
learner on the user computer device.
[0011] Alternatively, the computer server may facilitate
communication with a learning platform that provides the education
program to the at least one leaner.
[0012] The experiential learning cycle may comprise a plurality of
learning phases. The processor may be configured to process the
received event data to determine at least one of the plurality of
learning phases of the experiential learning cycle in which the
learner currently is. In addition, the computer server may be
configured to process the received event data to determine a phase
probability that is indicative of a likelihood that the at least
one learner is currently in the at least one determined learning
phase.
[0013] The processor may be configured to process the collected
data to determine a phase velocity of the at least one learner, the
phase velocity being indicative of a pace of the at least one
learner progressing through the plurality of learning phases of the
experiential learning cycle.
[0014] The processor may further be configured to process the
received event data and compare the processed event data with
stored event data to generate prediction data indicative of one or
more future events. The stored event data may comprise historical
event data generated by the at least one learner and/or one or more
other learners and/or one or more educators. The generated
prediction data may then be compared with the empirical data to
evaluate the generated prediction data. If a result of the
comparison meets predetermined criteria, the computer server may be
configured to automatically trigger an intervention.
[0015] The processor may further be configured to match received
event data with generated prediction data. If an event matches a
prediction, the prediction data may be stored as empirical
data.
[0016] The processor may be configured to compare at least one of
the determined learning phase and the determined phase velocity of
the at least one learner with the empirical data to evaluate the
progress of the at least one learner through the experiential
learning cycle. If a result of the comparison exceeds a
predetermined threshold, an intervention may automatically be
triggered by the processor. The processor may further be configured
to use the evaluation of the progress of the at least one learner
to generate a rating for the at least one learner.
[0017] The processor may be configured to compare the determined
process of the at least one learner with a predetermined outcome of
the experiential learning cycle. If a result of the comparison
meets predetermined criteria, an intervention may automatically be
triggered by the processor.
[0018] The event data may comprise at least one of: a time stamp, a
type of event, a virtual location of the generated event, content
information indicative of the length of the event or sentiment, for
example, and association information such as an association to one
or more other learners, an educator, or one or more other
events.
[0019] The experiential learning cycle may comprise an embedded
derivative experiential learning cycle. In this regard, the
processor may be configured to process the received event data to
determine whether the at least one learner is currently in a
derivative experiential learning cycle. In this case, an
intervention may automatically be triggered. The experiential
learning cycle may be associated with one or more additional
experiential learning cycles through which the at least one learner
progresses. In this regard, the processor may be configured to
process the received event data to isolate the cycles and evaluate
the progress and need for intervention independently as well as
collectively.
[0020] In an embodiment, the experiential learning cycle of the at
least one learner may be interdependent with an experiential
learning cycle of at least one other learner.
[0021] The memory may further store data associated with the
results of the determinations, evaluations generations of the
processor.
[0022] A computer implemented method of monitoring progress of at
least one learner through an experiential learning cycle, the
method comprising:
[0023] providing empirical data indicative of the experiential
learning cycle;
[0024] facilitating access to the computer through a communications
network;
[0025] receiving event data at the computer through the
communications network from a user computing device, the event data
being generated by the at least one learner progressing through the
experiential learning cycle; and
[0026] processing the received event data and comparing the
processed event data with the empirical data to determine the
progress of the at least one learner through the experiential
learning cycle.
[0027] The method may comprise a step of facilitating the at least
one learner to progress through the experiential learning cycle by
providing an education program. In this regard, information
indicative of the education program may be made accessible through
the communications network to the at least learner on the user
computer device.
[0028] Alternatively, the method may comprise a step of
facilitating communications with a learning platform that provides
the education program to the at least one learner.
[0029] The experiential learning cycle may comprise a plurality of
learning phases. The step of processing the received event data may
be conducted to determine at least one of the plurality of learning
phases of the experiential learning cycle in which the learner
currently is. In addition, the step of processing the received
event data may further comprise determining a phase probability
that is indicative of a likelihood that the at least one learner is
currently in the at least one determined learning phase.
[0030] In an embodiment of the present disclosure, the step of
processing the collected data may be conducted to determine a phase
velocity of the at least one learner, the phase velocity being
indicative of a pace of the at least one learner progressing
through the plurality of learning phases of the experiential
learning cycle.
[0031] The method may comprise a step of processing the received
event data and comparing the processed event data with stored event
data to generate prediction data indicative of one or more future
events. The stored event data may comprise historical event data
generated by the at least one learner and/or one or more other
learners and/or one or more educators. The generated prediction
data may be compared with the empirical data to evaluate the
generated prediction data. If a result of the comparison exceeds a
predetermined threshold, an intervention may automatically be
triggered.
[0032] The method may further comprise a step of matching received
event data with generated prediction data. If an event matches a
prediction, the prediction data may be stored as empirical
data.
[0033] The method may comprise a step of comparing at least one of
the determined phase and the determined phase velocity of the at
least one learner with the empirical data to evaluate the progress
of the at least one learner through the experiential learning
cycle. If a result of the comparison meets a predetermined
criteria, an intervention may automatically be triggered. The
method may further comprise a step of using the evaluation of the
progress of the at least one learner to generate a rating for the
at least one learner.
[0034] In an embodiment, the method may comprise a step of
comparing the determined process of the at least one learner with a
predetermined outcome of the experiential learning cycle. If a
result of the comparison meets a predetermined criteria, an
intervention may automatically be triggered.
[0035] Information in relation to the triggered intervention may
automatically be made available on one or more user computing
devices through the communications network. For example, the user
computing device may be associated with the at least one learner or
one or more other users, such as other learners and educators. When
an intervention is triggered, the information in relation to the
intervention may automatically be generated.
[0036] The event data may comprise at least one of: a time stamp, a
type of event, a virtual location of the generated event, content
information indicative of a length of the event or sentiment, for
example, and association information such as an association to one
or more other learners, an educator or one or more other
events.
[0037] The experiential learning cycle may comprise an embedded
derivative experiential learning cycle. In this regard, the method
may comprise a step of processing the received event data to
determine whether the at least one learner is in a derivative
experiential learning cycle. In this case, an intervention may
automatically be triggered.
[0038] The experiential learning cycle of the at least one learner
may be interdependent with an experiential learning cycle of at
least one other learner.
[0039] Software, that when executed by a computer system causes the
computer system to perform the method described above.
BRIEF DESCRIPTION OF DRAWINGS
[0040] FIG. 1 is a schematic representation of a system for
monitoring progress of at least one learner through an experiential
learning cycle in accordance with an embodiment of the present
disclosure;
[0041] FIG. 2 is an alternative representation of the system of
FIG. 1;
[0042] FIG. 3 is a schematic representation of empirical data
indicative of an exemplary experiential learning cycle;
[0043] FIG. 4 is a schematic representation of empirical data
indicative of multiple experiential learning cycles;
[0044] FIG. 5 shows a flow chart illustrating a method of
triggering an intervention;
[0045] FIG. 6 shows a flow chart illustrating a further method of
triggering an intervention; and
[0046] FIG. 7 is a flow chart illustrating a method of monitoring
progress of at least one learner through an experiential learning
cycle in accordance with an embodiment of the present
disclosure.
DESCRIPTION OF EMBODIMENTS
[0047] Embodiments of the present disclosure generally relate to a
computer system and a computer implemented method for monitoring
progress of at least one learner through an experiential learning
cycle. In this regard, access to the computer system is facilitated
through a communications network such as the Internet. In a memory
of the computer systems, empirical data indicative of the
experiential learning cycle is stored. The learning cycle may for
example have a plurality of learning phases and be associated with
interdependent experiential learning cycles of other learners.
Event data that is generated by one or more learners is received at
the computer system through the communications network. Event data
may for example comprise a chat message, a post on a group board,
the request for a meeting or the like. This event data is processed
and compared to the empirical data to determine the progress of the
learner through the experiential learning cycle.
[0048] Embodiments of the present disclosure may find application
in any experiential learning situation, such as at university,
school, internships, and within corporations.
[0049] Embodiments of the present disclosure provide significant
advantages. In particular, the computer system may provide an
objective system for monitoring the progress of a learner through
an experiential learning cycle. Furthermore, the computer system
may enable an adequate support system in the form of interventions
to provide feedback to the learners and assist the learners to
progress through the experiential learning cycle.
[0050] For some experiential learning cycles, the educator may not
know what the correct outcome of a learning phase may be. In this
regard, the computer system analyses the event data and determines
the outcome of the experiential learning cycle or at least a phase
of the experiential learning cycle. The rating of a learner would
be automatic and personal emotions of the educator can
significantly be reduced from the process of rating a learner.
[0051] Referring initially to FIG. 1 of the accompanying drawings,
there is shown a computer system 100 for monitoring progress of at
least one learner through an experiential learning cycle. The
computer system 100 comprises a computer server 102 which is
accessible through a communications network, such as the Internet
104, from user computing devices 106, 108, 110. In this example,
the user computing devices 106, 108, 110 include a tablet computer
106, a smartphone 108, and a personal computer 110. However, any
communications enabled computing devices that are capable of
communicating with the computer server 102 are envisaged, such as a
laptop computer or PDA.
[0052] In the present example, the computer server 102 includes a
processor 112 arranged to control and coordinate operations, a
memory 114, and a network interface 116 that communicate with each
other via a bus 118. The network interface 116 facilitates wireless
communications between the computer server 102 and the user
computing devices 106, 108, 110 through the Internet 104.
Specifically, the computer server 102 is accessible by the user
computing devices 106, 108, 110 through web pages served to the
user computing devices 106, 108, 110. This may be realised by
software implemented by the processor 112, and through an
application programming interface (API) that communicates with the
user computing devices 106, 108, 110 using a dedicated application
installed on the user computing devices 106, 108, 110.
[0053] The memory 114 stores instructions 120 and data 122 for the
processes as described in the following, and the processor 112
performs the instructions 120 from the memory 114 to implement the
processes. It should be noted that although the computer server 102
is shown as an independent network element, the computer server 102
may alternatively be part of another network element and functions
performed by the computer server 102 may be distributed between
multiple network elements.
[0054] A representation of an example implementation of the
computer system 100 is shown in FIG. 2, with functional components
of the computer server 102 shown instead of hardware components.
The functional components in this example may be implemented using
the hardware components shown in FIG. 1 such that network
interfaces are provided for facilitating communications with remote
user computing devices 106, 108, 110 and implementing actions in
response to the communications.
[0055] In this example, the computer system 100 comprises a
learning platform 102 that facilitates at least one user to
progress through a learning cycle using a user computing device,
such as the user computing devices 106, 108, 110. For example, the
learning platform 102 may provide an education program including
scheduled tasks and assignments to guide at least one user through
an experiential learning cycle to achieve a predetermined outcome.
Alternatively, the computer system 100 may be in communication with
a learning platform that provides the education program.
[0056] Referring back to the example shown in FIG. 2, the computer
system 100 comprises a control unit 202 for controlling and
coordinating operations of the components of the learning platform
102. This control unit 202 may for example be implemented by the
processor 112 shown in FIG. 1.
[0057] Further, the learning platform has a network interface 204
for facilitating communications through a communications network
104, such as the Internet between the learning platform 102 and a
remote computing device, such as user computing devices 106, 108,
110. In this example, a web server (not shown) of the learning
platform 102 is arranged to serve one or more webpages to the user
computing device, such as user computing devices 106, 108, 110,
that can be accessed through a browser installed on the user
computing device thereby facilitating communications with the
learning platform 102.
[0058] The learning platform 102 is configured to receive at an
event data collector 206 event data that is generated by at least
one learner using a computing device, such as one of the computing
devices 106, 108, 110. Exemplary event data may for example
comprise a chat message, an adjustment of a slider, a submission of
an assessment, and a post on a board. Event data may alternatively
be generated by speech recognition technology and recording of an
audio.
[0059] In this example, the learner enters the event data into a
web page displayed on the user computing device 106, 108, 110 that
is served by a web server of the learning platform 102.
Alternatively, the event data may be received by importing data
from a remote database such as a remote learning platform.
[0060] The learning platform 102 further comprises a data base
management system ("DBMS") 208 that stores data that is received
and/or generated in the learning platform 102 in a data storage
210. It will be understood that the data may alternatively be
stored in a remote database, such as in a cloud storage and can be
received at the learning platform 102 through the Internet 104 via
the network interface 204.
[0061] Empirical data 212 indicative of an experiential learning
cycle may be stored in the data storage 210 and is accessible by
the DBMS 208.
[0062] The Experiential Learning Cycle
[0063] The empirical data may comprise data indicative of an
experiential learning cycle. One exemplary experiential learning
cycle was proposed by David Kolb which will be used as an example
to describe the present disclosure. However, other experiential
learning cycles are envisaged.
[0064] In the Kolb Cycle there are four phases that a learner
progresses through:
[0065] Concrete Experience (CE): the learner undergoes an
experience, often as a result of a direct action from the fourth
phase (AE). This has an internalised impact on the learner;
[0066] Reflective Observation (RO): the learner observes the
consequences of the action and reflects on what was intended and
unintended, as well as the learner's own internal changes in
feeling from the experience;
[0067] Abstract Conceptualization (AC): the learner generalises
from observation and connects this with/seeks out knowledge to
better understand what happened. The learner is generally focused
on what didn't work as intended or how the result can be
improved;
[0068] Active Experimentation (AE): the learner plans for, or sets
up the next set of actions that will trigger a new experience.
[0069] A learner's progression through the experiential learning
cycle may slow down or stop in a phase of the cycle, or the learner
may skip one or more phases. This is particularly likely when there
is a high degree of ambiguity or there are multiple learners that
need to collaborate and therefore move through the experiential
learning cycle together.
[0070] Furthermore, the learner may progress through a plurality of
experiential learning cycles. For example, the experiential
learning cycle may comprise an embedded sub-cycle. For example,
within a phase of the experiential learning cycle an obstacle, such
as a team communication issue or a communication issue with a
teacher, may trigger a new, derivative cycle in which the issue is
worked through and resolved. Embedded sub-cycles may be identified
using statistical analysis of engagement patterns of the one or
more learners compared against engagement patters of learners that
are progressing through the primary experiential learning cycle
only. In particular, the presence of this new derivative cycle may
be identified by collecting a significant divergence in event data
at the event data collector 206 in comparison to predicted event
data generated at the prediction generator 218. Exemplary algorithm
for determining the presence of such derivative cycle may include a
multi-class algorithm and an algorithm using the cross-entropy
method. In this way, event data associated with the primary
learning cycle may be isolated from event data associated with the
derivative cycle. In this example, event data may be generated by
the learners of the team that are indicative of meeting requests
and chat messages with a negative sentiment score which indicates
an issue being worked through. Therefore, a learner may have
several experiential learning cycles happening in parallel, and the
experiential learning cycles may impact each other.
[0071] Outcomes and experiences of an experiential learning cycle
for each learner appear to be highly dependent on the quality and
timeliness of support, for example in the form of an intervention.
Support may be needed to continue or accelerate the progress of the
learner through the experiential learning cycle, or rewind one or
more learning phases of the experiential learning cycle when a
learning phase was skipped.
[0072] In this example, the Kolb cycle is used to construct
computational empirical data of an experiential learning cycle. In
this regard, a probabilistic representation of the learning is
chosen as it is assumed that a learner may rarely be completely in
one phase or another. Furthermore, the different learning phases of
the experiential learning cycle such as the Kolb cycle are only
approximations of a learner's internal state of mind. There are no
externally visible signals that help to identify exactly what is
happening.
[0073] An exemplary probabilistic representation of the
experiential learning cycle including the four above described
learning phases is illustrated in FIG. 3.
[0074] An experiential learning cycle may have a plurality of
characteristics. For example, an experiential learning cycle may
have one or more learning phases where the last phase output is the
input to the first phase. In FIG. 3, the learning phases are
notated P.sub.a, P.sub.b, P.sub.c, P.sub.d . . . P.sub.n and the
entire learning cycle is notated [P.sub.a, P.sub.n] where n is the
last phase.
[0075] As the learner moves from phase to phase, the relative
probability for the learner to be in a given phase will change. In
an ideal experiential learning cycle, the phase probability for a
single phase would be 100% and 0% for the other phases, until the
phase changes, at which point the next sequential phase will show
100% probability and the other phases 0%. The Kolb Cycle, which is
the experiential model used in this example, uses a probabilistic
representation and a learner may appear to oscillate between phases
or skip a phase. Furthermore, measuring the internal mental state
of a learner is imprecise, so practical models will likely show a
distribution of probabilities, e.g. a Gaussian distribution centred
around the most probable phase. In FIG. 3, the probability for the
learner to be in a given phase is notated P.sub.pa, P.sub.pb,
P.sub.pc . . . P.sub.pn.
[0076] The computational model of the experiential learning cycle
supports the interaction of multiple learning cycles. For a single
learner, contemporaneous experiences or complex experiences may
create overlapping primary learning cycles and derivative
sub-cycles that are associated with the primary cycle. With
multiple learners in a collaborative environment, the experiential
learning cycles of one learner may influence the cycles of the
other learners and are therefore interdependent.
[0077] An example of overlapping interdependent learning cycles is
illustrated in FIG. 4. Each phase of the primary learning cycle is
linked to other cycle phases using a mesh of probabilistic vectors.
Values of the probabilistic vectors may regularly be refined by
re-training and thereby improving an underlying machine learning
model with new event data such and data indicative of specific
learning outcomes, amongst others.
[0078] Given the exemplary configuration of probabilities as shown
in FIG. 4, if the learner is in phase P.sub.a of the primary
learning cycle, an algorithm may calculate that there is a 30%
probability that the learner will also be in phase P.sub.a1 of a
derivative cycle and a 10% probability that the learner will be in
phase P.sub.b1 of the derivative cycle. It follows that there is a
60% probability that the learner does not experience the derivative
cycle. Furthermore, FIG. 4 illustrates that at the end of the
derivative cycle, there is an 80% probability that the learner is
in phase P.sub.a of the primary cycle and a 20% probability that
the learner will have progressed to P.sub.b of the primary
cycle.
[0079] These probabilities may be used to determine what event data
is associated with a derivative cycle and at what period of time
this event data is entered so that this event data can be filtered
from an analysis of the primary learning cycle. Furthermore, the
probabilities together with the event data may be analysed to
determine specific circumstances that may trigger a derivative
learning cycle. In addition, the data may be used to determine the
progress of the one or more learners through the primary learning
cycle with and without interventions.
[0080] FIG. 4 further illustrates that if the learner is currently
in phase P.sub.a of the primary cycle, there is a 90% probability
that a collaborating learner will be in phase P.sub.a2 of a
corresponding collaborative learning cycle, and a 10% probability
that the collaborating learner will be in phase P.sub.b2 of the
collaborative learning cycle. If the learner is in phase P.sub.b of
the primary cycle, then there is an 80% probability that the
collaborating learner will be in phase P.sub.b2, and a 20%
probability in P.sub.c2 of the collaborative learning cycle. This
simplified example illustrates how the learning phases of the
learner in the primary learning cycle influence the learning phases
of the collaborating learner. It will be understood that the
collaborating learner may in return influence the learner in the
primary learning cycle. This influence may be different as the
collaborating learner may have a stronger or weaker influence on
the learner in the primary cycle.
[0081] With regard to a collaboration of multiple learners, the
probabilities shown in the simplified diagram in FIG. 4 may be
further processed (for example multiplied back into the phase
probability for each cycle) thereby creating a composite phase
probability that considers the interdependent cycles. The composite
phase probabilities may be notated. This may for example be
accomplished by processing event data of multiple learners using a
phase determination algorithm as described above, or by processing
event data of each individual learner and segregating out
derivative cycles, if present, using a multi-class algorithm. The
probabilities for the primary learning cycle and the probabilities
for the derivative learning cycles may then be combined to
determine composite phase probabilities.
[0082] A further characteristic of the experiential learning cycle
relates to phase velocity which is indicative of the pace of the
learner progressing through the experiential learning cycle. In
light of the probabilistic learning cycle described above, the
phase velocity in this example relates to a change in probabilities
for each phase over time. The overall cycle velocity is a composite
of the different phase velocities, which can be a simple delta
between the current and previous most probable phases, or a more
complex formula taking the probabilities of all phases into
account, and also any overlapping and/or associated cycles. In FIG.
3, the phase velocities for given phases is notated P.sub.va,
P.sub.vb, P.sub.vc, P.sub.vd . . . P.sub.vn.
[0083] A further characteristic of the experiential learning cycle
relates to the expected progress of a learner through the
experiential learning cycle. Specifically, an expected progress
maps one or more experiential cycles to a timeline wherein the
timeline represents which phase of the experiential learning cycle
the learner is expected to be at, at a given point in a learning
experience. In structured learning programs, this expected progress
may be predefined based on a program duration and scheduled
activities. In flexible or unstructured programs the expected
progress may be determined by machine learning from historical or
current data such as data from other, similar learners. The
expected progress may also be determined procedurally based on past
events and parameters. The historical data may include learning
outcomes such as learner satisfaction, grades, cycle stage
estimates, that may be manually or automatically determined. This
data may be used to train a prediction algorithm, such as a
decision forest multi-class classifier algorithm. For example, a
Kolb cycle for a specific learning experience might be defined as
follows--phase CE x days, phase RO 2.times. days, phase AC 3.times.
days, and phase AE 2.times. days. In other words, the duration of
the time spent in the first CE cycle sets the "pace" for the
subsequent cycles, where the RO cycle takes 2.times. as long, AC
3.times. as long, AE 2.times. as long. It is possible that a
combination of these and other methods for specifying expectation
are used. In FIG. 3, an expected process is notated Ex.sub.1,
Ex.sub.2, Ex.sub.3 . . . Ex.sub.n.
[0084] Event Data
[0085] When event data is received at the event data collector 206
of learning platform 102, the DBMS 208 of the learning platform 102
stores the event data in an event database 214. The event data is
further processed and compared to the empirical data in the
empirical database 212 to determine the progress of the learner in
the experiential learning cycle. For further processing the event
data, the data storage 210 further stores one or more algorithms in
an algorithm database 216.
[0086] Event data relates to events that may be entered by the
learner on the user computing devices 106, 108, 110 and received
through the Internet 104 at the learning platform 102. Event data
represents the collected and derived data associated by a learner
or learning system action. In FIG. 3, events are notated E.sub.1,
E.sub.2, E.sub.3 . . . E.sub.n as shown in FIG. 3.
[0087] One particular example of event data relates to chat
messages. In this regard, the learning platform 102 would
facilitate communications between the learner on a user computing
device such as user computing device 106, 108, 110 and one or more
other learners and/or educators. The event data collector 206 of
the learning platform 102 automatically collects event data in the
form of messages between the users. An example of chat messages
between a group of users is provided below:
TABLE-US-00001 Bob .fwdarw. Alice (12:00pm): Hi Alice, how is the
assignment going? Event: E.sub.1 Timestamp = 2015-01-01 12:00:00
Classes [ C.sub.1 = AE , C.sub.2 = Chat ] State [User = `Bob,
Program = X101] Data [ "Hi Alice, how is the assignment going?" ]
Contexts [ ] Scores [ M.sub.1 = 4, M.sub.2 = .5 ] Alice .fwdarw.
Bob (12:01pm): It's going well... but I need some help. Do you know
how to do machine learning? Event: E.sub.2 Timestamp = 2015-01-01
12:01:00 Classes [ C.sub.1 = AE , C.sub.2 = Chat ] State [User =
`Alice, Program = X101] Data [ "It's going well... but I need some
help. Do you know how to do machine learning?" ] Contexts [E.sub.1]
Scores [ M.sub.1 = 7, M.sub.2 = .8 ] Bob .fwdarw. Alice (12:02pm):
no, but check with the rest of the team. Event: E.sub.3 Timestamp =
2015-01-01 12:02:00 Classes [ C.sub.1 = AE , C.sub.2 = Chat ] State
[User = `Bob, Program = X101] Data [ "no, but check with the rest
of the team." ] Contexts [E.sub.1, E.sub.2] Scores [ M.sub.1 = 4,
M.sub.2 = .3 ] Alice .fwdarw. Team (12:03pm): Hi team, does anyone
know machine learning? Event: E.sub.4 Timestamp = 2015-01-01
12:02:00 Classes [ C.sub.1 = AE , C.sub.2 = Chat, C.sub.3 = Group ]
- added a group class because of team chat State [User = `Alice,
Program = X101] Data [ "Hi team, does anyone know machine
learning?"] Contexts [E.sub.1, E.sub.2, E.sub.3] - the previous
messages because it directly follows the chat and has a keyword
(bold). Scores [ M.sub.1 = 4, M.sub.2 = .5 ] Carol .fwdarw. Team
(3:00pm): Yes, I know machine learning - I did two units on it last
year. What do you want to know? Event: E.sub.5 Timestamp =
2015-05-27 15:00:00 Classes [ C.sub.1 = AE , C.sub.2 = Chat,
C.sub.3 = Group ] - added a group class because of team chat State
[User = `Carol, Program = X101] Data [ "Yes, I know machine
learning- I did two units on it last year. What do you want to
know?" ] Contexts [ E.sub.2, E.sub.4 ] - the previous message
because it appears to be a response, E2 because of the keyword.
Scores [ M.sub.1 = 8, M.sub.2 = .7 ]
[0088] Event data, such as the chat messages above, may be received
in binary or textual form and a number of characteristics of the
event data may be derived from further processing the event data.
Exemplary characteristics may include one or more of the
following:
[0089] Timestamp: indicative of when the event occurred or was
entered by the learner;
[0090] Content data: indicative of the raw data associated with the
event. In the example of the chat messages, the content data may
relate to the message itself;
[0091] Type: indicative of the type of the event, exemplary types
may include: chat, post, email, phone call, set sliders, upload,
attend meeting/seminar/workshop, schedule meeting. The type of the
event may be weighted indicative of the probability that the event
is of the associated type;
[0092] State: the "state" of the system or data source at the time
of the event generation. The state might include information about
the learner, such as details about the particular education program
that they are accessing at the time of the event.
[0093] Context data: indicative of references to other events
and/or users. For example, a chat event where user A sends to user
B may reference the prior message from B to A and the prior message
from A to B. Other chat messages with similar keywords may be
referenced, as well as non-chat events that are considered
related;
[0094] Phase: indicative of a phase of the experiential learning
cycle in which the user currently is, the phase may be weighted
representing a probability that the user is in the determined
phase;
[0095] Class: indicative of a label given to an item or set of data
to show association with other data. A classifier is an algorithm
that applies classes to the event data. Each phase, expected
progress, algorithm and outcome may have associated
representational class labels. For example, a phase may be part of
a class label. In FIG. 3, the class is notated C.sub.1, C.sub.2,
C.sub.3 . . . C.sub.n. It should be noted that other classifiers
may be used to link data that is not formally associated. For
example, tags or keywords or sentiment data may be extracted from
event data, such as communication data between learners for further
analysis.
[0096] Score: indicative of characteristics of the content of the
event. For example, for a chat message, a score may be derived that
is indicative of length, sentiment, and semantic score. A score may
for example be: `length`: 30, `sentiment`=>+0.9,
`semantic`=>0.5 representing a 30 character message with
positive sentiment and where 50% of the words are semantically
meaningful.
[0097] Besides chat events, there are multiple other types of event
data that the learning platform can receive or generate. The
following table illustrates further examples of event data together
with associated phases of the experiential learning cycle.
TABLE-US-00002 Phase Role Class Event Action Component Class CE Add
Idea/Concept/Post Collaboration Capture CE Notify Idea Followers
Collaboration Trigger CE Ideation Metrics Analytics Report CE
Deliver/Submit Assessment Capture CE Notify Submission Status
Assessment Trigger CE Present Assessment/Video Pitch Capture CE
Peer Review Assessment/Video Pitch Trigger CE Take Quiz Assessment
Capture CE Quiz Results Assessment Trigger RO Give Feedback
Assessment Capture RO Get Feedback Assessment Trigger RO Give
Feedback Assessment Capture RO Get Feedback Assessment Trigger RO
Do Reflection Assessment Capture RO Reflect Sentiment Report
Assessment Trigger RO Set Sliders Stats Capture RO Slider Metrics
feedback Stats Trigger RO Ideation/project comment Collaboration
Capture RO Notify comments Collaboration Trigger AC Track
Communication Comms Capture AC View Communication Comms Trigger AC
View Lesson Content/Project Capture AC Recommend Lesson
Content/Project Trigger AC Earn Achievement Achievement Capture AC
Show Locked Achievements Achievement Trigger AC Ask Question
Collaboration Capture AC Browse Questions Collaboration Trigger AC
Attend Workshop Project Capture AE Have chat/video chat Chat
Capture AE new chat notification Chat Trigger AE Add new
task/Complete Task/Project Capture AE See tasks & completion
Task/Project Trigger AE Schedule Meeting Events Capture AE See
upcoming meetings Events Trigger
[0098] Algorithms
[0099] One or more algorithms may be stored in the algorithm
database 216 of the learning platform 102. The one or more
algorithms may perform calculation, data processing and automated
reasoning of the received event data.
[0100] For example, an algorithm, such as a decision forest
algorithm or a decision jungle algorithm for creating a multi-class
classification, may be applied to the event data to determine the
progress of the learner through phases of the experiential learning
cycle. The phases may be represented as classes in the algorithm.
An alternative algorithm may use the concept of multilayer
convolutional network to determine a probability associated with a
phase of the experiential learning cycle. This approach may be
useful when multiple learning cycles are in effect, for example due
to a plurality of learners interacting with each other, or a single
learner progressing through overlapping or derivative learning
cycles.
[0101] In a simplified example, the type of the event data may be
determined and compared with the table provided above to determine
the phase in which the learner currently is.
[0102] Further, a score of the event data may be determined by
applying an algorithm to the event data. For example, in the case
of event data relating to a communication between two learners, a
level of conflict between the two learners may be scored (for
example between 0 and 1), as well as a sentiment score of the
communication message between the two learners. For example, the
sentiment may be within a range between -1 and 1. These scores may
be generated using a two-class classifier algorithm, such as
Support Vector Machine or Decision Forest. In FIG. 3, the score is
notated M.sub.1, M.sub.2, M.sub.3 . . . M.sub.n, in which "M"
stands for metrics. A Metric-Class association is notated M1[C1,C5]
to show that Metric 1 can be applied to Events of Class 1 and Class
5.
[0103] By processing the event data, a learning outcome may be
determined, notated for example O.sub.1, O.sub.2, O.sub.3 . . .
O.sub.n. In this regard, the progress of a learner may be
determined against relative to an outcome by applying one or more
algorithms. Predefined learning outcomes may be used for training
machine learning algorithms, such as supervised machine learning
algorithms. For example, event data or a subset thereof may be
associated with a grade for a particular learner. Event data may
also be associated with an indication from a satisfaction survey.
If there are several outcome measurements throughout a learning
cycle, for example as a result of interim surveys or external
audits, these measurements may be used to label event data that is
fed into the machine learning algorithm. In this way, the machine
learning algorithm can be trained to predict learning outcomes for
another learner. This can also provide an estimate of the progress
of the learner against the outcome. A given experiential learning
program may have specific targets or estimated percentages/rates of
improvement for learning outcomes.
[0104] Processing of Event Data
[0105] The event data received at the event data collector 206 may
be processed and compared with the empirical data that is stored in
the empirical database 212 to determine the phase in which the
learner currently is.
[0106] For example, the event data may be processed to determine a
class associated with the event data, the class may be associated
with a phase of the experiential learning cycle. A series of
algorithms may be run against the received and/or stored event data
to determine additional classes and/or phases or potentially remove
associated classes or phases. Specifically, in one exemplary
implementation time stamps may be used to align event data and
empirical data thereby creating additional event features with the
empirical data. A selection of windows of combined event and
empirical data may be labels with experiential phases. The labelled
data may then be fed into a decision forest to train a multiclass
decision forest model. The algorithm may be used to predict the
class or phase for the learner's current event data window and
available empirical features.
[0107] Further, association information indicative of an event
context may be determined by running an algorithm against the
received event data. For example, the information may be
pre-determined by the computer system 100, or could be dynamically
determined by matching classes. A simple match of predetermined
criteria would look for exact matches of class lists and weights. A
more sophisticated matching algorithm may use event intervals
and/or similarity thresholds to determine a maximum distance
between differing class lists which would be considered part of the
same context. For example event 1 has associated classes C1 (100%),
C2 (50%). Event 2 has associated classes C1 (100%), C2 (25%).
Predetermined criteria of an exact match algorithm would not relate
event 1 to event 2 as in being in the same context. However, a
similarity threshold algorithm may determine that events 1 and 2
have been generated in the same context.
[0108] One or more algorithms may be associated with a predefined
outcome of the experiential learning cycle. In this regard,
outcomes of the learner may be updates and/or predicted outcomes
for the learner may be determined.
[0109] Predictions
[0110] The event data received at the event data collector 206 of
the learning platform 102 may further be processed and compared
with event data that is stored in the event database 214. In this
way, prediction data indicative of one or more predictions of an
event in the future may be generated at a prediction generator 218.
A prediction may represent a collection of events that might occur
in the future. The stored event data may comprise historical event
data generated by the learner and/or event data generated by one or
more other learners.
[0111] Prediction data may be used to train semi-supervised machine
learning algorithms, such as decision forest algorithm or
multilayer convolutional matrix algorithm, and unsupervised machine
learning algorithms, such as k-means clustering algorithm, that may
be stored in the algorithm database. These algorithms may relate to
the ones that generate the prediction data, phase determination
algorithms, algorithms associated with the scores, algorithms that
determine an expected progress, or algorithms that trigger an
intervention, and what type of intervention would be necessary.
Exemplary algorithms for generating prediction data may relate to
the Bayesian logic, linear extrapolation and pattern analysis.
[0112] Prediction data may be generated having one or more of the
following characteristics:
[0113] Timestamp: indicative of when the prediction data was
generated;
[0114] Half-life: indicative of a rate of decay of the prediction.
For example, if a prediction is generated for the response to a
chat event, the prediction data may have a short half-life as past
a few hours the next chat message is less likely to be part of the
same conversation;
[0115] Phase: indicative of a predicted phase in which the learner
will have progressed to; the phase may be weighted by the
probability that the prediction will apply to a given phase;
[0116] Class: indicative of a predicted class; the class may be
weighted by the probability that the prediction will apply to a
given class;
[0117] Score: indicative of value for each of the algorithms that
are used to generate the prediction data; for example each of the
one or more future events may be predicted using a respective
algorithm;
[0118] Match: indicative of events that match the prediction.
[0119] The prediction data generated by the prediction generator
218 may then be compared with the empirical data stored in the
empirical database 212, such as an expected progress of the
learner, to evaluate the generated prediction data. If a result of
the comparison meets predetermined criteria, such as a
predetermined threshold, an intervention may automatically be
triggered at an intervention trigger determiner 220 (see for
example FIGS. 5 and 6 as described in further detail below).
[0120] Furthermore, the prediction data may be filtered based on
matching classes, half-life, interval or the like. For each
predicted score, a percentage match may be determined relate to the
event score. If the aggregate match percentage is greater than a
threshold, the event data may be added to the list of prediction
matches. This information may be used to optimise the prediction
algorithm by rewarding the parameters that led to the successful
prediction.
[0121] Using the previous example of the chat messages within the
group, the following prediction data may have been generated at the
prediction generator 218 of the learning platform 102 as a result
of Alice's question to the group:
TABLE-US-00003 Prediction: Pr.sub.1 - made by AI routine A
Timestamp = 2015-05-27 12:02:00 Classes [ C.sub.1 = AE (80%) ,
C.sub.2 = Chat (100%), C.sub.3 = Group (80%) ] Halflife = 36000
seconds = 10 hrs Scores [ M.sub.1 = 2, M.sub.2 = .4 ] - implies a
short, neutral to negative answer (e.g. no, sorry, I don't) Matches
[ ] Prediction: Pr.sub.2 made by AI routine B Timestamp =
2015-05-27 12:02:00 Classes [C.sub.1 = AE (90%) , C.sub.2 = Chat
(100%), C.sub.3 = Group (80%) ] Halflife = 36000 seconds = 10 hrs
Scores [ M.sub.1 = 7, M.sub.2 = .8 ] - implies a longer, positive
answer Matches [ ]
[0122] When Carol responds with event E.sub.5, it matches the
second prediction because it has the predicted phases, is within
the half-life of the prediction and the scores are within a 10%
threshold. This is a relatively high match. As a result, the
prediction algorithm AI routine B that led to the generation of the
prediction data may be rewarded. The reward would increase the
probability of the AI routine being used for future predictions
with chat messages, either generically or just for Alice. Such
reward will be captured by storing the increased probability
associated with the AI routine B in the algorithm database.
[0123] Furthermore, with this prediction matched, the phase
probability of the users Alice and Carol for the AE phase would be
increased, as the chat event is associated with that phase.
[0124] If an outcome of the experiential learning cycle relates to
the collaboration of multiple learners, and an algorithm that was
used to determine the outcome was the amount of positive sentiment
group chat, then the collaboration outcome for both learners Alice
and Carol would also likely be increased as they had an exchange
that was positive in nature, as measured by the event scores.
[0125] Interventions
[0126] Interventions are phase appropriate actions that may assist
the learner in progressing through the experiential learning cycle.
For example, interventions may be used to accelerate the progress
through the experiential learning cycle, or to repair stalled or
otherwise dysfunctional cycles. Interventions may comprise
feedback, information, support, and resources.
[0127] A particular example of an intervention relates to a
customised reflection survey. This may be implemented to gather
additional information from a learner, for example in relation to
issues affecting the progress through the experiential learning
cycle. Questions in the reflection survey may be determined based
on a phase of the learning cycle, or a particular event triggering
such intervention.
[0128] In accordance with embodiments of the present disclosure,
there are a number of situations in which an intervention may
automatically be triggered at the intervention trigger determiner
220 of the learning platform.
[0129] A number of examples for events that may automatically
trigger an intervention are provided below:
[0130] Communication Breakdown: a lack of communication between
learners or a learner and a mentor may automatically trigger an
intervention. For example, a break down score indicative of the
lack of communication between learners and/or mentors may be
determined. This score may be combined with a corresponding
sentiment score indicating dissatisfaction between the learners or
the learner and the mentor. To identify this event data that
automatically triggers an intervention, frequency and distribution
of communication and collaboration events among and between
learners and/or mentors may be measured. These measurements may be
compared with other learners and/or mentors to normalise the data.
For example, if a learner A and a learner B are communicating on a
regular basis, and learner B and learner C are also communicating
on a regular basis, but learner A and learner C are not
communicating, an intervention may be triggered based on the lack
of event data between learner A and learner C. In addition, the
learner A or learner C may report dissatisfaction with the progress
through the experiential learning progress for example by
submitting a complaint to a mentor, or is not engaging in other
activities such as group activities. This may further increase the
breakdown score. If learners are given low feedback from a mentor
or supervisor, the break down score may further increase. Once the
score exceeds a predefined threshold, an intervention may
automatically be triggered at the intervention trigger determiner.
The predefined threshold may be based on an analysis of historical
event data and learning outcomes.
[0131] When an intervention is automatically triggered at the
intervention trigger determiner based on communication breakdown,
learners and/or mentors may automatically receive a survey that
includes question about how dynamic issues amongst the learners
and/or mentors are being addressed. A further intervention may be
triggered if communication between the learners and/or mentor does
not improve within a predefined period of time, such as a few days.
The further intervention may relates to a meeting with a
mentor.
[0132] Cycle Alignment: The current or future expected progress of
an experiential learning cycle is not aligned with the current or
future phase or phase velocity of the learner through the cycle. By
comparing at least one of the determined phase and the determined
phase velocity of the learner with the empirical data, the progress
of the learner can be evaluated. If a result of this comparison
meets predetermined criteria, such as exceeds a predetermined
threshold, an intervention may automatically be triggered at the
intervention trigger determiner. This comparison may be conducted
for historical, current and predicted phases and phase velocities
for an experiential learning cycle.
[0133] For example, if the prediction data for a learner is
associated with the CE phase, however the learner's expected
progress is in the RO phase, this may be an indication that the
learner's progress has stalled. An intervention event would be
triggered at the intervention trigger determiner 220 and
information in relation to the triggered intervention may be
generated, such as the prediction data and expected progress. This
information may be communicated through the Internet 104 to a user
computing device, such as user computing devices 106, 108, 110. The
intervention may have no outward manifestation, as it may only set
parameters within the learning platform 102, such as in event data
collector 206. These parameters may influence a threshold for
triggering an intervention at the intervention trigger determiner
220. Additionally or alternatively, it may result in an
implementation of a data presenting or data gathering interface to
one or more learners, educators or mentors. For example, an
educator may receive a notification that identifies a probability
of a stalled learner. Alternatively, the intervention may be in the
form of asking the learner to do a specific task, such as
conducting a pre-determined activity designed to help develop a
necessary skill. The intervention may also ask one or more learners
for collaboration, and data associated with that collaboration can
be captured to track outcomes. This collaboration may form a
derivative learning cycle.
[0134] The generated information in relation to the triggered
intervention will generate event data which will be received at the
event data collector 206 of the learning platform 102. In other
words, a feedback loop is generated. In accordance with the steps
of generating prediction data at the prediction generator 218, the
intervention event data may also be used to predict future
events.
[0135] Outcome Improvement: one or more outcomes or predicted
outcomes may be decreasing or not increasing at an appropriate
rate.
[0136] If an outcome has a target goal or rate of improvement, this
may be compared against outcomes history and predicted process. For
example, in the example of the chat messages above, if Alice and
Carol had a protracted negative interaction, both of their
collaboration outcome metrics would have decreased. If
predetermined criteria are met, such as the decrease was greater
than a predetermined threshold, an intervention event may be
automatically triggered at the intervention trigger determiner 220.
This intervention event may be communicated to the educator
prompting the educator to organise a meeting to have a conversation
with both Alice and Carol and trying to help them achieve a more
positive dialogue. Additionally or alternatively, the intervention
event may alter the user interface for Alice and Carol when they
are communicating with each other through the learning platform
102. For example, communication messages between Alice and Carol
may be run through sentiment analysis software and delay or stop
messages being sent if the sentiment score is below a predefined
threshold.
[0137] Predictability Shift: A substantial shift in the
predictability of the event data generated by a learner.
[0138] One or more algorithms stored in the algorithm database 216
of the learning platform 102, such as machine learning algorithms,
will eventually converge on parameters that maximise the ability to
predict future events based on received event data. However, if
predicted future events change beyond an expected amount, it could
indicate a shift in the learner's internal state. Lowered
predictability may indicate that the learner is stressed out or
confused, heightened predictability may indicate boredom,
disengagement, or gaming the system.
[0139] Using the example of the chat messages above, if the
prediction generator 218 generated the correctly predicted
responses for Alice for several days but then was unable to predict
a single response because there were no appropriate responses
within the half-life period, an intervention may be triggered at
the intervention trigger determiner. The intervention may be in the
form of an educator assessing if the team was disengaged. Further
reasons for this may be that an exam period or school break may not
have been taken into account. In the latter case, the educator may
generate event data indicating that the triggered intervention
event was incorrect. This may be used as a label for training the
machine learning classification algorithms that are fed into the
intervention trigger determiner 220. This may further be stored in
the algorithm database 216 to recognise vacations and other
breaks.
[0140] Referring now to FIGS. 5 and 6, there is shown exemplary
flow diagrams 500, 600 illustrating workflows for automatically
triggering an intervention. The workflows may be implemented via a
decision tree system based on historical data.
[0141] Referring initially to FIG. 5, there is illustrated three
workflows based on three different scenarios. The first scenario
502 relates to whether an assessment has been submitted within a
predefined period of time, for example by a submission deadline. If
the assessment is received after the submission deadline 504, event
data is collected 506 at the event data collector 206. The
collected event data in relation to the submission of the
assessment may be used to determine a reliability score for each
learner associated with the learner's performance on tasks that are
time bound. The reliability score may be generated by training a
machine learning algorithm on event data and labelled accordingly,
for example with early, on time or late. The reliability score may
alter a predefined threshold for triggering an intervention, for
example learners with a low reliability score may receive more
frequent reminder notifications.
[0142] If the assessment is not received within a predefined time
period 508, such as a few days after the submission deadline, an
intervention in the form of an email reminder may be automatically
triggered 510.
[0143] The second scenario 512 relates to the amount of learning
content that is consumed by a learner. This scenario may not result
in an outward manifestation and data in this regard may be captured
514 solely for determining learning outcomes, predicted data or
outcomes of interventions. A positive intervention outcome may be
determined if, after automatically triggering an intervention, the
associated learning content was consumed or increased.
[0144] The third scenario 516 relates to learner activity. In one
example, learner activity may be measured by collecting login data
of a learner 518. In a further example, learner activity may be
measured based on level of involvement and collaboration. If it is
determined that a learner is inactive, for example by not having
logged into the learning platform 102 within a predefined period, a
reminder notification may be automatically triggered. For example,
all inactive learners may receive reminder notifications at fixed
periods. Alternatively, all learners may receive a notification
indicative of an activity score. Content, timing and frequency of
reminders may vary based on a learner's historic activity score,
current activity score and the activity score of any
collaborators.
[0145] Referring now to FIG. 6, there is shown a flow diagram 600
illustrating exemplary workflows for learners in a team. The flow
diagram is broken into two categories, a first category 602
relating to interventions that are triggered due to interactions or
lack of interactions between learners, and a second category 604
relating to interventions that are triggered due to interactions or
lack of interactions between one or more learners and a mentor or
educator.
[0146] In a first two scenarios 606, 608 of the first category 602,
learners are actively engaging with a facilitator or using a system
tool for directly reaching out for help. This data is captured and
may be used for training an algorithm or validating/invalidating
prediction data. This scenario demonstrates 100% confidence for a
need of an intervention.
[0147] The next three scenarios 610, 612, 614 of the first category
602 relate to team collaboration. Interventions may be triggered
based on feedback response to a regular reflection survey which
asks the learners to rate the stage or phase of their team
collaboration. For example, Tuckman's model of team dynamics may be
used which includes the stages Forming, Storming, Norming and
Performing. Different interventions may be generated depending on
whether a majority of the team is in agreement (for example all
learners within the team indicate that the team is in the Storming
stage), whether there is relatively high dissonance among learner's
perceptions (for example indications of the stage for the team are
significantly different phases), or whether most learners are in
agreement but one learner is in stark disagreement. The latter
situation may further be cross referenced with event data to
determine the nature and frequency of the intervention.
[0148] Interventions in the second category 604 may be triggered
based on a series of scenarios: the result of regular reflection
surveys provided to mentors 616, a mentor reaching out to an
educator about their learners 618, an educator recording an
observation 620, or a student reaching out to the educator about a
mentor 622. In this particular example, the first scenario 616 is
handled similarly to the learner only reflection survey, where the
activity of learners determines the particular intervention. The
other scenarios 618, 620, 622 may be tracked as direct requests for
intervention and are used to validate/invalidate prediction
data.
[0149] Referring now to FIG. 7, there is shown a flow chart
illustrated a computer implemented method 700 of monitoring
progress of at least one learner through an experiential learning
cycle. In an initial step 702, empirical data is provided
indicative of a learning cycle. In a further step 704, access to
the computer is facilitated through a communications network, such
as the Internet. At the computer, event data is received at step
706 through the communications network and generated by the user at
a user computing device. The received event data is then processed
in step 708 and compared with the empirical data to determine the
progress of the learner through the experiential learning
cycle.
[0150] It will be appreciated by persons skilled in the art that
numerous variations and/or modifications may be made to the
above-described embodiments, without departing from the broad
general scope of the present disclosure. The present embodiments
are, therefore, to be considered in all respects as illustrative
and not restrictive.
[0151] KOLB, D. A. et al, Experiential learning: experience the
source of learning and development (1984).
* * * * *