Behaviour Analytics is one of three Moodle plugins designed and developed within the Personalized Study Guide research project that is intended for extracting sequential behaviour patterns of students from course access logs. Behaviour Analytics considers all the activities on a course page as nodes in a graph. The links between nodes are the student accesses of those activities. Each student then has a centroid point derived from their accesses to activities and the coordinates of the nodes. The student centroids can be clustered to group students and find common access patterns. The nodes of the graph can be manually positioned and/or removed from the graph, which will affect the student centroids. When students create new data for the system, the clustering results get updated and can be replayed to visually verify the grouping remains correct with the addition of the new data. Incorrect groupings can be manually altered. The plugin is intended for teacher use and will not be seen by students.

If the resource nodes are not configured prior to viewing the graph, the nodes are given automatic positions. The view interface consists of the graph, a menu of student groups, a menu of students, and a time slider. The group and student menus allow selection of which students to view on the graph. The time slider allows selection of which links to view. All users have their time start at access 1, so not all students will have links at the far end of the time slider. The two handles control the slice of time that is viewed. Hovering over a node will produce the name of the associated resource and a preview of the activity in an interactive window. Moving the mouse outside the preview removes it.

There is a button labelled "Cluster" above the student menu that moves from the graph viewing stage to the clustering stage. Clicking the button will show the same graph and student links, but with each student centroid denoted by a triangle. There are two radio buttons at the bottom left that determine the centroid type, which are weighted geometric (default) and decomposed. The slider in the clustering interface controls the stages of clustering. Moving the slider to the second position removes the graph and scales the student centroids to the edge of the viewing area. The third position on the slider allows clustering to be performed. A text box takes the number of clusters to use or the default of 3 will be used. The k-means clustering can then be stepped through one iteration at a time with the step button or it can be played to convergence by using the play button. The reset button will reset the clustering stage, clearing the current clustering results. All the clustering results are logged in the right side panel, which shows which cluster each student belongs to.

During clustering, student centroids and clustering centroids can have comments added to them. Clicking a centroid will bring up a comment box to record any notes about the cluster or its members. Hovering over a student centroid will show that student's graph. Hovering over a clustering centroid will show a graph of common links among members of that cluster. Unlike the student graph, the common links graph will remain visible until the mouse is clicked outside the graph. While the common links graph is visible, hovering over a node will produce a preview of that node and hovering over a link will produce previews of both nodes attached to that link.

During clustering, it is also possible to assist the clustering algorithm by dragging and dropping student centroids. When a student centroid is dragged away from its original location and dropped elsewhere, the clustering centroid closest to where the student was dropped will have that student included in the cluster. This feature can assist the clustering algorithm when its results are not quite what is desired based on the user's perception of the visualization. The clustering algorithm will still need to run again and may override the manual clustering at this point.

If no clustering has been done, only the interface is shown. The interface consists of a menu of clustering runs sorted by user, graph configuration, and clustering run. Selecting an item from the menu will bring up the same graph shown at the onset of clustering. The play and step buttons control the replay and determine which iteration is seen. The replay can be stepped through forward or back, but can only be played forward. Clicking a centroid will allow notes to be added or viewed if any comment was made for that centroid during clustering.

When a clustering run has used the full time slice of the time slider and also run to convergence, that clustering run will be updated with new student data when it is available. This updating will change student centroids, add iterations to the replay, and can alter the cluster membership. Using decomposed centroids causes the centroid to align on a node and can make the student centroid highly mobile. Geometric centroids tend to move less as there is more data.

Manual clustering is possible during the replay, but behaves slightly differently than during clustering. Once the replay has reached initial convergence, student centroids can be dragged and dropped to place them in another cluster. The manual clustering produces a second set of clustering centroids that are the same colour as the originals. The original set is unchanged while the manual clusters are more transparent and represent the changes the user has made. Both sets of clusters are updated with new data as it is available.

The interface includes the graph of all course activities, a weight slider, and a hierarchical legend of the nodes. Researchers will also have a menu of the other user's graph configurations which they can view, but not alter. The weight slider controls the link weights where positive values produce nodes that pull together and the negative value will push the nodes apart. A value of zero causes all the nodes to remain stationary. The nodes can then be dragged into position. Unwanted nodes can be removed by right clicking and choosing the remove option or by unchecking the associated box in the hierarchical legend. Hovering over a node will show the type and name of the activity as well as bring up an interactive preview. Moving the mouse outside the preview makes it disappear.

First, the LORD plugin must be installed in Moodle. Then, there is an administrator option to allow the integration. With the administrator option enabled, an extra link will appear in the block to configure the integration. There are 2 settings available from within the block. The first causes Behaviour Analytics to use the latest graph made in the LORD plugin. The second switches between using the system generated graph and the user manipulated graph. If there is no LORD graph to use, the graph made with Behaviour Analytics will be used instead. If activities have been removed from or added to the course since the LORD graph was last generated, the graph will need to be generated again before use.

When using the LORD generated graph, it is possible to manipulate the nodes using the "Configure resource nodes" feature. The graph will appear slightly different, as it will have grouping nodes and their associated links. If a LORD graph is manipulated within Behaviour Analytics, it will cease to be a LORD graph. This means that LORD integration must be switched off to use the newly manipulated graph. Such a graph will have all the nodes in the correct positions, but the links will appear differently, as there will no longer be links between nodes.

The Learning Object Relation Discovery (LORD) plugin is designed to discover the relationships between learning objects. This is done by first extracting the text content of each learning module in the course. The text content is then broken down into sentences so that each sentence can be compared with each other sentence. The comparison is done using a Web service (https://ws-nlp.vipresearch.ca/) built for this purpose. The Web service relies on WordNet (https://wordnet.princeton.edu/) to determine the similarity between words in the sentences, and as such, the plugin is currently limited to use with the English language. A final similarity value is assigned between each pair of learning objects, which is then used as a distance measure between the nodes in a network graph, where the graph nodes represent the learning modules. This process is designed to generate a graph configuration where similar nodes are grouped together. The generated graph can then be used with the Behaviour Analytics plugin, which uses the same learning object network graph for analyzing student interaction with the course material.

The plugin will not start the relation discovery process by default, so the plugin must be turned on after installation. This is done as Administrator from the plugin's global setting Site administration -> Plugins -> Plugins overview -> Settings (for LORD). But, this only turns the plugin on globally and for each course the plugin is installed in, the plugin must also be turned on there. The block itself has a link to its instance settings, the first of which is to start and stop the discovery process for that course.

With the discovery process activated both globally and locally, the extraction of learning content and the sentence by sentence comparisons are done through the Moodle scheduled task interface. The comparison process can be quite lengthy and time consuming, so it is run in the background. The block shows a progress table listing the number of learning objects in the course, the number of connections between learning objects, the number of connection analyzed so far, and the number of errors encountered.

Even before the relation discovery process is complete, it is possible to view the network graph of the learning objects. Until the discovery process completes, however, the similarity between nodes is not known, so the graph is not fully configured. When first viewed, there is no graph generated, so this will be done automatically to become the system generated graph. It is then possible to drag the graph nodes around to reorganize the graph, which becomes the custom user manipulated graph. There is a button to switch between the custom and system graphs. There are also 3 slider controls that determine the minimum and maximum distances between nodes, as well as the scaling factor used in converting the similarity score to a distance. These 3 sliders can be adjusted and the graph can be regenerated with the associated button. There is also a checkbox to allow or disallow changes to be made to the graph, which is used to keep the graph static when clicking nodes to view content and similarity.

Left-clicking on a graph node will show the data for the associated learning object. Right-clicking on another node will show the data for both nodes and the similarity between the 2 learning objects. The similarity is shown as a final value at the top of the data table with further data about the similarity following in the rest of the table. Each similarity calculation is shown for the words between each sentence, as well as the calculations between sentences and paragraphs. There can be quite a bit of data shown, but there are 2 basic types of tables shown. The first is a matrix of words comparing one sentence against another, where the intersection represents the similarity between words. The optimal assignment of values is used to determine the similarity between sentences. The second is a cost matrix that shows similarity values between sentences and paragraphs instead of words. Again, the optimal assignment of values determines the similarity between sentences and between paragraphs.

The LORD block has a "Settings" link, which shows the local settings for the plugin. The first is to start and stop the relation discovery process. The next three settings control the maximum number of paragraphs to compare between learning objects, the maximum number of sentences to compare from each paragraph, and the maximum number of words to allow in a sentence. The Web service used will not compare sentences for more than 2 minutes, so excessively long sentences could cause a timeout and a 0 similarity value (error). There are also options for 3 weights, which are used in calculation the final similarity value. The learning object's content is broken down into names, introductions, and paragraphs (or all other content), each of which can have a separate weight applied to it during the final calculation.

There are 3 reset options available. The first will reset all the comparison errors (anything with a 0 value) so that these comparisons can be tried again. These errors arise from comparing long sentences, so when resetting the errors, it is recommended to reduce the maximum number of words in a sentence as well. The second will reset all the comparison values, which means that the relation discovery process will have to start all over again. This may be useful when stop words are added or removed from the dictionary. The other reset option will reset the comparisons and learning content data. This may be useful if the learning content changed after the discovery process was run. No reset is necessary when maximum sentence and paragraph values are changed, but increasing either value will require more sentences to be compared, which will take some time for the background task. However, the progress table in the block may not reflect that there are more comparisons to be made. Nor is a reset necessary if learning objects are added to or removed from the course, as the plugin is designed to find new learning objects and delete data for removed modules.

The last section of the settings page gives to option of adding and removing stop words from the dictionary. Stop words are removed from the sentences before using the Web server and are not considered in the similarity calculation. There is a default set of stop words, but these can be augmented with others or removed, depending on user preference. The dictionary is global and used in all courses where the plugin is used, so changing the dictionary in one course will affect all other courses.

The background task is set to run every 5 minutes by default, which helps the discovery process complete more quickly. Once completed, however, this schedule is no longer needed and can be changed by from Site administration -> Server -> Scheduled tasks. Alternatively, the discovery process can be turned off for the courses in which it is no longer needed, thereby allowing other courses to use the frequent schedule without introducing unnecessary computation to the courses that are complete.

For each module to be compared, the text learning content is extracted from the activity. When the activity is one of Moodle's built-in types, the content is easy to extract. However, with some module types, it is possible to incorporate arbitrary files such as PDF or images. Extracting the content from these files is achieved with another program called AbiWord (http://www.nl.abisource.com/) that can convert files to text. AbiWord can not be included with the plugin and will need to be installed on the server for this plugin to use it. However, the plugin will also work without AbiWord installed, in which case the file name will be used instead of the file content.

To use Lag Sequential Analysis (LSA) to arrange the graph nodes, the feature needs to be turned on. The switch is located in the instance settings, which can be accessed by turning editing on in the course, then selecting "Configure Behaviour Analytics" from the block gear icon. Turning on the LSA feature will run LSA on each student's data and combine the results to determine which nodes are the most important and the strength on the links between those nodes. This node and link data will be used to configure the graph. When LSA is on, it will override using the LORD graph and also generate a new graph everytime. To use the same graph multiple times, turn on the LSA, generate a graph, then turn LSA off. As long as LORD is not being used, the latest LSA graph will be shown.

The Personalised Study Guide (PSG) plugin is a course format designed to integrate with Behaviour Analytics and provide a customized course page based on the student's learning style. The Behaviour Analytics plugin must be installed and set up for the PSG plugin to work. The PSG plugin can either use the ILS questionnaire results or the clustering analysis results. Using clustering results involves generating at least one clustering analysis and selecting it for use in prediction. When replaying clustering, there is a checkbox at the bottom of the analysis to select for prediction. Only an analysis that has run to convergence can be selected for prediction. Regular, LORD, or LSA graphs may be used in prediction. Once an analysis is selected, the PSG plugin will use the selected analysis to perform learning style prediction based on the cluster that the student belongs to and the common links in that cluster. Each node in the common link set has its learning style determined and summed to determine the learning style of the cluster. The learning style of the course learning objects is also determined and then the objects are arranged based on their relevance to the student's predicted style.

The plugin contains some global settings that affect what a user sees when they use the program. There is an option to allow the students to see their study IDs and to show student names in the graphing menu rather than sequential numbers. If the LORD plugin is installed, there is an option to allow integration with it. The settings also give the option to grant or revoke the role of researcher to any non-student user enrolled in a course the plugin is installed in. The researcher role allows the user to see the current graph configurations and clustering results of other users in that course. The researcher can only see another user's data, they can not change it. The settings can be accessed as administrator from Site administration -> Plugins overview, then searching for "Behaviour Analytics" and clicking the associated settings link.

The block has a scheduled task that is, by default, set to run once a day. The frequency that the task runs can be changed by going to Site administration -> Server -> Scheduled tasks, then clicking the settings icon for "Update Behaviour Analytics." From here, it is also possible to run the task manually.

Teacher users will see an "Export" button to export a complete anonymized data set, which will export everything the plugin has, but change all student IDs to their study IDs. Additionally, administrator users will see export and import forms in the block which are used to export or import student resource access data. The export form has two check boxes, one for currently enrolled students and the other for previously enrolled students. Clicking the export button allows the data to be downloaded. Exporting can also be done through Moodle's reports interface by navigating to Site Administration -> Reports -> Logs. Pick the course to export for and set the actions to "View", then click the "Get these logs" button. Once the logs are retrieved, scroll down to the bottom of the page and download the logs as a JavaScript Object Notation (.json) file. Exporting from the command line is also possible by running the export-cli.php script located in the cli directory. Running the script without parameters will show a usage message.

The import form can take a dragged and dropped file or a file can be chosen using the "Choose a file" button. Only JavaScript Object Notation (.json) file types are accepted. The file name must also contain the first word of the course short name to ensure that the user is importing the right data into the right course. Either a file exported through the export form or through Moodle's report interface can be imported.

Administrators can delete course data stored by the plugin. This may be useful when teaching a course with a new group of students. Or you may just want to delete old data and start clean. The delete data page has 3 options to delete clustering, graph configuration, or student log data. The clustering data is what is made when viewing and clustering students. This includes the clustering and membership data, as well as comment data. The graph configuration data is made when positioning the graph nodes. Deleting this data removes all graph configurations for the course as well as student centroid data associated with those graph configurations. The student log data is extracted from the Moodle logs and copied into another database table. It is the duplicated data that is deleted, not the Moodle log data itself. Data for all users in the course is deleted.

The dashboard consists of a number of links at the top of the page that will show different data. These include the clustering summary tables, by graph and by iteration, with by graph being what is shown when the dashboard is first viewed. The summary by graph shows the latest iteration of clustering and its results, while the summary by iteration shows results for all iterations. To view the summary for a single graph configuration, the graph can be selected from the select graph menu. Multiple configurations may be selected at once for comparison. When viewing graph summaries it is possible to download the shown data as a CSV file, less the bar graphs.

There is also a surveys link that allows for survey management. There are 3 surveys installed by default and there is the option to make custom surveys or change the defaults. Note that changing the default surveys may invalidate the current scoring functions for those surveys. The link in the surveys menu can be used in a URL type course module to give students access to the survey. Currently, a student may only take a survey once. When viewing survey responses, it is possible to download the shown data as a CSV file. Viewing the results of with the ILS or BFI surveys will show radar charts of the respondents data. The ILS radar chart has the option to show all student on a single chart or show 16 charts, one for each learning style category. When viewing the SUS survey results, the total is calculated and shown at the top.