Description
A Learning Management System (LMS) is a software application for the administration, documentation, tracking, reporting and delivery of e-learning education courses or training programs.
E-Learning Benchmark Report: Learning Management System (LMS) usage
1 June 2009
Project Leaders
Kevin Ashford-Rowe Acting Director, Flexible Learning and Access Services (FLAS), Griffith University
Assoc Professor Janne Malfroy Head, Teaching Development Unit, University of Western Sydney
Griffith-UWS E-Learning Benchmark Report
Page 1 of 8
Executive Summary
Griffith University and the University of Western Sydney (UWS) are predominantly face-toface, multi-campus teaching institutions with similar size student bodies and academic communities. Both Griffith and UWS utilise a single enterprise wide e-learning system. Early in 2008, as part of the benchmarking exercise initiated between the two universities, senior staff requested an overview of e-learning. The project team from both universities developed a framework that provides regular, ongoing ‘snapshots’ of Learning Management System (LMS) usage. This will enable each university to track change over time, as well as provide useful data that can contribute to better LMS management, staff development and institutional planning. This represents an enduring strategic benefit to both institutions. The overall results of the benchmarking exercise showed that staff and students at both institutions are using the LMS in fairly similar ways. Tables in the following pages show details of the results. The majority of staff use the LMS to post a wide range of lecture related material such as unit outlines, lecture notes, reading material, and web links. This provides a rich range of resources for students to access at any time and at any place during the semester. The profile also shows that staff are using communication and assessment features in a range of ways to support student learning. Although not all staff are using all features, there is a wide and varied use of different features for specific purposes evident in the sites. Some of the differences between the institutions possibly reflect recent priorities at each institution. At UWS, for example, there is a greater use of the assessment related tools such as the ‘drop-box’ submission process for tracking assessment submission and use of the ‘My Grades’ function to record assessment results. UWS also shows a much greater use of sites for ‘explicit learner support’. At Griffith, there is a greater use of ‘interactive resources’ and ‘rich media’ to support student learning which reflects the greater investment in this interactive media development through FLAS at Griffith. The overall results from both institutions show that most academics are predominantly using the features and capabilities of the LMS as a source of wide ranging learning materials and support for students. This may be a result of the strong emphasis at both universities on maintaining quality face-to-face interaction. There are a small number of academics at both universities who are extending their LMS use to more sophisticated ways but it appears that there is an institutional need to further develop the role of academics as e-teaching practitioners. Whilst the organic ‘ground up’ use of the LMS has worked well in allowing academics to embrace the technology as they wanted to, it would seem that now would be timely to adopt stronger institutional directions, ‘top down’, in order to support the move to a more constructivist approach in designing learning environments. The benchmarking project offers a simple snapshot of LMS usage. At a unit/course level this data has to be seen as only part of the total learning experience for the students. The framework and process used in this project will provide tools for further and regular reviews and reports. The benchmarking project has also established an enduring partnership between the UWS and Griffith project teams and there is the intention to build on the recent project and to continue sharing work and resources in this area.
Griffith-UWS E-Learning Benchmark Report
Page 2 of 8
Project team
The project team consisted of: Griffith University Kevin Ashford-Rowe Acting/Director, FLAS University of Western Sydney Associate Professor Janne Malfroy Head, Teaching Development Unit (TDU) Lynnae Rankine E-Learning Manager (TDU)
Project leaders
Project team
Leigh Stevenson Manager, Learning and Teaching Systems Brad Nielsen Educational Designer Michael Garner Team Leader, Learning@Griffith
Tom Bowring E-Learning Staff Developer (TDU)
Terminology
It should be noted that both Griffith and UWS use different terms to describe a single taught subject. Thus at Griffith a ‘program’ is comprised of a number of ‘courses’, and at UWS a ‘program’ is comprised of a number of ‘units’. In this report, the unit of analysis is ‘course/ unit’ use of the LMS.
Project aims
Griffith and UWS are predominantly face-to-face, multi-campus teaching institutions with similar size student bodies and academic communities. Both institutions support multiple campuses dispersed within a similarly sized geographical area. In addition, both institutions operate within a large conurbation and compete with other universities within these geographical areas. Finally, both Griffith and UWS utilise a single enterprise wide e-learning system. This project commenced in May 2008 and aimed to determine the level and quality of the usage of the enterprise learning management system (LMS) at both UWS and Griffith University. It built upon previous work that had been undertaken at UWS to identify and measure academic uptake of the tools within the enterprise learning management system, and at the same time to determine the degree of enterprise return on investment in such systems. For the purposes of this research, a 10% sample or 'snapshot' of the courses/units on the learning management systems of each of the two institutions was taken over one semester in 2008. This report covers Stage 1 and Stage 2. Stage 3 is a future project. Stage 1 Stage 2 Stage 3 Design and develop a framework for the project and generate the sample of 10% of sites Apply the framework designed and developed at Stage 1 to identify how the sites were being used Use a more detailed framework to identify quality of e-learning usage
Griffith-UWS E-Learning Benchmark Report
Page 3 of 8
LMS usage framework
The way in which e-learning is being used pedagogically is a relatively un-researched area in Australian universities. Previous research has often relied primarily on determining ‘tool’ use. The difference with the approach adopted for this project is the recognition that LMS tools can be used for a variety of learning and teaching initiatives, and that ultimately there is a need for more mature indicators of their use to be better able to determine what good elearning practice is. A framework (see below) was jointly developed by the two institutions which built upon a framework used in a previous sample study at UWS. The purpose of this framework was to further develop a set of pedagogical content types and tools common to online unit/courses. The framework can be broken down into smaller elements to support greater quantitative and qualitative data collection and more detailed analysis.
Table 1: shows the LMS usage framework developed by the benchmarking team
Content
Communication
Collaboration
Assessment
Explicit Learner Support
Text-matching software (e.g. SafeAssign, Turnitin) Links to student support materials (internally provided) Links to student support materials (externally provided) Additional learning support advice (i.e. learning guide, discipline guides)
Unit/Course Outline
Chat
Wikis
Reflective learning journals
Lecture and Tutorial Notes
Announcements
Discussion Board *
Assignment drop boxes
Media used in lectures and tutorials (i.e. lectopia, podcast, videocasts) Links to scholarly information (readings)
Discussion Board *
Virtual Classroom
Quizzes
Email
Voice-based communication
My Grades
Links to content resources (i.e. websites) Interactive resources (.swf .fla .flv and other file types)
*
Blogs
Templates, practice activities and past exams Surveys
Discussion board can be used as either a communication tool or a collaboration tool
Method and sample size
In order to generate a representative sample of 10% of current unit/course sites from Semester 1, 2008, both institutions sought to ensure that the sample, though generated randomly, included a representative sample of the courses/units in terms of undergraduate: postgraduate ratio and overall number of courses/units as provided by each academic element within each institution. The following additional decisions were made prior to selecting the sample. Course hub sites were excluded, as well as courses/units that had received direct development support from an educational designer (particularly at Griffith). As noted above, the number of undergraduate and postgraduate sites was distributed across the sample and the number of sites was distributed across each school. The sample size from both institutions was 10% of the course/unit sites in a semester. However the overall allocation of sites operates differently at the two universities. At Griffith, all offerings of a course/unit have a discrete site, whereas at UWS a course/unit has a single
Griffith-UWS E-Learning Benchmark Report
Page 4 of 8
site, no matter how many offerings (campuses) it has. Therefore, because the total number of sites was much larger at Griffith, 10% was considered an adequate sample set. However, at UWS, the total number of sites was much smaller, and the project team had some concerns about the representation of the 10%. Therefore, in addition to the initial 10% sample measured, UWS undertook a second sample of 10% as a means to validate the first sample. The results from the additional 10% of UWS sites replicated the first 10% results, indicating that although smaller than the Griffith sample set, it appeared to be representative of the total set of sites. The framework (at Table 1) was initially trialled by both institutions with a sample of 3 sites. These sites were then blind reviewed by the other institution and results were compared to make sure the framework was being consistently applied by both institutions. Slight modifications were made to the framework before the full assessment of sites commenced.
Results
Overall, the benchmarking activity showed similar results across both universities, as shown in Table 2, although one or two features did show marked differences.
Table 2: LMS Usage Benchmarking Results – Semester 1 (Griffith) Spring Semester (UWS) 2008
Unit/Course Outline Lecture Notes Content Media used in lectures Links to scholarly information (readings) Links to content Interactive resources Communication *Chat Announcement Discussion Board *Email Blogs ^ Wikis Discussion Board ^ Virtual Classroom Voice Communication Reflective Learning Journals Assessment Assignment Drop boxes Quizzes My Grades Models & Practice Surveys Text Matching s/w Explicit Learner Support Links to internal support materials Links to external support materials Additional learning support 0 10 20 30 40 50 60 70 80 90 100 Griffith UWS
Function / Tool
Collaboration
% of Sampled Unit/Course Sites Using Function/Tool
* Chat and Email usage data not available in Griffith LMS ^ Wiki or Virtual Classroom tools not enabled in UWS LMS
The results from this survey are also described in terms of the grouped criteria originally applied, i.e. content, communication, collaboration, assessment, explicit learner support: Content: Both universities require unit/course outlines to be posted in sites and the results show that there is almost 100% compliance for this item. For both universities there is strong engagement with unit/course outline, lecture notes and other core unit/course materials, and almost 40% of units/courses showed additional links to content. At Griffith there is slightly greater uptake of lecture notes than at UWS. For both institutions there is relatively lower use of interactive and media resources, although this is higher for Griffith.
Griffith-UWS E-Learning Benchmark Report
Page 5 of 8
Communication: For both universities there is extensive use of announcements to communicate important information to students, with a much greater uptake of this feature at Griffith. There is similar usage of discussion boards for communication purposes across both institutions with UWS having a slightly greater uptake. Collaboration: The study indicates that the online collaboration features of the LMS are not extensively used in many of the sites sampled for both universities. Assessment: There is similar use of quizzes and assignment drop boxes across both institutions. There are two areas of substantial difference in My Grades and the provision of models and practice tasks for students with UWS demonstrating a greater uptake in these areas. Explicit learner support: For both universities there is substantial provision of explicit learner support with UWS having a much greater uptake in this aspect of LMS usage. The use of text matching software at both universities was minimal as it was in pilot mode and not yet fully enabled at the time of data collection. A complete set of results is available in Appendix A.
Discussion
The overall results of the benchmarking exercise showed that staff and students at both institutions are using the LMS in fairly similar ways. Most sites contain a wide range of lecture related material such as course/unit outlines, lecture notes, reading material, and web links. This provides a rich range of resources for students to access at any time and at any place during the semester. This approach to usage certainly supports a more traditional pedagogical approach compared to sites that are media rich and foster collaborative learning. The profile also shows that staff are using communication and assessment features in a range of ways to support student learning. Although not all staff are using all features, there is a wide and varied use of different features for specific purposes evident in the sites. There are also a small number of academics at both universities who are extending their LMS use to more sophisticated ways. In future reviews we would expect to see an increase in the online collaboration usage in the LMS. The data from this project also shows how the affordances of the LMS have helped academic staff in administering their course/unit, and managing assessment submissions and grades. Not only can these affordances improve efficiency in academic management but they also offer more systematic approaches to identifying students at risk. It was also very positive to note the high number of course/units sampled that provided a wide range of explicit learner support such as guides, website links, and exam examples. These resources are providing access to all students. The framework deliberately included aspects of usage that are in relatively early uptake stages in order to establish a benchmark level for future reports. Therefore there is comparatively limited uptake of wikis, virtual classroom, text matching software and learning journals. It is acknowledged however that these technologies are relatively new at both institutions and it is anticipated that future studies would show a greater uptake of these technologies.
Conclusion
Griffith-UWS E-Learning Benchmark Report
Page 6 of 8
Overall, this has been a valuable and engaging activity and has produced a series of results that have enabled both universities to gain an interesting and useful insight into the uptake and usage of the LMS; however it is important to note that it does not make a judgement about the quality of the learning experience for students. The LMS usage is only one part of the blended learning experience at both universities. The framework and process used in this project will provide tools for further and regular reviews and reports. Regular usage reports may be generated within each institution to help schools plan their future enhancement of the learning environment. It should also be noted that the framework was not designed to assess quality of usage, and should be used with other data about e-learning to provide a more comprehensive overview of the role of e-learning in the contemporary teaching environment. These results will be used by the respective units, Information Services (Flexible Learning and Access Services) at Griffith University and the Teaching Development Unit at the University of Western Sydney, to identify opportunities for further development and improvement at each institution. In this respect, this benchmarking report gives Griffith and UWS a snapshot of usage at a particular point in time, a process and framework for further evaluation, a mechanism to inform strategies for staff development in the use of the LMS, a range tools to enable further reporting and an enduring partnership in this highly relevant and important field.
Griffith-UWS E-Learning Benchmark Report
Page 7 of 8
Appendix A: Data Summary
Content
Communication
Collaboration
Assessment
Explicit Learner Support
Links to scholarly information (readings)
Voice Based Communication
Reflective Learning Journals
UWS Raw data UWS data % Griffith Raw data Griffith data %
64 88 120 93
49 67 102 80
3 4 10 8
38 51 48 37
29 40 50 39
1 1 5 4
5 7 0* 0*
37 51 118 91
27 37 32 25
38 52 0* 0*
2 3 2 2
0^ 0^ 8 6
1 1 4 3
0^ 0^ 14* 11*
0 0 1 1
2 3 1 1
16 22 19 15
7 10 15 12
37 51 22 17
14 19 11 9
0 0 3 2
0 0 3 2
68 93 67 52
4 5 11 9
0 0 3 2
* Denotes items that could not be checked through the audit. Virtual classroom data was generated by detecting user time inside the virtual classroom. ^ UWS does not have wikis or virtual classroom tools enabled in the LMS.
Additional learning support
Links to external support materials
Links to internal support materials
Assignment Drop boxes
Media used in lectures
Interactive resources
Unit/Course Outline
Text Matching s/w
Models & Practice
Virtual Classroom
Discussion Board
Discussion Board
Links to content
Announcement
Lecture Notes
My Grades
Quizzes
Surveys
Wikis
Email
Blogs
Chat
doc_148483214.pdf
A Learning Management System (LMS) is a software application for the administration, documentation, tracking, reporting and delivery of e-learning education courses or training programs.
E-Learning Benchmark Report: Learning Management System (LMS) usage
1 June 2009
Project Leaders
Kevin Ashford-Rowe Acting Director, Flexible Learning and Access Services (FLAS), Griffith University
Assoc Professor Janne Malfroy Head, Teaching Development Unit, University of Western Sydney
Griffith-UWS E-Learning Benchmark Report
Page 1 of 8
Executive Summary
Griffith University and the University of Western Sydney (UWS) are predominantly face-toface, multi-campus teaching institutions with similar size student bodies and academic communities. Both Griffith and UWS utilise a single enterprise wide e-learning system. Early in 2008, as part of the benchmarking exercise initiated between the two universities, senior staff requested an overview of e-learning. The project team from both universities developed a framework that provides regular, ongoing ‘snapshots’ of Learning Management System (LMS) usage. This will enable each university to track change over time, as well as provide useful data that can contribute to better LMS management, staff development and institutional planning. This represents an enduring strategic benefit to both institutions. The overall results of the benchmarking exercise showed that staff and students at both institutions are using the LMS in fairly similar ways. Tables in the following pages show details of the results. The majority of staff use the LMS to post a wide range of lecture related material such as unit outlines, lecture notes, reading material, and web links. This provides a rich range of resources for students to access at any time and at any place during the semester. The profile also shows that staff are using communication and assessment features in a range of ways to support student learning. Although not all staff are using all features, there is a wide and varied use of different features for specific purposes evident in the sites. Some of the differences between the institutions possibly reflect recent priorities at each institution. At UWS, for example, there is a greater use of the assessment related tools such as the ‘drop-box’ submission process for tracking assessment submission and use of the ‘My Grades’ function to record assessment results. UWS also shows a much greater use of sites for ‘explicit learner support’. At Griffith, there is a greater use of ‘interactive resources’ and ‘rich media’ to support student learning which reflects the greater investment in this interactive media development through FLAS at Griffith. The overall results from both institutions show that most academics are predominantly using the features and capabilities of the LMS as a source of wide ranging learning materials and support for students. This may be a result of the strong emphasis at both universities on maintaining quality face-to-face interaction. There are a small number of academics at both universities who are extending their LMS use to more sophisticated ways but it appears that there is an institutional need to further develop the role of academics as e-teaching practitioners. Whilst the organic ‘ground up’ use of the LMS has worked well in allowing academics to embrace the technology as they wanted to, it would seem that now would be timely to adopt stronger institutional directions, ‘top down’, in order to support the move to a more constructivist approach in designing learning environments. The benchmarking project offers a simple snapshot of LMS usage. At a unit/course level this data has to be seen as only part of the total learning experience for the students. The framework and process used in this project will provide tools for further and regular reviews and reports. The benchmarking project has also established an enduring partnership between the UWS and Griffith project teams and there is the intention to build on the recent project and to continue sharing work and resources in this area.
Griffith-UWS E-Learning Benchmark Report
Page 2 of 8
Project team
The project team consisted of: Griffith University Kevin Ashford-Rowe Acting/Director, FLAS University of Western Sydney Associate Professor Janne Malfroy Head, Teaching Development Unit (TDU) Lynnae Rankine E-Learning Manager (TDU)
Project leaders
Project team
Leigh Stevenson Manager, Learning and Teaching Systems Brad Nielsen Educational Designer Michael Garner Team Leader, Learning@Griffith
Tom Bowring E-Learning Staff Developer (TDU)
Terminology
It should be noted that both Griffith and UWS use different terms to describe a single taught subject. Thus at Griffith a ‘program’ is comprised of a number of ‘courses’, and at UWS a ‘program’ is comprised of a number of ‘units’. In this report, the unit of analysis is ‘course/ unit’ use of the LMS.
Project aims
Griffith and UWS are predominantly face-to-face, multi-campus teaching institutions with similar size student bodies and academic communities. Both institutions support multiple campuses dispersed within a similarly sized geographical area. In addition, both institutions operate within a large conurbation and compete with other universities within these geographical areas. Finally, both Griffith and UWS utilise a single enterprise wide e-learning system. This project commenced in May 2008 and aimed to determine the level and quality of the usage of the enterprise learning management system (LMS) at both UWS and Griffith University. It built upon previous work that had been undertaken at UWS to identify and measure academic uptake of the tools within the enterprise learning management system, and at the same time to determine the degree of enterprise return on investment in such systems. For the purposes of this research, a 10% sample or 'snapshot' of the courses/units on the learning management systems of each of the two institutions was taken over one semester in 2008. This report covers Stage 1 and Stage 2. Stage 3 is a future project. Stage 1 Stage 2 Stage 3 Design and develop a framework for the project and generate the sample of 10% of sites Apply the framework designed and developed at Stage 1 to identify how the sites were being used Use a more detailed framework to identify quality of e-learning usage
Griffith-UWS E-Learning Benchmark Report
Page 3 of 8
LMS usage framework
The way in which e-learning is being used pedagogically is a relatively un-researched area in Australian universities. Previous research has often relied primarily on determining ‘tool’ use. The difference with the approach adopted for this project is the recognition that LMS tools can be used for a variety of learning and teaching initiatives, and that ultimately there is a need for more mature indicators of their use to be better able to determine what good elearning practice is. A framework (see below) was jointly developed by the two institutions which built upon a framework used in a previous sample study at UWS. The purpose of this framework was to further develop a set of pedagogical content types and tools common to online unit/courses. The framework can be broken down into smaller elements to support greater quantitative and qualitative data collection and more detailed analysis.
Table 1: shows the LMS usage framework developed by the benchmarking team
Content
Communication
Collaboration
Assessment
Explicit Learner Support
Text-matching software (e.g. SafeAssign, Turnitin) Links to student support materials (internally provided) Links to student support materials (externally provided) Additional learning support advice (i.e. learning guide, discipline guides)
Unit/Course Outline
Chat
Wikis
Reflective learning journals
Lecture and Tutorial Notes
Announcements
Discussion Board *
Assignment drop boxes
Media used in lectures and tutorials (i.e. lectopia, podcast, videocasts) Links to scholarly information (readings)
Discussion Board *
Virtual Classroom
Quizzes
Voice-based communication
My Grades
Links to content resources (i.e. websites) Interactive resources (.swf .fla .flv and other file types)
*
Blogs
Templates, practice activities and past exams Surveys
Discussion board can be used as either a communication tool or a collaboration tool
Method and sample size
In order to generate a representative sample of 10% of current unit/course sites from Semester 1, 2008, both institutions sought to ensure that the sample, though generated randomly, included a representative sample of the courses/units in terms of undergraduate: postgraduate ratio and overall number of courses/units as provided by each academic element within each institution. The following additional decisions were made prior to selecting the sample. Course hub sites were excluded, as well as courses/units that had received direct development support from an educational designer (particularly at Griffith). As noted above, the number of undergraduate and postgraduate sites was distributed across the sample and the number of sites was distributed across each school. The sample size from both institutions was 10% of the course/unit sites in a semester. However the overall allocation of sites operates differently at the two universities. At Griffith, all offerings of a course/unit have a discrete site, whereas at UWS a course/unit has a single
Griffith-UWS E-Learning Benchmark Report
Page 4 of 8
site, no matter how many offerings (campuses) it has. Therefore, because the total number of sites was much larger at Griffith, 10% was considered an adequate sample set. However, at UWS, the total number of sites was much smaller, and the project team had some concerns about the representation of the 10%. Therefore, in addition to the initial 10% sample measured, UWS undertook a second sample of 10% as a means to validate the first sample. The results from the additional 10% of UWS sites replicated the first 10% results, indicating that although smaller than the Griffith sample set, it appeared to be representative of the total set of sites. The framework (at Table 1) was initially trialled by both institutions with a sample of 3 sites. These sites were then blind reviewed by the other institution and results were compared to make sure the framework was being consistently applied by both institutions. Slight modifications were made to the framework before the full assessment of sites commenced.
Results
Overall, the benchmarking activity showed similar results across both universities, as shown in Table 2, although one or two features did show marked differences.
Table 2: LMS Usage Benchmarking Results – Semester 1 (Griffith) Spring Semester (UWS) 2008
Unit/Course Outline Lecture Notes Content Media used in lectures Links to scholarly information (readings) Links to content Interactive resources Communication *Chat Announcement Discussion Board *Email Blogs ^ Wikis Discussion Board ^ Virtual Classroom Voice Communication Reflective Learning Journals Assessment Assignment Drop boxes Quizzes My Grades Models & Practice Surveys Text Matching s/w Explicit Learner Support Links to internal support materials Links to external support materials Additional learning support 0 10 20 30 40 50 60 70 80 90 100 Griffith UWS
Function / Tool
Collaboration
% of Sampled Unit/Course Sites Using Function/Tool
* Chat and Email usage data not available in Griffith LMS ^ Wiki or Virtual Classroom tools not enabled in UWS LMS
The results from this survey are also described in terms of the grouped criteria originally applied, i.e. content, communication, collaboration, assessment, explicit learner support: Content: Both universities require unit/course outlines to be posted in sites and the results show that there is almost 100% compliance for this item. For both universities there is strong engagement with unit/course outline, lecture notes and other core unit/course materials, and almost 40% of units/courses showed additional links to content. At Griffith there is slightly greater uptake of lecture notes than at UWS. For both institutions there is relatively lower use of interactive and media resources, although this is higher for Griffith.
Griffith-UWS E-Learning Benchmark Report
Page 5 of 8
Communication: For both universities there is extensive use of announcements to communicate important information to students, with a much greater uptake of this feature at Griffith. There is similar usage of discussion boards for communication purposes across both institutions with UWS having a slightly greater uptake. Collaboration: The study indicates that the online collaboration features of the LMS are not extensively used in many of the sites sampled for both universities. Assessment: There is similar use of quizzes and assignment drop boxes across both institutions. There are two areas of substantial difference in My Grades and the provision of models and practice tasks for students with UWS demonstrating a greater uptake in these areas. Explicit learner support: For both universities there is substantial provision of explicit learner support with UWS having a much greater uptake in this aspect of LMS usage. The use of text matching software at both universities was minimal as it was in pilot mode and not yet fully enabled at the time of data collection. A complete set of results is available in Appendix A.
Discussion
The overall results of the benchmarking exercise showed that staff and students at both institutions are using the LMS in fairly similar ways. Most sites contain a wide range of lecture related material such as course/unit outlines, lecture notes, reading material, and web links. This provides a rich range of resources for students to access at any time and at any place during the semester. This approach to usage certainly supports a more traditional pedagogical approach compared to sites that are media rich and foster collaborative learning. The profile also shows that staff are using communication and assessment features in a range of ways to support student learning. Although not all staff are using all features, there is a wide and varied use of different features for specific purposes evident in the sites. There are also a small number of academics at both universities who are extending their LMS use to more sophisticated ways. In future reviews we would expect to see an increase in the online collaboration usage in the LMS. The data from this project also shows how the affordances of the LMS have helped academic staff in administering their course/unit, and managing assessment submissions and grades. Not only can these affordances improve efficiency in academic management but they also offer more systematic approaches to identifying students at risk. It was also very positive to note the high number of course/units sampled that provided a wide range of explicit learner support such as guides, website links, and exam examples. These resources are providing access to all students. The framework deliberately included aspects of usage that are in relatively early uptake stages in order to establish a benchmark level for future reports. Therefore there is comparatively limited uptake of wikis, virtual classroom, text matching software and learning journals. It is acknowledged however that these technologies are relatively new at both institutions and it is anticipated that future studies would show a greater uptake of these technologies.
Conclusion
Griffith-UWS E-Learning Benchmark Report
Page 6 of 8
Overall, this has been a valuable and engaging activity and has produced a series of results that have enabled both universities to gain an interesting and useful insight into the uptake and usage of the LMS; however it is important to note that it does not make a judgement about the quality of the learning experience for students. The LMS usage is only one part of the blended learning experience at both universities. The framework and process used in this project will provide tools for further and regular reviews and reports. Regular usage reports may be generated within each institution to help schools plan their future enhancement of the learning environment. It should also be noted that the framework was not designed to assess quality of usage, and should be used with other data about e-learning to provide a more comprehensive overview of the role of e-learning in the contemporary teaching environment. These results will be used by the respective units, Information Services (Flexible Learning and Access Services) at Griffith University and the Teaching Development Unit at the University of Western Sydney, to identify opportunities for further development and improvement at each institution. In this respect, this benchmarking report gives Griffith and UWS a snapshot of usage at a particular point in time, a process and framework for further evaluation, a mechanism to inform strategies for staff development in the use of the LMS, a range tools to enable further reporting and an enduring partnership in this highly relevant and important field.
Griffith-UWS E-Learning Benchmark Report
Page 7 of 8
Appendix A: Data Summary
Content
Communication
Collaboration
Assessment
Explicit Learner Support
Links to scholarly information (readings)
Voice Based Communication
Reflective Learning Journals
UWS Raw data UWS data % Griffith Raw data Griffith data %
64 88 120 93
49 67 102 80
3 4 10 8
38 51 48 37
29 40 50 39
1 1 5 4
5 7 0* 0*
37 51 118 91
27 37 32 25
38 52 0* 0*
2 3 2 2
0^ 0^ 8 6
1 1 4 3
0^ 0^ 14* 11*
0 0 1 1
2 3 1 1
16 22 19 15
7 10 15 12
37 51 22 17
14 19 11 9
0 0 3 2
0 0 3 2
68 93 67 52
4 5 11 9
0 0 3 2
* Denotes items that could not be checked through the audit. Virtual classroom data was generated by detecting user time inside the virtual classroom. ^ UWS does not have wikis or virtual classroom tools enabled in the LMS.
Additional learning support
Links to external support materials
Links to internal support materials
Assignment Drop boxes
Media used in lectures
Interactive resources
Unit/Course Outline
Text Matching s/w
Models & Practice
Virtual Classroom
Discussion Board
Discussion Board
Links to content
Announcement
Lecture Notes
My Grades
Quizzes
Surveys
Wikis
Blogs
Chat
doc_148483214.pdf