2016 IQAP Review of Bachelor of Arts (General) Programme
Final Assessment Report and Implementation Plan
In accordance with the Royal Military College (RMC) Institutional Quality Assurance Plan (IQAP), this Final Assessment Report (FAR) provides a synthesis of the external evaluation and internal response and assessments of the Bachelor of Arts (General) degree offered by RMC.
This report identifies the significant strengths of the programme, together with opportunities for programme improvement and enhancement, and sets out and prioritizes the recommendations that have been selected for implementation.
This report includes an Implementation Plan that identifies who will be responsible for approving the recommendations set out in this report; who will be responsible for providing any resources entailed by those recommendations; any changes in organization, policy or governance that will be necessary to meet the recommendations and who will be responsible for acting on those recommendations; and timelines for acting on and monitoring the implementation of those recommendations.
Overview of Programme Review Process:
The Programme Self-Study Report was completed in June 2016.
The External Review Committee (ERC) included one arm’s-length external reviewer, Dr. Jill Scott, Vice-Provost of Queen’s University, and one internal reviewer, Dr. Lucien Haddad, from the Department of Mathematics and Computer Science, RMC, who were selected from a list of possible reviewers and approved by the Dean of Social Sciences and Humanities (SSH). They reviewed the Self-Study Report and conducted a site visit to RMC on 9-10 March 2017. The visit included interviews with the Interim Dean of SSH, the Dean of Continuing Studies, the Vice-Principal Academic, the Chief Librarian, Director of Quality Assurance, members of the Division of Continuing Studies (DCS), as well as members of the Faculty of SSH. The ERC subsequently produced a report based on the Self-Study and site visit. The report was circulated to all those associated with the degree programme, as well as to the Self-Study team.
The ERC submitted their report in April 2017. They find much to commend in the degree programme and offer a number of proposals for strengthening the degree. In the absence of a traditional department, the report has been reviewed by the Dean of SSH, the Associate Dean of SSH (Programme) and the Self-Study team.
All of the actors involved have seized the opportunity to provide useful and important analysis and recommendations about the current context for the degree programme and the way forward.
The focus of the ERC feedback and recommendations is on five linked issues:
- The segregation between the BA (Gen) degree and the other Faculty of SSH degree programmes.
- The way in which resources are allocated in a variety of the services and elements in the degree programme.
- Instructor compensation and performance.
- The Degree Level Expectations (DLE), both in terms of alignment with peer institutions and in terms of articulating and implementing learning outcomes in line with provincial standards.
- Accessibility of distance courses to persons with disabilities.
Significant Strengths and Areas of Concern of the Programme:
The ERC made particular note of RMC’s leadership in establishing early on a distance education programme geared to working professionals well before other universities saw the need and benefits of precisely this kind of programme. The ERC also made particular mention of the competency and dedication of the DCS staff, which in turn contributes to a student body that are satisfied with and well served by the support services available to them. This strength is deepened by the strong Prior Learning Assessment (PLAR) process, which is on a par or exceeds the process available to students at other institutions, and in doing so facilitates the ability of students in the programme to complete their degrees in a timely manner.
The Associate Dean of SSH (Programme), after consultation with faculty and staff in the programme, submitted a response to the ERC Report in June 2017. The Dean of SSH in consultation with Faculty of SSH Department Heads and Dean of Continuing Studies prepared this Final Assessment Report in October 2017. In fall of 2018, the Dean of Continuing Studies was provided a further opportunity to comment on the FAR being submitted to RMC Senate. Specific recommendations are discussed, and follow-up actions and timelines provided.
Summary of the Reviewers’ Recommendations with the Programme’s and Dean’s Responses
Issue #1 – Integration of onsite and offsite degrees
Work toward a greater integration of the Bachelor of Arts (General) within the Faculty of SSH. Many post-secondary institutions are working now to integrate distance and online learning into regular academic programming. For a variety of reasons, distance education has been administered through extension units, but there is now general agreement in the literature on online learning that it is best for students and for the institution when online courses are overseen by regular academic units and when those who teach courses are members of those same academic units whether contract or permanent. While the BA (Gen) is very well supported by the Division of Continuing Studies and the administration within the Faculty of SSH, it seems there is less a sense of ownership among departments and permanent faculty. The reviewers heard that online learning is not understood as a regular part of RMC operations. It seems that there are considerable structural barriers to assigning permanent faculty to courses taught in the BA (Gen), for example a department head who assigns a regular faculty member to teach an online course loses an instructor for the on-campus programme. The difficulty of salary budgets being sharply
divided between SWE and O&M means that there are in fact incentives to hire contract faculty for online courses and use permanent faculty for on-campus courses. This, however, may lead to inconsistency of student experience between DL and onsite students, with onsite students benefitting from more and easier access to full-time, regular departmental faculty.
The reviewers have four sub-recommendations:
- Develop a budgetary mechanism to create incentives for departments to assign permanent faculty to courses.
- Leverage online learning for faculty development in teaching and learning / technology-enhanced pedagogies.
- Increase the percentage of BA (Gen) courses taught by permanent faculty.
- Increase the range of courses available to enable students to fulfill requirements.
Degree Programme Responses
The Degree Programme agrees with the ERC’s four recommendations and the emphasis they place on them. The ERC-perceived segregation of the BA (Gen) degree from the other onsite undergraduate degree programmes and the Faculty of SSH was seen as a significant problem that has negatively impacted the students in the programme and that it will take time and effort at a variety of levels to overcome. The ERC have additionally noted that there are a number of structural institutional hurdles to be overcome to better integrate the degree into the Faculty of SSH and ensure its viability over time.
Since the time of the ERC visit in March 2017, RMC has already been working on activities related to some of the areas identified by the ERC as requiring attention, stemming from the implementation of recommendations identified in recent reports, namely: the Report on the RMC by the Special Staff Assistance Visit (SSAV) led by Vice-Admiral (Retired) Maddison (Maddison et al., 2017); the Auditor General of Canada Report 6-Royal Military College of Canada (2017); and the IQAP Final Assessment Report for the Bachelor of Military Arts and Science (BMASc) Degrees (2016).
The integration of online and onsite BA programmes recommended by the ERC already started to occur through increased dual delivery course offerings i.e. concurrently delivering courses to residential and distance learning students, by some departments such as the Departments of Management, of History and of English, Culture, and Communication. More consequentially, the Division of Continuing Studies was closed in June 2020 with its functions and responsibilities integrated with other units within RMC, including full program responsibility for the BA (Gen) and BMASc assigned to the Faculty of SSH.
The Associate Dean of SSH (Programme) will work closely with the Dean of SSH to put the required steps in place to better integrate off and onsite BA programmes. With integration of DCS’s former functions, many of the required actions now fall exclusively within the Dean of SSH’s purview. In addition to supporting the Dean in pursuing those objectives, the Associate Dean of SSH (Programme) will work directly on the two linked objectives of strengthening technology-enhanced pedagogies within the Faculty at large and increasing the range of courses
available to BA (Gen) students. The Programme will work toward this goal in consultation with the Dean of SSH and the Centre for Innovation in Teaching and Learning (CITL).
Dean of SSH’s Responses
Much of the source of the segregation of the BA (Gen) degree had budgetary roots. The Dean of SSH, in consultation with department heads, is responsible for planning faculty allocation of onsite and offsite faculty. There is no incentive for departments to allocate onsite faculty as distance course instructors. The Dean of SSH agrees with the ERC and the degree programme that this situation should be addressed with a view to ensure that the offsite students may experience greater access to full-time RMC faculty. Resolving this situation involves ensuring that the Dean of SSH, who is responsible for the degree programme is able to make the decisions about the allocation of human resources within the program (i.e. assignment of faculty). The Dean of SSH will need to continue to work with the VPA to increase the percentage of offsite courses taught by onsite instructors (recommendation 1(a)). This would also allow for the achievement of ERC recommendation 1(c) – an increase in the percentage of onsite faculty teaching in the programme.
The Dean of SSH agrees with the ERC that advanced online learning technologies and pedagogies, involving for example of dual-delivery or hybrid courses (courses that have both online and onsite components), must be better exploited (recommendation 1(b)). Furthermore, such technology and pedagogy will allow RMC to collaborate better with its sister institution in St-Jean, QC. Providing courses to both military colleges either asynchronously or synchronously will better utilize scarce resources and thus achieve some of the concerns raised by the Auditor General.
This change would also facilitate the achievement of recommendation 1(d), the need to increase the range of courses available for the BA (Gen) students. In furtherance of this particular goal the Dean supports the pursuit of hybrid onsite/distance course options in all years as outlined in the Degree Programme response and facilitated the implementation of this proposal, with adoption completed in some departments and trials ongoing in others.
Issue #2 – Resource and services
The reviewers had concerns regarding a number of services that are important for online students. They are broken down into four recommendations.
- Take immediate action to increase access to library resources and support. From discussions with three librarians, administrators and faculty, it is clear to the reviewers that the library is in urgent need of resources: with regard to human resources, the collection, and accessing the collection. Reviewers were concerned to hear about the difficulty students have in accessing online resources and the lack of availability of key learning supports, such as a module for online students to help them transition to university learning and using library resources. Given current staffing, it is not possible to integrate information literacy directly into courses, which is now considered best practice in online learning. We were also concerned to hear that in some cases on-campus students are
considered a higher priority than students enrolled in a distance programme. Access to library support outside of regular working hours is also a concern given that most BA (Gen) students are working professionals and often live in different time zones. In the student survey used to gather information for the Self-Study, the library ranked second lowest for satisfaction among the items queried. The urgent situation regarding the library at RMC threatens to impact on the student experience in the BA (Gen). - Ensure that students in the BA (Gen) have adequate access to information technology support. Because students enrolled in the programme are largely working professionals, they tend to study during evenings and weekends. Providing some options for IT support during non-business hours is essential to student success.
- Ensure that students in the BA (Gen) have access to the Writing Centre. The Self-Study states that students in the BA (Gen) do not have access to programming and resources from the Writing Centre. The needs of students in distance and online programmes for writing support are no different from those of on-campus students. In fact, because many BA (Gen) students are returning to an academic environment for the first time in many years, there may be individuals who are lacking in academic writing skills. It is vital that students in the programme have the same level of access to learning support services, including from the Writing Centre, as students enrolled in on-campus programmes.
- Increase resources for instructional design. Reviewers heard that DCS was greatly affected by Work Force Adjustment (WFA) and that the unit lost many positions. While it is clear that the discontinuation of the Officer Professional Military Education (OPME) programme meant a reduction in service responsibility, there is concern from the ERC regarding DCS’s capacity to serve the BA (Gen) particularly in the area of instructional design. The unit is responsible for ensuring the academic quality and maintenance of approximately 180 online courses and the design of approximately 10-12 new courses every year. With a staff of 1.6 instructional designers, this seems like a very difficult task and raises concerns about the academic quality of courses. Regular maintenance of courses requires checking links and updating resources before each delivery, and designing a quality online course is a very involved process requiring intense work with a content expert / faculty member. Distance and online learning has suffered in the past from a reputation of being a less rigorous and therefore less valuable learning experience, which at times has led to the devaluing of a distance / online degree. If RMC would like to ensure that such attitudes do not persist and that its programmes are known for their excellent student engagement opportunities and rich learning experiences, then reviewers recommend that RMC invest appropriate resources in instructional design. The reviewers do not presume that there is not sufficient quality to current online courses; they merely wish to reinforce the notion that instructional design is critical to the design and maintenance of high-quality online courses and that resources in this area may be insufficient to maintain the level of quality desired by RMC and by students.
Degree Programme’s Responses
The degree programme agrees with the ERC’s judgement of the situation and their recommendations for each of the four areas. Specifically, the lack of online, after-hours, available library, IT, and writing resources is an untenable situation for a distance learning degree programme.
Regarding recommendation 2.a, the recent SSAV has put in motion some institutional changes that will result in increased resources being provided to the library, including ensuring that standard digital resources being available to students on a more consistent basis. While the SSAV process has not and will not fully resolve the library resource issue, it has provided some momentum and resources towards addressing this problem that were not foreseen when the ERC undertook its review.
The Associate Dean of SSH (Programme) commits to creating an online learning module to provide students with the instructional tools they need for university learning generally and the Massey Library specifically.
The Associate Dean of SSH (Programme) will work with the Dean and RMC’s IT services to develop a plan to address recommendation 2.b.
The Associate Dean of SSH (Programme) will request that the Dean recommend that the VPA direct the incoming Director of the Writing Centre to include this in their mandate and thus more fully address recommendation 2.c.
Following one of the recommendations of the SSAV, RMC has established a College Success Centre which encompasses a CITL with the mandate to provide learning support to onsite and online students and instructors. The standing up of the College Success Centre and the CITL has provided momentum for one of the recommendations in the IQAP FAR for the BMASc Degrees regarding providing writing and math support to online students. As such, starting fall of 2018, both the RMC Writing Centre and the Math Assistance Centre have been made available also to students in online courses, although with limited hours. This addresses in part ERC recommendation 2.c for the BA (Gen).
Regarding recommendation 2.d, the Degree Programme agrees that instructional design is a crucial element in ensuring the quality of the course delivery process and requests that the Dean of Arts will work with the VPA, as well as the College Information Systems (CIS) department, to ensure that instructional design resources are optimized.
Dean of SSH's Responses
As a result of the recent SSAV, the way in which library resources are purchased and maintained is changing. This should result in greater consistency and availability of digital resources for onsite and distance students alike.
The Dean supports the need for an online learning module to assist students in familiarizing themselves with how to access library resources as well as the requirements of university-level research. The Dean will assist the Associate Dean (Programme) in determining how best to
fulfill this need, potentially through the CITL under the VPA. Note that this would benefit onsite students as well as distance students. Furthermore, a review of current limitations on the Chief Librarian’s delegated authority for the purchase of electronic journals (e-journals) is currently underway. Improved access to consortia pricing for e-journals will improve the experience for all RMC students.
The Dean of SSH recognizes that IT support for our online learning management systems (LMS) is an important issue in ensuring that BA (Gen) students are able to undertake their courses as required. The Dean has addressed concerns with the level of IT support to the Cmdt, Director of Support Services (DSS) and Principal. An IT service review was conducted in 2015 by DSS that may inform this work. Interruptions of LMS that affect our distance students will be reported in order to ensure improved service in the future. Effective April 2020, CIS support transitioned from RMC to CDA, so ensuring service levels are maintained will require additional effort.
The Dean of SSH fully supports the Departmental Response that more Writing Centre (WC) resources must be made available to RMC offsite students. The standing-up of the CITL will provide two full-time equivalent employees to the WC. This will allow WC resources to be available to more students during both the academic workday and evenings.
The Dean of SSH supports this approach, noting that a total of three Instructional-Designer (ID) positions are on the establishment and could be funded if considered a priority. The Dean of SSH will work with the Vice Principal Academic to ensure adequate ID resources are allocated.
Issue #3 – Instructor compensation and performance
The ERC had three recommendations related to improving both the instructor compensation and their performance.
- Ensure that contract instructor compensation is in keeping with practices at peer institutions. The rate of pay for contract faculty may result in difficulty in recruiting top talent. When reviewers inquired about the rate of pay, one administrator likened the work to that of a teaching assistant, suggesting that little course preparation was necessary due to the fact that content material has been prepared and that it was a matter of facilitating the course. Given that students are enrolled in university-level courses in a research institution, it would be much better for the student experience if all instructors, whether contract or permanent, had input into course content.
- Look for a means to compensate contract faculty for hours worked after the exam period. Reviewers heard that contract instructors regularly work with students after the exam period and that they are not compensated for their time. When students need extra time to complete course assignments or the exam is postponed, there is currently no mechanism to pay contract instructors. Reviewers recommend working to find a solution to remedy this situation.
- Reduce turnaround times for feedback on assignments. Students expressed frustration at the length of time it took to receive grades and comments on assignments. Students find it
helpful to know how they are progressing in a course and use this information to decide when and how to seek support for their learning. This is particularly challenging for students who are at a distance and have no opportunities to get regular feedback in class.
Degree Programme’s Responses
The question of fair compensation and changing the associated attitudes about what is involved in distance-learning course instruction are crucial to the degree’s continued success, and links to the need for full integration of the degree programme with the Faculty of SSH. The approach to distance courses that sees their implementation as facilitation or tutoring is not on a par with practice in peer institutions, nor is it acceptable in terms of the nature of the degree programme. The BA (Gen) is a degree programme, not a distance degree programme. It must meet the same standards of instruction and course quality as any other undergraduate degree, onsite or distance. The degree programme agrees with the ERC recommendation that these issues must be addressed and that instructor compensation and performance is key in doing so. The Associate Dean of SSH (Programme) requests that the Vice-Principal Academic prioritize changes to the contract process to address compensation levels (3.a), the need to establish a mechanism to compensate work after the exam period (3.b), and the need to build in an accountability mechanism for assignment feedback (3.c).
Dean of SSH's Responses
The Dean of SSH supports all the recommendations of the ERC and the programme’s approach. Specifically, the Dean will work with Faculty Services to undertake a review of practice and compensation levels of other institutions. The Dean will work with VPA to bring RMC practice in line with best practice (3.a). The Dean will task Associate Dean of SSH (Faculty) to work with Faculty Services to ensure that a clause is added contracts to provide for compensation in situations where instructors continue to work with students after the exam period (3.b). The Dean will task the Associate Dean of SSH (Faculty) to work with Faculty Services to insert an accountability mechanism in instructor contracts. The Dean of SSH will work to ensure that course evaluations and other feedback are used in decisions to select instructors (3.c).
Issue #4 – Degree Level Expectations (DLE)
The ERC made two recommendations related to DLE and course mapping.
- RMC review degree requirements to ensure that they are aligned with peer institutions. Ensure that the number of core courses, number of courses required for the concentration and the number of upper year courses required is appropriate and aligned to degree requirements peer institutions.
- Continue to work toward a greater alignment between DLE and course- and programme-level learning outcomes. The shift to articulating student learning by way of learning outcomes and the necessity of mapping these to the Ontario DLE’s presents a considerable challenge to faculty and administrators at RMC and other institutions. This transition does not happen quickly or easily, and often is something that takes many attempts to reach the level of refinement. In successive redesigns of courses and curriculum, the reviewers recommend that attention be given to this area and that appropriate expertise be sought. RMC is clearly committed to providing students with clearly articulated and appropriate learning outcomes, and continuing to devote resources to this area will be a valuable investment for the future.
Degree Programme Response
The Degree Programme agrees with both of the ERC recommendations. The Associate Dean of SSH (Programme) will establish a committee to examine this issue.
Dean of SSH’s Response
The Dean of SSH supports this action.
Issue #5 - Accessibility
RMC must continue to make accessibility, diversity and inclusivity a priority for course design and redesign. The Accessibility for Ontarians with Disabilities Act requires post-secondary institutions to ensure that courses comply with the requirements of Universal Design for Learning. It is clear that RMC places great value on providing accessible learning environments for all of its students, whether on campus or at a distance. Ensuring that when courses are designed or redesigned that matters of accessibility, diversity and inclusivity are given priority will be a valuable investment for the future.
Degree Programme Response
The Degree Programme agrees with this recommendation.
Dean of SSH’s Response
The Dean of SSH supports this response. The Dean of SSH will work with CITL to ensure that issues of accessibility, diversity and inclusivity are addressed when new courses are developed or existing courses updated.
Conclusion:
The External Review Committee (ERC) provided a thorough and fair review of the BA (Gen) program. The Dean of SSH and Associate Dean of SSH (Programme) accept the comments and guidance in the report and will work to ensure that all recommendations are satisfactorily addressed.
BA (Gen) Implementation Plan
| Recommendation | Proposed Follow-up and Resource Implications | Responsibility | Timeline |
|---|---|---|---|
|
1. More fully integrate the BA (Gen) into the Faculty of Arts (Recommendation 1)To that end: Develop a budgetary mechanism to create incentives for departments to assign faculty to courses. (Recommendation 1a) 2. Leverage online learning for faculty development in teaching and learning technology-enhanced pedagogies (Recommendation 1b) 3. Increase the percentage of BA (Gen) courses taught by permanent faculty (Recommendation 1c) 4. Increase the range of courses available to enable students to fulfill degree requirements. (Recommendation 1d) |
|
Dean of SSH lead with support from:
|
|
|
5. Immediate action to increase access to library resources and support. |
|
|
|
| 6. Ensure that students have adequate access to information technology support. (Recommendation 2b) |
|
Dean of SSH, DSS | On-going. 15 new CIS personnel will be hired to support IT needs at RMC and other CMCs |
| 7. Ensure that students have access to the Writing Centre. (Recommendation 2c) |
|
VPA, Director of Writing Centre | Completed |
| 8. Increase resources for instructional design. (Recommendation 2d) | Work with the Dean of DCS to ensure that instructional design resources maximized. | Dean of SSH, Dean DCS | Reorganization is complete and the increase of instructional designers by 0.6 FTE is under way |
| 9. Ensure that contract instructor compensation is in keeping with practices at peer institutions. (Recommendation 3a) |
|
Dean of SSH, Associate Dean of SSH (Faculty), Faculty Services | Review has been completed at the change in compensation was completed in 2019-20 |
| 10. Find means to compensate contract faculty for hours worked after the exam. (Recommendation 3b) |
|
Dean of SSH, Associate Dean of SSH (Faculty), Faculty Services | AY 20/21 Implementation Plan TBD |
| 11. Reduce turnaround times for feedback on assignments. (Recommendation 3c) |
|
Dean of SSH, Associate Dean of SSH (Faculty) Faculty Services |
AY 20/21 |
|
12. Review degree requirements to ensure alignment with peer institutions. (Recommendation 4a) 13. Continue work on alignment between DLEs and course and programme learning outcomes. |
|
Associate Dean of SSH (Programme) | AY20/21 |
| 14. Continue to make accessibility, diversity and inclusivity a priority for course design and redesign. (Recommendation 5) |
RMC has a position dedicated to ensuring this | Associate SSH (Programme), CITL | Discussion ongoing on how to best implement a person responsible for EDI at RMC |