Welcome to AI. This guide has been prepared to help you and your team get the most out of the course, and to introduce the tools we'll be using throughout the semester. It's important that you understand the pedagogical approaches we will be using, that of the flipped classroom model, and peer -assessment of your work.
Our class takes a flipped classroom strategy, in which class time is often used to review the topics covered in the assigned readings, carry out exercises, and assess your learning, both individually and in groups. In short, this means that most of your learning will be self-directed, and little class time will be devoted to lecturing.
For this reason, it is important that you complete the assigned readings and optionally, view the related videos in advance of each class. If you find yourself struggling with any of the assigned materials, raise these points for discussion in class. Note that many of the classes have a set of optional readings listed. These are only intended as references for those students who wish to delve deeper into any specific topic, but are not required. You will not be assessed on this content.
In addition to the course website, we will be using two tools throughout the course: the Moodle learnware system and Learning Catalytics, an interactive class response system. As expected with any non-trivial software, neither of these systems is without its problems, but to a large extent, they work well for what they are intended to do. Unfortunately, this also means that you'll need to make regular visits to three web sites and keep track of three sets of login credentials.
In advance of the first class, you'll be receiving an email with your Moodle registration details, with which you can log in to the tool.
Moodle is an Open Source Learning Platform or course management system (CMS), similar in concept to McGill's myCourses.
We'll be using Moodle for class discussions, exercise submissions, and all phases of peer assessment. Electronic communication with your instructor should be conducted through Moodle.
Students use PeerWise to create and to explain their understanding of course related assessment questions, and to answer and discuss questions created by their peers. This is a great tool to use to test your understanding of the course materials, and to contribute questions that your peers can use to test their understanding. At the same time, your contributions through Peerwise will form part of your grade so it pays to take it seriously!
In advance of the first class, make sure you've registered with Learning Catalytics with the access code you've been provided. You'll need to bring a tablet, or laptop, or in the worst case, a smartphone, to each class, so that you can participate in assessments with this tool.
Learning Catalytics is an interactive student response tool that encourages team-based learning by using students' smartphones, tablets, or laptops to engage them in interactive tasks and thinking.
We will be using Learning Catalytics for both informal review and formal assessment of your learning through the course. You will be notified in class of planned formal assessments--quizzes--at least one class in advance. These quizzes will be biased toward recently covered material, but will also include questions relating to content seen earlier in the course.
For in-class quizzes, you will first answer the questions individually. Your individual score accounts for 80% of your grade. You will then have another opportunity to answer the questions in teams for a further 20% of your grade. We hope you will take advantage of this opportunity to discuss the quiz items and learn from each other. For each question, if your team answers it correctly on the first attempt, all members of the team earn full marks. Otherwise, if your team answers the question correctly on the second attempt, your team earns half marks. Answering correctly on the third attempt earns your team quarter marks.
Most of your grade in the course comes from your individual assignments and the term project, as well as your peer assessments of these deliverables.
All peer assessment will be conducted through our class Moodle. For many of you, the process of assessing someone else's work is likely to be a new experience, so we offer the following pointers to make this a success.
Peer assessment is the assessment of students' work by other students of equal status. Students often undertake peer assessment in conjunction with formal self-assessment. They reflect on their own efforts, and extend and enrich this reflection by exchanging feedback on their own and their peers' work.
Peer assessment is a powerful meta-cognitive tool. It engages students in the learning process and develops their capacity to reflect on and critically evaluate their own learning and skill development. It supports the development of critical thinking, interpersonal and other skills, as well as enhancing understanding within the field of knowledge of a discipline.
[Only relevant for group work:] Peer and group assessment are also often undertaken together. Typically, the members of a group assess the performance of their peers in terms of their contribution to the group's work.
At each submission deadline, Moodle will transition to the assessment phase. You will first practice with a couple of submissions that were already assessed in previous years. After entering your own assessment of how well these submissions met the criteria of the various rubrics, you will be able to compare with the reference, which was vetted by the instructor. If you find that there were significant discrepancies between your assessment and the reference, make sure you understand where you might have gone wrong. This is very important, because 20% of your grade for each deliverable comes from your assessments. The assessment grade is split between the accuracy of your scoring, and the quality of your written feedback.
Once you have completed the practice round, you can proceed to assess the submissions of other students that have been randomly assigned, as well as your own. Below the set of assessment rubrics, there is a textbox where you can provide some explanation and justification for your assessment, and offer critique that helps suggest improvements that the assessed student should keep in mind. This written feedback is important not only to the student being assessed, but also in the scoring of the quality of your assessment. This is an important opportunity for you to demonstrate that you've thought carefully about the work you are assessing. As such, you earn marks for thoughtful critique that is meaningful from an AI perspective. Don't forget to be positive in your critiques where appropriate! Feedback is evaluated according to the following examples.
| Score | Example | Justification |
|---|---|---|
| 0 | *no comment* or Great work | Trivial feedback that does not give the assessed any explanation for the grades they received. |
| 1 | Could have done with more polish, and better following of the guidelines. In particular, the code documentation could have been improved, and the methodology for testing could have been better explained. |
Meaningful feedback is given, but not all of the criterion are justified. |
| 3 | Criterion 1: Great Criterion 2: The explanation of an obvious appreciation of the performance benefits of alpha-beta pruning is not that thorough, but it achieves the requirements. Criterion 3: There are no graphs but the report explains how to achieve optimal savings in theory. Criterion 4: I can't understand how the heuristic evaluation function works exactly without something like a formula. I think he gives every position weights, but I would have liked to know what those weights are exactly. Criterion 5: Nothing about memory. Criterion 6: I think the program is well organized but the code should of had more comments. |
Meaningful feedback is given and all of the criterion are justified. |
After assessing the work of your peers, you are encouraged to informally self-assess your own deliverable. Take this opportunity to identify what went well, and what that you can do better for the next deliverable.
The due date for each deliverable is noted in the course syllabus and the Moodle course calendar. Assessments are due one week after each project deliverable deadline. The Moodle course calendar can be exported in ics format and imported into your favourite calendar tool.
Students may well be concerned about the peer-assessment process. Here are some concerns that were expressed in the past, and your instructor's response to each:
Perhaps surprisingly, peer-assessment has been found to work extremely well, and may offer advantages over conventional TA-grading. Although graders and TAs have typically taken the same course, or an equivalent, in the past, they are not necessarily experts in the subject either. In any case, since our department decimated the budget for TA and grader support, that option is no longer even feasible.
It's quite likely that some assessors won't take the time to go through your work with anywhere near the level of attention that it merit. However, the same can be true for TAs, graders, academic conference, journal reviews, and job performance assessments. The hope is that with multiple assessors, the effects of outliers are minimized.
Keep in mind that part of your grade for each assignment or project comes from your assessments, both the quantitative portion (scored by Moodle), and your written feedback (scored by human). Given multiple assessments, Moodle generally does a good job of identifying the outliers, because of the differences in scoring against the idealized assessment. Thus, while it is true that sloppy peer assessments have the potential to pull down your grade, this is more likely to penalize a lazy assessor disproportionately because of the metrics Moodle employs.
If so, that's one of the primary benefits of the peer-assessment process, helping to maximize the learning opportunities for everyone in the class. Ideally, every student would benefit from the same high calibre feedback, but if your peers' critique of your own work does not provide helpful feedback, you are free to ignore them, so your subsequent deliverable isn't adversely affected. If you are seeking additional critique regarding your work, please let the instructor know of any specific questions on which you'd like such input.
Not only are the numeric components of the peer assessments of each deliverable worth 15% of your grade, but the additional evaluation of your written feedback, worth 5% of your grade. Thus, those students who make a serious attempt to provide quality feedback are likely to find these efforts rewarded.
Yes indeed, but developing the experience in carrying out such assessment, offering constructive criticism, and reflecting on your own submissions, are important soft skills from which you are likely to benefit significantly, regardless of whether you end up in industry or academia.
If you find yourself in a situation where your peers don't appear to be contributing equally, and you're unable to resolve such issues internally within a few days, or at most a week, please speak with the instructor right away! In the instructor's experience, students in such situations tend to wait until the end of term to raise such issues, by which point it is far too late to address the problem.
When working on the term project, you should regularly assess yourself and each of your group members against these teamwork criteria:
Score the criteria above as follows:
Although such internal scoring is meant to be an informal process, hopefully, everyone in your group is doing well on teamwork. However, if you find that your assessment of any of your group members falls below a score of 10 (out of a possible 18), this is a good indication that it's time for a meeting to discuss your group dynamics. In this case, based on past experience, you are strongly advised to meet with your instructor if you can't resolve matters right away within your group. Please do so before it's too late.
Last updated on 5 January 2020