TTM4502 (for external evaluators)
Important dates (Autumn 2024):
- Nov 8 (14:00): Deadline for students to submit their reports
- Dec 10: Presentation day (09:15-15:00)
- Jan 7: Deadline for external evaluators to submit the evaluation and feeback forms to the department's administration
- Jan 8: Beginning of the quick-retake period for students whose original report was graded "not passed" and who want to submit a revised version of their report
- Jan 27 (14:00): Deadline for students whose original report was graded "not passed" to submit a revised version of their report or for students that did not submit a report
- Feb 12: Deadline for external evaluators to submit a final evaluation and feeback form for the reports that were (re-)submitted on Jan 27 to the department's administration
Summary of the tasks:
- Read through the reports during the 3 weeks preceding the presentation day.
- Be present on campus on the presentation day and chair the sessions where the students whose reports you have read present their pre-project.
- For each pre-project, fill in an evaluation and feedback form together with the main supervisor and submit the form evaluation and feedback forms to the department's administration.
About the course
This is a preliminary project for the master's thesis for master's students at the department in Trondheim.
The course is assessed as passed/not passed based on the written report that the students submit, but it is also compulsory for students to present their project during a presentation day.
The students may work either alone or in groups of two on a project under the supervision of one or more supervisor. The main supervisor is also an associate professor or a professor at the department. Co-supervisors may be others associate professors, professors, researchers, post-doctoral fellows or Ph.D. candidates at the department or people that are external to the department and possibly the university
The project topics have either been proposed by the one of the supervisor and selected by the students or by the students themselves in colloboration with one of the supervisors.
The report must have a maximum of 15 pages, the structure is given, and the students must use this LaTeX template.
Students that do not submit a report or whose report is graded "not passed" may submit a (revised) version at the end of January.
There are guidelines for the use of AI Tools and the students are required to submit an AI declaration form when submitting their report.
Evaluation of the reports and grading
The grades do not have to be registered and approved in Inspera Assessment. This is done by the course coordinator for all the students.
Each report is to be jointly evaluated and graded by an external and an internal evaluator. The internal evaluator is the main supervisor of the project. The external evaluator is appointed by the department. The department normally appoints 4-5 external evaluators so they each evaluate and grade 10-12 reports.
In order for a report to be graded as passed, all the work items c) – f) and h) listed in the course description under 'course content' must be addressed in a passable manner. Also work item g) must be addressed in a passable manner if specific tools are planned to be used in the work. The assessment criteria for each of the work items are detailed below and are communicated to the students at the beginning of the semester by the course coordinator.
The evaluation and grading must be documented by filling out an evaluation and feedback form for each report. The form will be made available to the student·s, who should take it into account when starting working on his/her/their master’s thesis. All the forms should be submitted to the department's administration within a month after the presentation day.
Evaluating, grading, filling out the evaluation and feeback form and submitting it in time is the joint responsibility of both the external evaluator and the main supervisor.
Assessment criteria for each of the work items
The student·s shall explicitly provide a motivation for the assignment. The motivation of the assignment should discuss how the research project (1) relates and contributes to solve or address societal challenges, and / or (2) contributes to advance the research field by providing new knowledge or refine methodologies. The motivation of the assignment should be qualitatively discussed by the student·s. The student·s should provide references to other sources that support the motivation for the assignment.
The project must be placed into the context of current knowledge and practice. A limited literature survey should be included, featuring prominent research both in the general area and in research closely related to the project. The most important unsolved problems in the related areas should be identified and an explanation provided of how the planned work will contribute to addressing one or more of those problems. Relevant literature should be cited appropriately. The reference list must be consistently formatted with a focus on peer-reviewed literature but supplemented by web and proprietary documentation as necessary.
The project must have clearly defined objectives, aimed to address some well-defined specific research questions related to the motivation. The objectives should be clearly stated in the report. A good research question should be feasible, interesting, novel, ethical and relevant (FINER). In addition to the motivation, the research questions should also be driven by the hypotheses. The research questions and hypotheses should be developed before or at the start of the work and guide the research objectives throughout the work.
The student·s must demonstrate that it is understood what a method is. One definition in this context can be is a particular systematic established procedure for approaching the objectives. The nature of the research wuestions must be defined. Research questions can be related to a traditional science or a design science context or combinations. Traditional science is related to models of systems in the real world, while design science is related to the modeling and also validation of human created artifacts. Research tasks and method must be defined related to research questions. Research questions can be related to various research tasks. Examples of research tasks are: constructive design, traffic modeling, behavior simulations, traffic simulations, measurements, validation of design and traffic models verification. Methods are related to the nature of the research tasks. The research method specifies in more detail how the research tasks are accomplished.
By the end of the pre-project, the student·s must clearly present an assessment of the tools expected to be used in the thesis. These tools may include software, hardware, theoretical frameworks or a combination of any of them. The mandatory assessment shall include a reasoning behind the choice of each tool, referring to the required functionalities and how they can be exploited to address specific issues in the thesis. For each tool the student·s shall perform an initial hands-on evaluation, confirming with mock experiments that expectations are met and necessary resources are accounted for. The student·s should also include details of existing documentation, development communities, as well as known drawbacks/disadvantages when compared against known alternatives.
The work during the thesis project should be divided into a reasonable number of tasks (~ 5) adapted to the project. If not obvious, there should be a brief motivation for the subdivision into tasks. The task objective, technical approach and expected outcome should be specified together with start and end dates and resource utilization (working hours, lab usage, tools, etc). Milestones should be put at reasonably regular intervals (~ average 3rd week, at end of tasks) where specific intermediate results should be ready. Milestones should enable tracking of the progress of the work. Dependencies between tasks and if relevant external input, should be identified. Visual support, e.g., as a Gantt diagram, is a pre. Potential risks should be identified and if significant, means to deal with them should be outlined.
How to proceed
It is common practice for the external evaluator and the main supervisor to first go through the report individually and afterwards discuss and agree upon the evaluation and feedback during a physical or on-line meeting. It can also be handled by e-mail, but it might be less efficient.
As for filling out the form, this may be handled by both preparing comments that are merged after the coordination meeting (this is usually done by the external evaluator); alternatively, one of the parties may make a draft that is supplemented by the other.
Since you will have to have meetings with several different main supervisors, it is recommended that you plan a process that suits you and then contact them to inform them of that process.
Presentation day
The final programme for the day is distributed by e-mail at the latest a week ahead of the presentation day.
On the presentation day, there are as many parallel tracks in different rooms as there are external evaluators. Unless otherwise indicated, the programme starts at 09:15 and finishes at the latest at 15:00.
Each presentation lasts for 20 min.; 15 min. for the presentation of the project and 5 min. for questions and comments from the audience. The presentation, questions, comments and answers must be in English.
To ensure and stimulate participation, each student/group shall at least ask one question to at least one other students/groups according to a prepared schedule distributed with the detailed programme for the day. Of course, they may all also ask questions/give feedback to other students/groups as well.
Who attends the presentations
-
You/the external evaluator/session chair.
-
The students that are presenting. It is compulsory for the students to attend all the sessions in the track in which they are themselves presenting. Exceptions are however made for students that have another exam on the same day. Students must be physically present in the room unless otherwise agreed upon.
-
The main supervisors and the co-supervisors working at the department are expected to attend the sessions where the students they have be supervising are presenting. Because of travels and other pressing activities this is unfortunately not always the case.
-
Everyone at the department, employees and students, is invited to come.
Role as session chair; passive listener or challenging questioner?
It is of course up to you. As session chair you should invite for questions from other attendees. It may be hardest to get them from the students, but once a student asks a question, it may inspire others. It is as an advantage that you as external evaluator ask challenging questions. It should also be a preparatory training for what students will encounter after their studies and being able to present their own work and ask critical questions after listening to someone else presenting are part of the general competence the students should acquire through this project. If relevant, it is also an opportunity to give the student(s) the possibility to clarify parts of the report and for you to gain better insight into what has been done and what is planned and a better idea of his/her/their actual understanding, which can be helpful for the evaluation. Some external evaluators also choose to end with a short oral feedback to the student(s) – adapted to the setting, but still indicating where improvements may be made.
Practicalities
When it is his/her/their turn to present, the student(s) should join the MS Teams meeting for the track and share his/her/their screen. The audio on his/her/their laptop should be turned off and the student(s) should use the microphone available in the room.
This setup allows those that cannot attend physically (main or co-supervisors and/or students) to attend on-line and removes the need for any cables/converters.
To be sure that those following on-line also hear the questions that are asked from the audience in the room, the student(s) should repeat the questions before answering them (this also allows the student·s to get a confirmation that he/she/they have understood them correctly).
Remuneration and travel expenses claim
The remuneration covers 2 hours per report, 4 hours for pre- and post-work for all the tasks together, as well as the time spent on the presentation day.
Your travelling expenses would also be covered by the Department.