Teacher's Guide to Ceilidh 2.6 S D Benford, E K Burke, E Foxley ltr @ cs.nott.ac.uk Learning Technology Research Computer Science Department University of Nottingham NOTTINGHAM NG7 2RD, UK ====================================================================== Note: Revision 2.6 for Ceilidh versions 2.6 or later. Generated December 17, 1996. ====================================================================== 1. Introduction This guide is to help a teacher administer a course which uses the Ceilidh system. It should be read in conjunction with the student[1] and tutor[2] guides. The users of the Ceilidh system fall into the following classes. students Those using the system as a learning tool. They have read access to public information (notes etc) and to their own submitted work and marks. tutors Those with access to student progress monitoring.. They essentially have read access to all student marks and work on a particular course. teachers Administering a particular course. Per- mission to open and close exercise, set exercise weights. Controlled write per- mission to selected files. developers Course material creation and amendment. They essentially have write access to all material. system adminUpdating the system commands. This guide is for users given "teacher" status within a par- ticular course under the Ceilidh system. Any user in this category will automatically have access to all the student and tutor facilities. The tutor essentially has read per- mission to all Ceilidh information, the developer essen- tially has write permission. The facilities offered to a teacher fall between these two, allowing the teacher to amend such features as are thought necessary for the running of a course, without the danger of general write permission to all information. Suggestions on extra features to add to the teacher menu, or features to omit, are always welcome. The support provided by the Ceilidh system falls into the four distinct levels system Department-wide view; public documents visible; nothing special for teachers. course/unit One university teaching module; special teacher menu. exercises Assessment definition for one course- work; special teacher menu. It is important to control access to the information pro- vided by the system. Access to the list of those allocated teacher facilities for a particular course is controlled by standard Unix file permissions, and set by the system administrator. The relevant file is "~ceilidh/course./staff.lst" for teacher facilities (only people whose login names occur in this file have access to teacher administration facili- ties for this course) The administrator or any teacher can edit the corresponding list of those permitted "tutor" facilities, which is held in the file "~ceilidh/course./tutor.lst" Only the main Ceilidh system administrator can change the names in the "staff.lst" files. Any teacher named in that file can access the teacher facilities for this course, and through these can change (command "etl" to edit tutor list) the names of tutors in the corresponding "tutor.lst" file. All access involving write access to Ceilidh files is per- formed for the teacher through a special SUID Ceilidh pro- gram. Filestore layout The current filestore structure is headed by a directory "~ceilidh" with the shell scripts and programs for the com- mands in the directory "~ceilidh/Tools," help information in the directory "~ceilidh/help," and research papers in the directory "~ceilidh/papers." See also the directories "~ceilidh/bin.mnu" for the shell scripts giving the course, student, tutor etc menus, and "~ceilidh/bin.cli" for the command line interface commands. Below the "~ceilidh" directory each course has a directory such as "course.pr1" for the Nottingham course "PR1" (C++ programming in semester 1), called the "course directory". Below the course direc- tory are directories for each course unit (we previously used words such as week or chapter instead of unit ), typi- cally "unit.3" to contain the third unit of the "pr1" course. The unit name must be numeric. Below the unit directories are directories for each item of coursework, such as "ex.1" for the first exercise. Exercise identifiers can be any set of up to three characters. The course directory contains files "motd" (message of the day, displayed when selecting this course in Ceilidh), "sum- mary" (containing a summary of the lectures, times, course- works set and hand-in dates), "notes.cat" (a notes file for the course as a whole), and "type" and "type.tch" defining the overall parameters of the course. The contents of the directories and the commands available at each of these levels are described below. 2. Commands at the system level In response to the command ceilidh the user is offered the standard Ceilidh top level menu, see the student and tutor guides for details. There are no spe- cial teacher commands at this level. 3. Commands at the course and unit level The user may either move to this level from the system level using the "sc" (select course) option, or will normally enter the course "pr1" for example directly, by calling ceilidh -c pr1 either directly or using a preset alias. There will always be a currently set default unit number and exercise name set by the teacher. If you have already used the system, the defaults will be set to the values when you last left Ceilidh. If you are an authorised teacher for the current course, an extra "teach" entry will appear on the course level menu as in the example Course and unit menu for course "pr1" unit "1": ________________________________________________________________________ |lu list unit titles | su set unit code | |lx list unit exercise titles | sx move to named exercise (1) | |lux list units and exercises state current exercise state | |vn view notes on the screen pn print notes on beth | |csum read course summary usum read unit summary | |vm view all marks | |clp change printer h for more help | |co make a comment to teacher q quit | |_______________________________________________________________________| =========================================== Course level command: The teacher administering a course should find a "teach" option at the foot of the course/unit menu. If this is not the case, then your name has not been appropriately set in the course "staff.lst" file by the system administrator. This can be done only from within the "ceilidh" username by the Ceilidh administrator. If the "teach" option appears on the menu, you can enter the teacher menu by typing "teach" at the course menu, or directly by typing ceilidh -c pr1 -x teach as a command; an alias to this command is useful for the teacher. A teacher menu such as the following is then displayed. ====================================================================== Note: * In versions of Ceilidh before 2.6, the course, unit and exercise level commands are combined. In version 2.6 onwards, the exercise commands appear on a separate menu resulting from use of the "sx" command. ====================================================================== Teacher's (course tst unit 2) course menu: _________________________________________________________________________________ |ecm edit course motd | ecs edit course summary | |ewt edit course weights | esc edit course scale factors | |est edit student register | etl edit tutor list | |ety edit course type.tch file | lwwww list WWW exercises | |cs count solutions | | |su set unit | sx set exercise | |vo view overheads | po print overheads to beth | |cc close course | oc open course | |cu close unit | ou open unit | |vm view marks | | |ss register studs from master list | rs register students by name | |csa add unregistered students | csd delete students with no work | |nw set new week's defaults | etl edit tutor list | |________________________________________________________________________________| =========================================== Type teacher course command: The teacher menu at this level offers facilities such as o edit course motd (message of the day) and summary files o edit the weights and scales files o set up student register for the course o check the register against all submitted work for the course report total absentees, offer to delete them from register report unknown submissions, offer to add them to register o edit the register o view the overhead transparency version of the notes o print these overheads to a PostScript printer o set current unit o set current exercise in current unit o set the new week's defaults o open or close the course o view student marks o adjust the course (ect) and exercise (ext) type files o edit the weights and scales files affecting mark calculations o e-mail mark details to students and/or tutors Details of these operations will now be explained further. 3.1. Editing files The various files which the teacher can edit (message of the day, exercise weights, mark scaling etc) are not directly available to the teacher. They are made available by spe- cial Unix facilities. Each time a file is amended, the ear- lier version is archived for reference, so that a history of all the message of the day, for example, is available at the end of the course. These archive files can then be viewed by the teacher. Files in this category include: ecm Edit course message of the day (seen by students when they log onto this course in Ceilidh). ewt Edit the weights file. The entries in this file deter- mine the set of exercises to be assessed and the weight and status. For details see the "vm" (view marks) com- mand below. esc Edit scale factors. The overall mark calculation for a student involves first adding the marks for each exer- cise multiplied by the appropriate weighting factor (from the "weights" file). The total thus obtained is then piecewise linear scaled according to the "scales" file. If the file contains 0 0 70 40 80 60 100 100 then a Ceilidh mark of 0 scales to 0, 70 scales to 40, 80 scales to 60, and 100 scales to 100. Intermediate values are linearly scaled. There is an upper limit of 20 interpolation points. 3.2. Student register The student register for a particular course is kept in a file "students" in the course directory. The format of each line of the file is : (): Users are prompted to register the first time they use a course, although this may be made automatic later. Some teachers prefer to set up the file by hand before the course starts. If the user's login name does not appear in the register, the user is asked to type in the details of their full name (and perhaps tutor). If you have a place where these details cam be picked up automatically, you can replace the code in "~ceilidh/Tools/SOURCE/register.c" at the point where the Nottingham code refers to "/master/common/users," which is out master file of student login names, full names and tutors. It may be better to set up the register by hand before the course starts, since stu- dents' typing is usually at its worst at the start of the course! The tutor's logname is used in various email commands. Commands related to administration are: est : edit the student register file directly This (in common with all other "edit ... file" com- mands) takes a local copy of the file, allows you to edit it, and then copies (after prompting you) the edited version back to the Ceilidh system area. Note that teachers and tutors names need not appear in the student list; they can submit work without being registered students for the course. This allows "vm" (view marks) to ignore them, and to produce mark sum- maries only for real students. ss : register studs from master list We keep a master file of all students, with their login name, full name, course code, tutor, modules being taken, and other information. We then add students to the register from this file by simple selection (per- formed using "grep" command). This command is very site dependent. rs : register students by name The system allows the teacher to give the login names, full names and tutor's login name of additional stu- dents to be registered on the course. csa : add unregistered students The system searches for all files and marks submitted so far, and notes any login names which are not in the student register. The teacher is offered these one at a time to be added to the student register. csd : delete students with no work The system searches for any students who are in the register but who have not submitted any work, and offers interactively to delete them from the register. 3.3. Overhead transparencies Most Ceilidh courses have the notes in several forms in each unit directory, in files named "notes.cat" (ASCII version for an ordinary pager, no diagrams), "notes.ps" in PostScript (for viewing on a X terminal or printing to a PostScript printer), "notes.ohp" (overhead transparencies in PostScript for viewing or printing), and "notes.html" (automatically generated HTML version for the WWW). The teacher can view the overheads (the "vo" option, it needs the Ceilidh environment variable PSVIEW to be set to the name of an appropriate PostScript viewer such as "ghostview" for the viewing) and can print them (either from a button within the PostScript viewer, or by using the "po" (print overheads) command. The latter uses the printer specified by the Ceilidh variable "PRINTER_PS." 3.4. Miscellaneous file edits ecm : edit course message-of-the-day The current "motd" is copied to the teacher's direc- tory. The teacher can then edit the file, and the amended version is copied back to the Ceilidh system. This is used for urgent messages such as lecture re- scheduling and coursework changes. We would expect an archive copy of this file to be kept whenever it is changed, to give a complete history of the course. ecs : edit course summary The current "summary" is copied to the teacher's direc- tory, the teacher can edit the file, and the amended version is copied back to the Ceilidh system. We use this to keep an up-to-date schedule of lectures and work as well as an outline of the course content. ety : edit type.tch file Each course has a "type.tch" file giving course-wide defaults for environment variables. COMPILE Allow compilation in Ceilidh (yes or no) typically "yes" MARKVERB Marking verbosity 0 1 or 2 (usually 2) typically "2" 0 gives no overall mark to students 1 gives just a total mark 2 gives the typographic etc submarks MAXSUB Max no of submissions (0 for no limit) typically "5" MINGAP Min seconds between submissions typically "300" OUTOF Marks scaled out of typically "100" GRADE Give letter grades, not numeric marks "." for numbers, "E" for 5 grades, "C" for 3 etc PRINT_NOTES Allow printing of notes (yes or no) typically "no" PRINT_QUESTION Allow printing of questions (yes or no) typically "yes" VDATA View test data (yes or no) typically "yes" These parameters apply to the whole course, but can be overridden on a per-exercise basis by information in an individual exercise's "type.tch" file, which is edited in a similar way using the "ety" command from the teacher's exercise menu. If the MAXSUB value is negative, say -5, there is no limit on the number of submissions, but warnings will be given after the given number (in this case 5) of submissions have been completed. If the GRADE option is used, the strings printed for each grade and the borderlines to be used can be speci- fied by setting GRADE to a value such as 0:fail:70:pass:90:good:95:super:100 The fields are alternate numbers (borderlines) and strings (name for a mark less than or equal to the upper borderline). The first integer MUST be 0, the last MUST be 100. If the GRADE string is a single character such as E the strings and borderlines default to values stored in the source file "~ceilidh/Tools/SOURCE/grade.c" . To change them your system administrator must edit this file and remake the system. This is generally a more efficient solution. Experience at Nottingham has made us decide on the use of five grades with Ceilidh borderlines below 70 : pass (grade E) 70 to less than 80 : third class (grade D) 80 to less than 90 : lower second class (grade C) 90 to less than 95 : upper second (grade B) 95 or above : first class (grade A) The "ety" command ("edit course type.tch file") offers each environment variable in turn, and invites you to take the default value, or to change it. There is a final confirmation prompt. oc (open course) and cc (close course) After the use of the "cc" command, only tutors and teachers can log into the course. At the end of a course, it is wise both to close it (to avoid any changes taking place), and to take an archive copy of the complete course for reference purposes. ou (open unit) and cu (close unit) Individual units can be opened or closed. If you prefer students not to read ahead, you can close all future units, and open them as you reach that material in lectures. ox (open exercise) and cx (close exercise) This is normally done as part of the "new week" (nw) command. 3.5. Tutors etl : edit tutor list The login and full names of all who require access to the tutor facilities should be in the file in colon separated form typified by ef:Eric Foxley sdb:Steve Benford 3.6. Marking vm : view marks This gives a summary of all marks. It first asks whether the marks are to be viewed, stored (in which case it asks for a filename) or printed. It then asks whether the marks requested are those for a particular student (it will request a login name), or for one exercise (the exercise currently set by "su" and "sx" ), or for the whole (currently set) course. There is a summary at the end of the marks of the breakdown into the different ranges defined by the "scales" file (see below). ewt : edit course weights The file offered by this command (the original is in a file "weights" in the course directory) contains for every exercise set on the course one line containing the unit number the exercise code the weight to be attached to this exercise mark an indicator 0 = open, 1 = late, 2 = closed The total mark awarded by the "vm" command for the course is then the sum for each late or closed exercise of mark awarded x weight factor divided by the total of the weights. NOTE: It is vital that this file be kept up-to-date in order that the student "vm" command works. esc : edit scaling factors The marks from Ceilidh will be high relative to normal examination results. They will perhaps average 95%. The marks may therefore need to be scaled down to match other marks from other courses. The file offered by this command (the original is in a file "scales" in the course directory) is of the form ___________ | 0 0 | | 50 40 | | 70 50 | | 80 60 | |__________| The student summary output from the "vm" command shows the raw mark awarded (the sum of each exercise mark times its weighting factor) and the scaled mark using the above piecewise linear scale. With the file shown above, a mark awarded by Ceilidh of 0 would scale to 0, a mark awarded by Ceilidh of 50 would scale to 40, a mark awarded by Ceilidh of 70 would scale to 50, and so on, with linear interpolation between the given points. A summary at the end of the list of marks gives the marks broken down by scaling interval, so that additional unnecessary entries in the scales file can be used to provide a more detailed breakdown of marks. 3.7. Weekly administration The recommended practice is for each exercise to be "opened" when it is announced to the class, to be made "late" at a certain handing-in time (submissions are still accepted, but are considered late), and to be "closed" (no more submis- sions accepted) shortly afterwards. The "nw" command assists in these three operations. nw : set up a new week This command asks in turn for: Exercises to close The teacher specifies the unit and exercise numbers for any exercises (perhaps already late) to be closed completely. The questions and solu- tions for these exercises are concatenated, and left in a file "closed.qus" in your home direc- tory. It is suggested that these be made avail- able to the students either by duplication or through the filesystem. The end of the list of units and exercises to be closed is indicated by a blank line. For each exercise closed, an appropriate one-line message is appended to the "motd" file. For each exercise closed, the teacher is offered o for the overall class metrics to be computed; o for the plagiarism results to be computed; o for the model solution to be made public. All such computations will be forked off at the end of the "nw" command, and will hammer the machine! The use of these facilities is to be encouraged, since the teacher should be aware of class overall metrics. The teacher will also be asked whether the results as stored for this exercise are to be e-mailed to all students (a confirmation of their mark for this exercise), or to tutors (each of their tutees marks so far on the course) or both. If so, the shell script which is generated is offered to the teacher to fork or execute on-line as required. Each student will be e-mailed his/her own mark/grade, and each tutor the summary of marks so far for all their tutees. The question and solution for each closed exercise are concatenated into a file "closed.qus" which may be printed or e-mailed to students if you so wish. Exercises to make late The teacher specifies the unit and exercise numbers for any exercises to be made late. For each exercise made late, a one-line message is appended to the "motd" file. Again, the teacher will be invited to obtain metrics and plagiarism details. Exercises to open The teacher specifies the unit and exercise numbers for any new exercises to be opened. An exercise weight is requested, an entry for each new exercise is added to the "weights" file. A list of the questions and skeletons is con- catenated in a file "opened.qus" in your home directory. It is suggested that this be printed and photocopied to the class. In all of the above cases, the relevant entry in the "weights" file is appended or amended, and a one-line entry in the "motd" file added. It may be wise to hand edit (using "ecm" ) the message of the day afterwards. After the "nw" command, you may still wish to edit the "weights" file with the "ewt" command to reset the weight (third column) to be attached to that particular exercise. The teacher is then asked for the default unit and exercise to be set, which students will default to when they log in to Ceilidh. All the usual miscellaneous commands apply here, "h" for help, "q" to quit, "EDITOR=vi" to reset environment vari- ables temporarily, "cd" to change directory, and "!command" to execute any command. By default, the "teach" command will move you into the course directory. The above should assist in the administration of the course, and in following up student progress reports. 4. Exercise level commands Versions after Ceilidh 2.5 have split the teacher commands related to a specific exercise onto a separate "exercise level" menu. This menu is reached using the sx command from the teacher's course and unit level menu, or by using the teach command from the student's exercise level menu. The system will change directory to the exercise directory. The exercise level menu looks as follows. Teacher's exercise (course swt unit 7 ex typ_sh) menu: em enter marks ety edit exercise type.tch file nw set new week's defaults pm exercise plagiarism & metrics | q return to user menu h help | H view teacher guide cx close exercise | ox open exercise mi show missing students | uk show unknown students mm mail to missing students | mu mail to unknown students mk e-mail marks | vm view marks ves view exercise statistics =========================================== The commands on the exercise menu are as follows. 4.1. em : enter marks This allows the teacher to enter marks by hand for this exercise; the marks may represent modifications to computer generated marks, or may result from hand marking of work such as essays. See the tutor guide for full details. Bear in mind that, after marks have been entered by hand, there should be no more automatic submissions, since these marks would over-ride the earlier hand entered marks. 4.2. ety : edit exercise type.tch file The marking characteristics for a particular exercise may be modified with this command. Any parameters set here over- ride just for this exercise the course defaults set at the course level. The list of options which can be set are identical to the course level ones (see the course level "ety" command above). 4.3. pm : exercise plagiarism & metrics During plagiarism checks, every submitted program is com- pared with every other one, and a distance metric calculated for each pair. The pairs are then ordered by closeness, and the first few reported. (The number of pairs reported is set by the shell variable "PRINT_NUMBER" in the shell script "~ceilidh/Tools/CPlag" .) For each of the pairs reported the system prints the difference metric, the lognames of the two programs, the reason for the closeness if detectable, the times of submissions and sizes of the two programs. You may choose to infer that perhaps the later program was copied from the earlier; you may choose to infer that if the later program was submitted only shortly before submission time, the student may have panicked and looked for other solutions to copy. Closeness in this test DOES NOT PROVE collaboration. Plagiarism output in version 2.6 is typified by No. 1 : Difference 1 : Users lbc cje : Comments removed lbc Linda Bertha Currie (sdb) 333 May 23 12:12 lbc.C cje Claire Jane Easterbrook (mfd) 318 Jun 6 13:06 cje.C =============================================== No. 2 : Difference 3 : Users prg cje : Idents white space comments removed prg Phil Green (ef) 320 May 15 10:38 prg.C cje Claire Jane Easterbrook (mfd) 318 Jun 6 13:06 cje.C =============================================== No. 3 : Difference 3 : Users prg lbc : Idents white space comments removed prg Phil Green (ef) 320 May 15 10:38 prg.C lbc Linda Bertha Currie (sdb) 333 May 23 12:12 lbc.C =============================================== No. 4 : Difference 68 : Users dem cje : General differences dem David Edward McEwen (mpj) 2665 May 24 12:26 dem.C cje Claire Jane Easterbrook (mfd) 318 Jun 6 13:06 cje.C =============================================== No. 5 : Difference 68 : Users lbc dem : General differences lbc Linda Bertha Currie (sdb) 333 May 23 12:12 lbc.C dem David Edward McEwen (mpj) 2665 May 24 12:26 dem.C =============================================== No. 6 : Difference 68 : Users prg dem : General differences prg Phil Green (ef) 320 May 15 10:38 prg.C dem David Edward McEwen (mpj) 2665 May 24 12:26 dem.C =============================================== No. 7 : Difference 80 : Users sqc cje : General differences sqc Simon Carter (sdb) 2911 May 23 10:48 sqc.C cje Claire Jane Easterbrook (mfd) 318 Jun 6 13:06 cje.C 4.4. mi : show missing students This lists students whose lognames appear in the course register, but who have not yet submitted work for the current exercise. 4.5. uk : show unknown students List the lognames of students who have submitted work, for this exercise, but whose name does not appear oin the course register. 4.6. mm mail to missing students 4.7. mu mail to unknown students 4.8. mk e-mail marks 4.9. eth : edit tutor help file The developer and teacher can both access a file intended to be viewed by tutors. It may contain Likely problems Subtleties Points to watch out for References to books or notes The tutor will view it (ASCII) through a pager. It should NOT be readable by students. ves view exercise statistics This performs a similar function to the overall course metrics, but in a different format. For each metric the output is, for example: Metric: Average identifier length; mark 10 Average 5.687; Stddev 1.035 Borderlines : .... 3(L)- 5(S)- 10(F)- 15(H) Distribution: 3 - 10 - 57 - 0 - 0 L to S spread 0 0 0 0 0 1 1 4 1 3 S to F spread 14 17 11 8 4 2 1 0 0 0 F to H spread 0 0 0 0 0 0 0 0 0 0 This is interpreted as: The "Average identifier length" metric is awarded 10 marks. The overall average was 5.687, and standard deviation 1.035. The four "L, S, F, H" key values (see the Oracle[4] document, full marks are awarded to values between "S" and "F") are 3, 5, 10 and 15 respec- tively. There were 3 students with an average less than 3 (no marks awarded), 10 from 3 to 5 (part marks), 57 from 5 to 10 (full marks), 0 from 10 to 15 (part marks), and 0 above 15 (no marks). Each of the three internal intervals is then divided into 10 sub- intervals for the following three lines, to show whether the students tend to be near a particular bord- erline or not. In the "L" to "S" interval, most stu- dents were near the top end. In the spread of the "S" to "F" marks, most students were at the lower end, so that students are all near the "S" (5) figure. Each metric is detailed in the same format. 5. Summary Teachers should also read the more general papers discussing experience with the use of Ceilidh. A number of points are gathered together in the revised Overview document[3] which is in course of preparation. The document discusses the use of weighting factors for exercises, maximum number of sub- missions, etc. c Eric Foxley 1996 References 1. Steve Benford, Edmund Burke, and Eric Foxley, Student's Guide to the Ceilidh System (2.4), 1995. LTR Report, Computer Science Dept, Nottingham University 2. Steve Benford, Edmund Burke, Eric Foxley, N H Gut- teridge, and A Mohd Zin, Tutor's Guide to the Ceilidh System (2.4), 1995. LTR Report, Computer Science Dept, Nottingham University 3. Eric Foxley, Colin Higgins, and Cleveland Gibbon, An overview of the Ceilidh system 1996, 1996. 4. Abdullah Mohd Zin and Eric Foxley, The Oracle program, 1992. LTR Report, Computer Science Dept, Nottingham University