The Design Document for Ceilidh Version 2 S D Benford, E K Burke, E Foxley, N Gutteridge, A Mohd Zin ltr @ cs.nott.ac.uk Learning Technology Research Computer Science Department University of Nottingham NOTTINGHAM NG7 2RD, UK ====================================================================== Note: Revision 3.2 October 23, 1996 ====================================================================== 1. Introduction Ceilidh is a course management system. The main objective of Ceilidh is to support teaching and learning through com- puter. As a course management system, it also provides some facilities for the course teacher to organise the teaching and to monitor the progress of the students attending the course. Historically, Ceilidh was designed for teaching and learning of programming through computer.[1] The main motivation for Ceilidh was to solve the problem of teaching a large pro- gramming class. It has long been realised that programming cannot be learned without extensive hands-on experience. It is also necessary that all programs written by students should be marked and checked by the teacher, and the feed- back should be given to the student as soon as possible. However, handling and marking 150 or 200 student programs every week is impossible to perform quickly and fairly by hand. Ceilidh allows students to develop their programs on the terminal, and each program will be marked automatically. The experience with the first version of Ceilidh has motivated us to explore further. During this present stage in the development of Ceilidh, efforts are being made to provide a more comprehensive system, covering other types of courses. This document describes the design of Ceilidh ver- sion 2, due for release mid-1993. 2. Usage of Ceilidh 2.1. Users The users of Ceilidh can be divided into five categories: o student: someone using the system as a learning tool o course tutor: those with access to student pro- gress monitoring o course teacher/administrator: the person in charge of managing a course o course developer: the person in charge of the creating of a particular course o system manager/administrator: the person in charge of the running of the whole system Facilities provided by Ceilidh for each group of users are as follows: Students: - reading documents about Ceilidh. - selecting course and reading course summary. - selecting unit and reading unit notes for a particular course. - answering exercise, submitting answer, and in some cases getting automatic feedback. - make comments to course teacher. - getting some information about the progress of the course. Tutors: - view students' work. - mark students' work, automatically or by hand. - enter students' mark, in certain cases. - view students' mark. Course Administrator: - edit course "motd", "summary", "weights", and "scales". - set exercise as "open", "late" and "closed". - register students. - search for missing and unknown students. - informing students about the progress of their work. - gathering of overall exercise metrics. - search for plagiarism. Course Developer: - creating new unit and edit notes for that unit. - creating new exercise and edit question for that exercise. - set solution for each exercise. System Manager: - install system and users. - install and setup New Course - Edit system "motd", "staff list", "tutor list", and "help" files. - Monitoring the progress of the whole system. The permission for students are controlled mostly by the standard UNIX facilities. For the tutors and teachers facilities there is extensive use of SUID programs to permit access to confidential areas of the system. For developers and managers, they must login as "ceilidh" to perform their activities. 2.2. Basic concepts Some basic concepts of Ceilidh are described here: System We must distinguish at all times between the system and the courses which run underneath it. Papers and documents Within the Ceilidh system, there is a collection of research papers and documents related to Ceilidh which can be read by those who want further information about Ceilidh. Message of the day, "motd" Following the UNIX style, the system manager can make an urgent message available to all users of the system by using the Ceilidh system "message of the day" "(motd)" file. Similarly, the course teacher can inform students of urgent news by using the course or unit "motd" file. Course A course in Ceilidh is given a course name (up to three characters) and course title (one line). Each course is divided into a number of units. Within each unit there are unit notes and a number of exercises. Exercise Exercises can be divided into three types: o programming exercise o question/answer exercises o text submission (essay) Programming exercises can be divided to two types, depending on the type of language. o Imperative programming language: for example Pas- cal, Ada, FORTRAN and C. To write a program in this type of language requires an "edit, compile and run" development cycle. o Interpretive language: for example shell program- ming, Awk, BASIC, LISP, and Prolog. In this type of language, after the program is written, it is directly executed by the interpreter. The development cycle is thus "edit, run". The question/answer exercises allows for exercises involv- ing[2] a number of questions, where the answer to each may be o multiple choice (a single character from a res- tricted set); o a numeric value (or sequence of values, with tolerance specified by the teacher); o simple words (recognised by a regular expression oracle provided by the developer); and soon o simple sentences (with semantic recognition). For copyright reasons, this cannot be distributed at present. The text submission exercises act merely for the collection of files from the students. This can save endless hassle in the collection of paper scripts. 2.3. System overview The Ceilidh system is divided into three levels: system level, course and unit level, and exercise level. System level Facilities provided by Ceilidh at this level are: o select course. o reading papers. o getting some information about the system. Course and unit level Facilities provided by Ceilidh for reading notes: o select unit. o select exercise. o reading the notes for a particular unit. Exercise level Facilities provided for each exercise depend on the type of exercise. In general these facilities include: o read exercise. o prepare the solution for the exercise. o submit answer for the exercise. o look at the teacher's solution - after certain date. 3. Design objectives The design objective of Ceilidh can be described as follows: o To support multiple courses. o To support automatic feedback. o To support multiple interfaces. o To support remote learning. o To allow for extensibility. o To allow for portability. 3.1. Support for multiple courses Ceilidh is designed to support the learning and teaching of many courses. To date most of the effort has been geared towards supporting programming courses. Text submission (essay) oriented courses have now also been included within the system. In future, graphics courses and some mathemati- cally based courses should also be considered for inclusion in the system. 3.2. Support for automatic feedback Feedback from the teacher is important for students because it helps them to improve the quality of their work. 3.3. Support for multiple interfaces Different computer system supports different types of termi- nals. To enable Ceilidh to be used as widely as possible, it is designed to support three types of interface: o X windows interface o Dumb terminal interface o Command line interface 3.4. Support for remote learning Remote learning is an important learning technique in the future. Ceilidh is designed to support remote learning by allowing access through: o Email system o FTP system o Client-server system The degree of distribution of facilities is potentially unlimited. 3.5. Extensibility Ceilidh is designed to be open, so that new courses can be added without major changes to the system. We are gradually making more facilities tailorable, so that typographic, com- pilation and oracle facilities can be added on a per-course basis. 3.6. Portability Portability is very important so that Ceilidh can be used as widely as possible. Apart from security aspects (which rely on UNIX SUID facilities) all other aspects are portable. To ease the process of porting Ceilidh into PC, all file names should now conform to MS-DOS standards. 4. Ceilidh structure In line with the design of most software, the structure of the Ceilidh system is divided into three layers: user inter- face, basic tools and data base. The advantage of this approach is that we can change one layer without affecting the other parts of the system. The structure of the system is described in figure 1. Diagram goes here, see printed notes Figure 1: The Ceilidh data base 4.1. User interface The dumb terminal menu interface is a set of shell scripts stored in the directory "~ceilidh/bin.mnu." The command line interface is also a set of shell scripts stored in the directory "~ceilidh/bin.cli." The X windows interface consists of a set of executable pro- grams stored in the directory "~ceilidh/bin.x." The source programs are stored in "~ceilidh/bin.x/SOURCE." 4.2. Basic tools Basic tools are software tools that are integrated with the system. For example tools for setting up an exercise, sub- mitting an exercise, viewing students mark etc. These tools are divided into two categories: o The system wide tools are stored in the directory "~ceilidh/Tools" . These tools are programs and shell scripts. The program sources are stored in "~ceilidh/Tools/SOURCE" together with a Makefile. o The course related tools (for example the com- piler, static analyser, program marker, essay marker, etc.) are stored in the "course/bin" directory for the course concerned. Our strategy is to look for all available tools first. If the required tools are not available, then we have to develop our own tool. We have used UNIX tools such as "awk" and "sed" when appropriate. Note that the full pathname of the tools must be included in the code to avoid the substi- tution of student rogue versions. 4.3. Data base The Ceilidh data base is simply a collection of files. This data base consists of directories for "help," "papers," "lib" and "courses." Help The help information for the system is stored in two direc- tories: help The "help" directory contains all the text files displayed by the various "help" commands in the menu and command line interface versions. Each filename in general starts with sys for system level help cse for course/unit level help ex for exercise level and ends with usr for student level help tut for tutor level help tch for teacher level help dev for developer help The "exercise-user" level is further split, so that different help is offered depending on the stage the user has reached in the solution of a given exercise; does the user have a source or executable in the directory? xhelp This directory has a help file corresponding to each button on the X windows screen. Papers A number of papers and documents relevant to Ceilidh are stored here. It is our belief that students should have access to all information regarding the workings of the sys- tem, and should be encouraged to read around the topic. Each paper occurs as three files name.ttl a one-line title name.cat a version suitable for viewing through "cat". name.ps a PostScript version The originals (in "troff" format) are stored at Nottingham. New papers could be added and existing ones removed at your discretion. "lib" This directory contains a few miscellaneous text files such as the "roff" macros used in preparing the papers (one set of macros for the ".cat" version, one for the ".ps" version, and one for the ".ohp" version used with course notes), and a default "Makefile" for compilation exercises. Courses Below the ceilidh directory, each course has a directory such as "course.pr1" for the course "PR1" (C++ programming in semester 1). The layout of files under a typical course directory is shown in figure 2. Below the course directory is a "bin" directory containing any shell scripts or execut- able programs (sources and Makefile are in a directory such as "~ceilidh/course.pr1/bin/SOURCE" directory) specific to this course. These may be compilation, debugging or typo- graphic commands, for example. These commands will take preference over any system commands of the same name. Below the course directory are directories for each unit of the course, for example "unit.3" to contain the third unit or the "PR1". The unit name is "numeric", and they will appear in numeric order when listed. Below this are direc- tories for each item of coursework, such as "ex.1" for the first exercise. Exercise names can be any string of up to three characters. All course, unit and exercise directories contain a file "title" containing a one-line title for the course, unit or exercise. The course directories should also contains a "summary" file containing a summary of the lectures, times, courseworks set and hands-in dates. The "summary" files in the unit directories are a brief summary of the content of that unit. The content of the exercise directories depends on the type of the exercise. All exercise directories should contains the following files: type information about exercise type, suffix, compiler late.dat date when the exercise is considered late late lists of all late submissions close.dat date when the exercise is considered closed model.q the exercise question It also include a directory "solns" which contains all sub- mitted solutions from students together with a "marks" file, listing all students' marks, one line per student submis- sion. Diagram goes here, see printed notes Figure 2: A course filestore layout 5. Implementation considerations Ceilidh is designed to run under the UNIX Operating System environment. UNIX has many facilities which have enabled Ceilidh to be developed without much difficulty. Security One of the major weaknesses of the present UNIX file system is the problem of access control. Access to files in UNIX is controlled by dividing users into three categories: the owner, the owner's group and other people. For each category, there are three types of access: read, write and execute. This type of access control mechanism is not sufficient for the Ceilidh system. Some of the files requires access control which allows append only. For example Ceilidh programs can only append entries to the mark files. They are not allowed to change other parts of the file. Another type of access control is read thing relevant to you only. For example for the mark file, a stu- dent user is only allowed to see only the lines giving their own marks, not other people's. For the copies of submitted programs, there is no general public read access, but a stu- dent must be permitted to read their own solution. To solve this problem, we have to use UNIX SUID (set-user- id) and SGID (st-group-id) techniques. We use a number of "ceilidh" owned SUID program for, for example, the marking. Such a program can read files in the "ceilidh" file-system which have no public access, and can write a copy of the student's program within "ceilidh" and ensure that it has no public read access. Such a program must open the Ceilidh files within the program, after the "ceilidh" UID has become effective. The student program may not be readable to an SUID program (the student may not allow public read access to their program) so that in general the student will be fed to the SUID program as standard input. We hope that this problem will eventually be solved when the new UNIX V.4ES is available, where enhanced security features such as ACLs are provided. The major SUID programs are o "ccef" whose main purpose is to copy information into the Ceilidh area, and o "efread" whose main purpose is to read confiden- tial files within the Ceilidh area. Typical files within Ceilidh which must not be publicly readable include: student solutions These are owned be Ceilidh, with access mode 600. They are named "." in general. The student must be able to read their own file on demand, which is controlled by the program "efread" which acts as a controlled version of "cat." marks files These are the files "///solns/marks" in each exercise directory. Ceilidh never overwrites a marks file, each new mark is appended. These have no public access, and a par- ticular student must be able to read on their own entries (lines) from the file. This is done within "efread" by using a "grep" command. Oracle files The files for checking the accuracy of program output must not be publicly readable. The "ora- cle" program itself[3] will be called from the SUID marking program, so will run with UID as "ceilidh," and will be able to read the files. Archiving With traditional coursework and examinations, it is usual that all relevant documentation (exam papers and scripts, student projects etc) has to be kept for a fixed time after the examination. We have built in an archiving command, so that an archive copy of the course (using "shar" ) can be taken at the end of a course for two reasons: o to be kept as required above for regulatory rea- sons; and o to retain a copy of the course at the time of completion, ignoring any later changes that may occur, for access at a later time by the external examiner. Auditing The latest release of Ceilidh includes facilities for main- taining audit trails in the system. An audit trail can be set up to record either o all activities of a given student within Ceilidh; or o all usage of a particular Ceilidh facility (e.g. compiling, or printing). Each audit trail is collected in a separate file, under an "archive" directory, with one line appended per archive event. The line is of the form ::::::: The facilities for analysing these trails are not yet developed. The files can become very large, so any which are set up must be checked regularly. There must always be an audit file for "Error" messages.