Student's Guide to CEILIDH *

Student's Guide to CEILIDH *

Version 2.5

S D Benford, E K Burke, E Foxley

ltr @ cs.nott.ac.uk

Learning Technology Research
Computer Science Department
University of Nottingham
NOTTINGHAM NG7 2RD, UK
Revision 3.2 Tue Dec 17 10:42:59 GMT 1996

Contents

1. Introduction

Ceilidh is an on-line coursework administration and auto-marking facility designed to help both students and staff with programming courses. It helps students by informing them of the coursework required of them, and by permitting them to submit their work on the computer, instead of having to print things out and hand them in. It also marks programs directly, and informs the student and teacher of the mark awarded. The marking uses a comprehensive variety of static and dynamic metrics to assess the quality of submitted programs, of which details are in the paper by Zin and Foxley [ref 1] [content] (a copy of which may be stored on-line in Ceilidh, see below). Ceilidh also provides students with on-line access to notes, examples and solutions, and provides tutors with extensive course monitoring and tracking facilities.

This document is a guide for student users of the Ceilidh system.

* A ceilidh is an informal gathering for conversation, music, dancing, songs and stories. Concise OED.
The Ceilidh system acts in a number of ways for students, tutors and teachers, and can support a variety of different courses.

There are different facilities for students (reading notes and coursework definitions, looking at examples, developing programs, submitting and marking work), and tutors (observing submitted work and marks) and teachers (amending course material, setting up exercises, performing plagiarism tests). The appropriate facilities are offered to appropriate users by the Ceilidh system itself, which takes note of the login identification of the user and compares it with lists of authorised users.

2. Using Ceilidh as a student

There are two ways of calling the Ceilidh system. Ceilidh may be used to support several courses in your department. You can either enter the system at a general level, and then choose the particular course you are studying, or you can enter directly into the particular course you are interested in.

To enter the system at the general level, the appropriate command (which should have been set up by your computer systems administrator) is

To enter directly into the course pr1, type

Upon issuing the command ceilidh you will be greeted with the menu shown in figure 1.

lc list course titles | sc move to named course
vcn view course notes |
vp view papers | pp print papers
clp change printer | h for more help
co make a comment to teacher | q quit this session
fs find student | ft find tutees

Figure 1 : System Level Ceilidh Menu

** Example menus are shown in this document. Menus seen in practice may vary slightly from those shown, since the actual menu you are offered reflects only those facilities available at the time.

This is the "system" level of Ceilidh and represents a department-wide view of the system.

The commands which are available at this point are as follows.

which lists a selection of the available papers. If you reply with the short name of the paper (the first word on the line), the paper will be shown on the screen a page at a time through a paging command such as more (defined by your $PAGER environment variable). Diagrams may not appear correctly if you are viewing the papers on a terminal wich does not support graphics in PostScript.

  • If your system has been set up with the appropriate permissions, it is possible to print a given paper which looks interesting using the pp command.
  • Some papers containing diagrams may not view or print nicely on devices without appropriate facilities.
  • h The h command offers a little more information on the significance of the different commands available to you in the Ceilidh system. This command is available at various points when you are using Ceilidh, and should give help relevant at the time.
  • q This is the "quit" command to leave the Ceilidh system, and to return to your ordinary UNIX shell.

    For courses with student registers, the following commands are also available

  • fs To find details about any student registered on any of the courses supported by the system. You supply a logname or part of a full name; the command replies with the names of the courses for which that student is registered:

  • ft To find details of the tutees of a specified tutor. This gives output in the same format as the previous command, but of all the tutees of a named tutor.

    See below for discussion of the clp and co commands, both of which occur at many places in Ceilidh.

    In general you will wish to move fairly soon to work on a specific course which you are studying. A particular course is entered using the sc (select course) command by typing for example

    followed by return to enter the course "pr1". You must use lower case/upper case (small and capital) letters exactly as requested. If the given name is not a valid course, all available course names will be listed and a valid one should be selected.

    The system level can be bypassed by calling Ceilidh to enter a particular course directly with a flag argument -c as in the example

    which will start up Ceilidh in the course pr1 directly. You will find this quicker, and it is more efficient for the computer. Since you will be making extensive use of the Ceilidh system, you will generally find it simplest if you abbreviate this command using the alias facility of the C-shell as in

    This will enable you to type

    to enter the course directly. This alias command should be added to your .cshrc file (you will be given more details in lectures) if it is not already there.

    The first time you call up a particular course you will be asked if you wish to register for the course. This assists the administration of the course.

    3. The course and unit level

    When you have selected a particular course, the menu shown in figure 2 should now be displayed on the screen.

    lu list unit titles | su set unit code
    lx list unit exercise titles | sx move to named exercise (1)
    lux list units and exercises | state current exercise state
    vn view notes on the screen | pn print notes on letter13
    csum read course summary | usum read unit summary
    vm view all marks | co make a comment to teacher
    clp change printer |
    h for more help | H more help
    kh keyword help |
    q quit and save | q! quit Ceilidh, no saving

    Figure 2 : Unit and Course Level Ceilidh Menu

    This menu is identical whether it is obtained from the system level of

    Ceilidh using the sc command, or by entering Ceilidh with a -c argument as in

    We are now in a chosen course. The various possible commands have the following significance.

  • lu Each course is divided into a number of units, rather like the chapters of a book. This option lists the name of each unit, giving you a brief outline of the course as a whole. Typical output might be

  • lux This command lists all units and exercises within these units. Since there can be a lots of output (100 lines for course "pr1") the option of printing the list is allowed. You will not generally need this command.
  • csum This is a general "course summary" facility. If the teacher remembers to keep the information up-to-date, this command gives you a summary of the timetable for your course, with details of the courseworks to be set, and the hand-in dates for each one.
  • vcn There may be overall notes for the whole course available; these can be viewed using vcn (view course notes).
  • state As a course progresses exercises are opened, made late and then closed. This command gives a summary of the state of each exercise. Typical output might be

    For each exercise set by the teacher, the command gives the unit and exercise codes, the weighting attached to the exercise (showing how important the marks are in the overall course total) and whether the exercise is closed (and the date it was closed, you can no longer submit solutions) or open (you can submit solutions).

  • vm (View Marks) This shows you a summary your marks for all the exercises you have completed so far. See below for an interpretation of the output.
  • motd The displays the "message of the day" information. This is displayed each time you select the course, but you may wish to check some of the details again.
  • su This command enables you to select a chosen unit of the course. The menu remains the same, apart from the currently selected unit number which is included at the top of the menu. Commands below which relate to a specific unit use the currently selected unit number.
  • usum This will list a brief summary of the currently selected unit, usually at the level of section headings in the notes.
  • vn This command (view notes) allows you to view on-line the notes for the current unit of the course. If you are at an X-terminal, the PostScript version of the notes should be shown. If not, a crude pre-formatted ASCII version (without diagrams) is offered. The PostScript version is of much better quality. The command pn gives you a printed copy of the (PostScript version of the) notes. In general you will have been given duplicated copies of the notes, so that printing repeated copies is to be discouraged; it wastes paper and depletes the world's forests (this option will only appear if printing notes is enabled). The notes are also available for viewing on the WWW at page

  • q This is the command to quit the system. If you entered Ceilidh at the course level with a command such as

    the quit will return you to your shell. If you entered the course level from the system level using first

    and then

    for example, the quit returns you to the system level of Ceilidh, and you will need another quit to return to your shell.

    Your current unit and exercise will be noted, so that when you re-enter Ceilidh, you will default to the same unit and exercise as when you left. If you wish to quit without saving your current state, use

    instead.

    The following two commands occur at several points in Ceilidh.

  • co At many points in the Ceilidh system, the system allows you to make comments to the course teacher. Comments are always welcome. Comments may be a request for help ("What do you mean by .... in this week's question?"), a criticism of the system ("I think the mark it gave me was not fair"), or an apology for the late hand-in of work ("Sorry but I had an examination ..."). Please feel free to use this facility; the teacher will try to answer most queries. The comments are sent using email to the teacher in charge of the course.
  • clp Allows you specify the most convenient printer for printing questions and course notes. You specify two printers, one for simple character printing (ASCII files) and one for papers involving graphics (PostScript). You will be told appropriate printer names in class.

    The h (help) command offers you a brief summary of each command on the menu. The H command will show you the student guide (this document!) on-line. The kh (keyword help) asks you for a keyword, and searches all the Ceilidh documentation for that keyword. For each file in which it finds the keyword, it shows the name of the file, the line number (to give you a feel for where to look) and the actual line (so that you can see whether the context is relevant).

    To work on your coursework, you will need to move from the "unit" level of Ceilidh into the "exercise" level.

    4. The exercise level

    If, for a given coursework, you are asked to solve a nominated coursework exercise in this week's unit of the course, you will perhaps first select the appropriate unit using the su command, then list the names of all the exercises in this unit using the command

    at the course/unit level, and then enter the required exercise using

    for example, to select exercise 2 of the current unit.

    It is worth noting that at the course level, while the sx (select a particular exercise) command moves you to another level, the "exercise level" with another menu, the su (select a unit) command leaves you at the course level with the same menu. You can move around the different units in a course at will without changing your level in the system. To attempt exercises you must enter the exercise level, which has different menus depending on the type of exercise you are asked to complete. These exercises include compiled language exercises, interpreted language exercises, question/answer exercises and text submission (essay) exercises. For the moment we will consider the compiled language exercise menu.

    If you type

    to select exercise 1 in the current unit of the course you will see the menu given in figure 3.

    vq view question on the screen | pq print question on draft13
    cq copy question to file |
    co make a comment to teacher | set set up coursework
    h for context help | H for general help
    q to return to calling menu | q! quit Ceilidh
    ed edit your program | cm compile your program
    cv compile verbose | cks check whether submitted OK
    run run program interactively | rut run yours against test data
    sub submit/mark your program | std look at the test data
    rtn run solution executable | rtt run sol'n against test data
    state check details of exercise |

    Figure 3 : Exercise Level Ceilidh Menu

    This is the level at which most of your work will be undertaken. Each exercise will have been set up by the teacher, and will include a question, a skeleton solution, and all the necessary testing information.

    Your normal sequence of activity at this level might be as follows. First use

    (view question) to look at the question, or pq to print it out. You may need to study the question for a while before attempting its solution on the computer. It may be sensible to view or print it at least a day before the laboratory session during which you solve the problem.

    There is also a command cq which copies the question to a file in your directory. This can be useful for extracting test data, or for transferring the question to a floppy disc.

    You will then use

    to set up a skeleton solution. This command typically puts an outline of the required program into your directory, to give you a flying start in solving the problem. In more complex exercises later in the course, it may set up other data files as well.

    At this stage you can start to develop your program, using the commands

  • ed to edit your program,
  • cm to compile it (if the compilation fails, go back to ed to correct the error with the editor, and then try compilation again), and
  • run to try running your program. It is up to you to think of appropriate tests when running your program, to convince yourself that it is running correctly. If your program takes arguments, these can be passed to your program by typing "run <argument 1> <argument 2> ...." .
  • cv This command is given as an alternative to the cm command. When used it will compile your program more verbosely, giving compiler warning messages which can help identify problems in your solution.
  • db If this option has been set up by the course developer, it offers debugging facilities to you.

    Note: Not all of the options in the menu will appear on the screen at all times; if there is no executable, for example, the running options will not appear. If you have not executed set to obtain an outline program, the ed command for editing your program will not be shown.

    Once you have successfully compiled your program and tested it to your satisfaction, the system is ready to mark and submit it. It does this by looking at your program source code (checking that it is indented correctly, for example), and running your compiled program against various sets of test data and seeing that it produces the correct results. At this stage you may wish to use the following commands.

  • rut This runs your compiled program against the first set of test data used by the marking process, and enables you to see whether it appears to produce sensible answers.
  • std (show test data) This shows you each set of test data being used by the marking process. The teacher reserves the right to change the test data at any time, since your program should generally work on absolutely any data which it receives.

    When you have performed enough tests to convince you that your program is correct (and only then) you should ask the system to mark and submit it using the

    command.

    The exact form in which the computer responds with your marks depends on how the teacher has set up the system. A typical reply might be something like that shown in figure 4.

    item mark out of
    Simple test 50 50
    Negative distance 0 10
    Check "feet" "ins" 40 45
    Inches > 12 10 10
    Negative inches 0 10
    Score for Dynamic Correctness is 80.0%
    Mark summary
    category mark out of
    Dynamic correctness 55 69
    C++ typographic style 12 17
    C++ complexity measure 7 7
    C++ program features 3 7
    Overall mark awarded 77 100

    Figure 4 : Output from the marking command

    The significance of this output is as follows.

    Firstly your compiled program is run against several sets of test data. The system looks in the output generated by your program for evidence that you have produced the correct answer; this can be a non-trivial operation if your program does not print its results clearly! Each test produces one line of output, giving you a brief summary of the test, and the score you have been awarded. Different tests will be marked out of different totals, depending on the importance of the test.

    The marks from these runs against test data are then combined into a single "dynamic test" result for your program.

    This result is then scaled out of a particular value, and the next few lines give marks for various "static tests" (tests performed by looking at your program source, rather than by executing it) such as "typographic style" (your program layout, choice of identifiers, use of comments, etc, see the ASQA paper [ref 2] [content] for details of the theory, and Cleve Gibbon's document [ref 3] [content] for Ceilidh's implementation, copies are stored on Ceilidh), "complexity" (the complexity of your program is compared with the complexity of the course developer's model solution; the two should not differ by too large a factor) and lastly "features" (the computer looks for specific good or bad programming features associated with this particular coursework).

    All these marks are then combined with their weightings into a single mark which you are awarded. The Ceilidh system retains a copy of your program and of the mark awarded for future reference.

    If you are happy with the mark awarded, you can quit at this stage. Alternatively, you may try to improve your mark and try again. It is your last mark which is recorded as your actual mark for this coursework.

    To check that the mark has been correctly stored by the computer, use the command

    (check submission) which will show you what the computer has recorded. You should always use this checking facility after every exercise.

    item grade weight
    Simple test A 50
    Negative distance E 10
    Check "feet" "ins" D 45
    Inches > 12 A 10
    Negative inches E 10
    Grade for Dynamic Correctness is C
    Mark summary
    category grade weight
    Dynamic correctness C 69
    C++ typographic style D 17
    C++ complexity measure A 7
    C++ program features D 7
    Overall grade awarded B

    Figure 4A : Alternative output from the marking command

    5. Alternative marking formats

    The teacher can alter the above marking output in various ways.

  • The areas marked may be varied. In the early stages, perhaps only the dynamic correctness of the program will be measured. Later on we may add other categories as required. During a part of the course in which program run-time efficiency is discussed we may add efficiency as one of the areas to be assessed.
  • The marks may be given out of some smaller total such as 10, instead of being given as a percentage. This is to dissuade you from "tweaking" your program to obtain just one more percentage mark.
  • The marks may be given as grades (perhaps grade "A" for the best work, down to grade "E" for a fail). This again is to discourage "tweaking". Typical output in this style is shown in figure 4A.

    6. View marks

    There is also a command at the course/unit level "vm" which lets you view ALL your marks submitted so far. You can either view all the marks you have ever submitted, or just the last mark for each exercise (which is the one that matters). Typical output viewing just the final mark for each exercise might be as shown in figure 5.

    Student Pat R Gross
    Unit 1 exercise 1 mark 100/100 avge 98/100 weight 1
    Unit 2 exercise 1 mark 24/100 avge 96/100 weight 2
    Unit 2 exercise 7 mark 26/100* avge 93/100 weight 3
    Unit 2 exercise 8 mark 18/100 avge 93/100 weight 4
    Unit 3 exercise 8 mark 100/100 avge 92/100 weight 5
    Unit 3 exercise 9 mark 100/100 avge 94/100 weight 5
    Unit 4 exercise 3 mark 34/100 avge 93/100 weight 4
    Unit 4 exercise 4 mark 31/100 avge 96/100 weight 3
    Unit 4 exercise 7 absent avge 87/100 weight 5
    Unit 4 exercise 8 mark 98/100 avge 87/100 weight 8
    Unit 4 exercise 9 absent avge 84/100 weight 5
    Unit 5 exercise 1 mark 100/100 avge 98/100 weight 4
    Unit 5 exercise 4 mark 93/100 avge 92/100 weight 7
    Unit 5 exercise 7 absent avge 81/100 weight 8
    Unit 6 exercise 2 absent avge 92/100 weight 4
    Unit 6 exercise 4 absent avge 98/100 weight 4
    Unit 6 exercise 6 mark 38/100 avge 93/100 weight 4
    Unit 7 exercise 8 mark 100/100 avge 98/100 weight 8
    Unit 7 exercise 9 absent avge 88/100 weight 8
    Unit 7 exercise bg2 absent avge 93/100 weight 15
    Unit 7 exercise n1 absent avge 95/100 weight 15
    Unit 8 exercise 2 mark 100/100 avge 90/100 weight 15
    Unit 7 exercise n2 mark 100/100 avge 83/100 weight 30

    Figure 5 : Typical vm output

    This output shows, for each exercise forming part of the assessment so far, the mark you were awarded (or a grade, or "absent" if not submitted, the mark followed by a start if the work was late), the class average for that exercise (some exercises are hard, some are easy, compare yourself with the class average to see how you are doing), and the weighting to be attached to that exercise in the overall total.

    The penultimate line of the "vm" output gives your position in the class, and your overall mark or grade (formed by adding the weighted marks of all the exercises together). If a mark is given, a scaled mark will also appear; Ceilidh marks are normally scaled down to comply with standard educational borderlines. A pass mark in Ceilidh is generally about 70%, in examinations about 40%.

    The last line gives the number of the exercises you have submitted, the number submitted late in brackets, and the total number of exercises, and you total number of submissions.

    Notes:

  • (i) Do not waste hours trying to obtain an extra mark or two. It is a misguided waste of your time. Once you have achieved a good overall mark, leave the Ceilidh system and work on your other courses!
  • (ii) Do not use the system to find bugs in your program. Design and test your program thoroughly yourself before you submit it to Ceilidh for marking.
  • (iii) If you have to break in (use control-C) during the submission process, this gets counted as a submission, with a mark of zero awarded.Always check beforehand that your program doesn't get stuck in a loop, breaking in loses you one of your submissions.
  • (iv) Always check the submitted work (with "cks") and marks (with "vm") to see what is stored by the machine in your name. If you think that this is not what you actually submitted, report to the teacher.

    Other commands at this level are:

  • vs, ps, These commands are available only after the hand-in date of the coursework, and let you view the solution ( vs ) to the coursework, print the solution ( ps ), and copy the solution into your own directory ( cp ) so that you can try it out yourself.
  • rtn, rtt: These commands allow you to run the teacher's compiled program interactively ( rtn ) to see that it works the way you expected, and to run it against the first set of test data ( rtt ) to see the output which it gives. This may give you ideas on how to layout your output. These options may not exist if there is insufficient space on the disc for the teacher to store executable versions of all the solutions.

  • state Gives you details on the state of the current exercise. Information provided includes the maximum number of submissions, the minimum time between submissions, the number of submissions made so far, and some context sensitive help. Typical output is

    If you have completed at least one submission of your program, you will see a further command

    on the menu. This retrieves your submitted source code back into your own directory.

    When you quit ( q ) from the exercise level of Ceilidh, you return to the course level of Ceilidh, where you may perform other activities, or execute another quit to leave Ceilidh completely.

    7. Interpreted language exercises

    The menu and process for interpreted language exercises is similar to the compiled language menu described in the previous section. The compilation commands are, of course, excluded.

    8. Question/answer exercises

    The exercise level menu for these exercises is completely different from that of the Compiled Language menu shown above.

    For Question/Answer exercises you are given the following menu.

    vq view questions | pq print questions
    ans answer questions and submit | cks check submitted
    h help | q return to calling menu

    The options have significance as follows.

  • vq This allows you to view the questions before attempting to answer them. The pq command can then be used to obtain a printout of these questions.
  • ans When you are happy you know the answers to the questions set, you can enter your solutions using the ans command. This will then ask you the questions one at a time and read your response. Answers may be a choice between a few options, a word or a short sentence. To quit the exercise before answering all the questions type q as your answer.
  • cks This command allows you to check that your mark has been submitted correctly, and to check your answers.

    Some question/answer exercises are purely for collecting answers, such as those to the end-of-course questionnaire. Other will involve answers which are marked. The questions should make clear which of these cases holds.

    9. Text submission exercises

    For text submission (essay) type exercises, the system gives the following menu.

    vq view question | pq print question
    set setup a skeleton document | h help
    q return to calling menu | q! quit Ceilidh
    ed edit your document
    sub submit your document | cks check submission

  • vq To view the question,
  • set to copy the skeleton solution to your directory (typically some section headings for the essay you have to write),
  • ed to edit the skeleton solution, producing your essay or report, and
  • sub to submit your file. Note that when you submit a text file, an entry of zero is recorded in the marking file purely for administrative reasons, so that the marking process has evidence of submission.
  • cks In text submission exercises, "cks" allows you to check your submitted work.

    10. Miscellaneous additional Ceilidh commands

    At most points in Ceilidh, there are three additional commands available.

  • cd directory
    This permits you to change directory within Ceilidh. It does not change the directory you will be in when you leave Ceilidh.
  • !ls -l
    Any command starting with an exclamation mark is treated as a UNIX command, and executed.
  • EDITOR=vi
    You can temporarily reset your environment variables temporarily using this construction. There must be no spaces in the command. The setting is lost when you leave Ceilidh, or when you return to a higher level.

    As you become a more experienced programmer, you may find the system becomes restrictive. Courses that teach modular programming and hence require more complicated compilation techniques often require the student to leave the system to compile their solutions. To overcome these problems an alternative interface to the system has been developed.

    11. The command line interface

    This is a completely new interface in which, instead of using menus, each Ceilidh facility is represented by a UNIX command. It can be used on any terminal. Because there are no menus in this system, it is recommended that you use it only after some experience of the menu system. See also the CLI [ref 4] [content] document.

    To use this facility, there are two things you must do. First execute

    to set up an appropriate environment. You will need to check with your teacher just where the ~ceilidh directory is on the machine. This needs to be done once only (unless at a later stage you wish to reset your environment).

    In order to use these commands, the directory containing them must be included in your PATH variable. To do this, type

    at the start of each logged-on session during which you wish to use Ceilidh.

    From here on, type

    to get a list of Ceilidh commands currently available, or

    to show the currently set course, unit and exercise.

    The commands follow generally the pattern of the menu commands, but a few have had to be renamed to avoid clashes with existing commands.

    A typical starting sequence might be

    Command Purpose
    commands See commands available
    set.cse pr1 Select course "pr1"
    commands See extra course commands
    lu List unit titles
    set.unit 4 Set a particular unit
    lx List exercise titles
    set.ex 4 Select exercise to solve
    vq View question
    setup Set up program skeleton
    ep Edit program
    cm Compile program
    run Run program
    sub Submit
    cks Check submitted

    Advantages of the command line interface

    With this interface, you can execute other non-Ceilidh commands or even log out at any point. When you resume, the course, unit and exercise will remain set just as when you last issued a Ceilidh command (although you may choose to execute "status" to check the settings). This interface will be particularly useful for the "pr2" course, in which you need to perform all compilations yourself.

    With this interface there is never any need to use "q" to quit the various levels of Ceilidh.

    At any time, type

    to remind yourself of the commands currently available. The command

    shows the currently set course, unit and exercise.

    Typing

    will clear out the currently set values for course, unit and exercise. You will then need to use "set.cse", "set.unit" etc to reset them to the values you require.

    12. General points

    At certain times, the teacher may close a complete course, or a unit, or an exercise. These perhaps represent parts of the course which are under development, or which must be kept unmodified for administrative reasons.

    If a prompt appears as, for example

    the letters in between the square brackets give the alternative replies, the first one is the default if you just hit "return".

    13. Conclusions

    The Ceilidh system is an essential part of your learning process; learn to make good use of it.

    Copyright Eric Foxley, Nottingham 1996

    References

    Notes converted from troff to HTML by an Eric Foxley shell script, email errors to me!