TCBG Cluster Building Workshop

Evaluation of the Theoretical and Computational Biophysics Group Cluster Workshop

Beckman Institute, Room 5602
March 16-17, 2006

Questionnaire: Gila Budescu, TCB Group, UIUC, and modified by David Brandon, TCB Group, UIUC
Analysis and report: David Brandon, TCB Group, UIUC, and Kathryn Walsh, TCB Group, UIUC

The UIUC's Theoretical and Computational Biophysics Group (TCBG), an NIH Resource for Macromolecular Modeling and Bioinformatics is headed by Klaus Schulten, and CO-PIs Z. Luthey-Schulten,  E. Tajkhorshid, and A. Aksimentiev. As part of its outreach, the Resource offers workshops to introduce and transfer its programs and technological solutions to the biomedical community. The Resource produced a day-and-a-half workshop (March 16-17) on cluster building, held in Room 5602 of the Beckman Institute on the University of Illinois at Urbana-Champaign campus.  The program of the workshop consisted of lectures and hands-on sessions providing experience in building a cluster.  Instructors at the workshop were Jim Phillips, a Senior Research Programmer at the TCBG, and Tim Skirvin, Senior Systems Administrator at TCBG.

At the end of the second day of the workshop, participants were asked to complete a general evaluation survey.  The general evaluation form asks about topics such as outcomes of the program, ratings of attributes of lectures and tutorials, organization and communication, and so on; click here to see the form used.  Participation in the evaluation was voluntary.  A total of 22 evaluation forms were returned, providing an overall response rate of 100%.  Demographically, education levels of participants were mixed, with nearly equal groups of undergraduates, masters, doctoral students, and PhD recipients attending.

All responses in the following tables are reported in percentages, rows adding up to ~100% depending on rounding. Not all respondents answered to all items; the number of responses per item is presented next to each question. References to 'agreement' among respondents is calculated by adding together the percentages for the 'agree' and 'strongly agree' responses.

I. Outcome

  N Strongly disagree
%
Disagree
%
Unsure
%
Agree
%
Strongly agree
%
I1. The Workshop broadened my understanding of concepts and principles in designing and operating clusters. 22     9 18 73
I2. The Workshop improved my ability to set up a cluster. 22       32 68
I3. The Workshop improved significantly my computational skills. 21 4.8   33 48 14
I4. The Workshop taught me techniques directly applicable to my research and/or job/career. 21       50 50

Items I1-I4 refer to desirable outcomes for those attending the workshop.  A majority of participants indicated that the workshop broadened their understanding of cluster building concepts (91%), improved their ability to set up a cluster (100%), and taught them techniques directly applicable to their research and/or job/career (100%).  Well over half (62%) of respondents indicated the workshop significantly improved their computational skills.

II. Lectures

  N Strongly disagree
%
Disagree
%
Unsure
%
Agree
%
Strongly agree
%
II1. The instructors knowledge of the subjects was good. 22       9 91
II2. The instructors explained the material well. 22       23 77
II3. The instructors provided real-world examples. 22       18 82
II4. The instructors were prepared for the lectures. 22       14 86
II5. The lectures were coordinated between instructors. 22       18 82
II6. Lectures incorporated recent developments in the field. 22     9 27 64
II7. The range of lectures captured the overall essentials of the field. 22     5 36 59
II8. The level of the lectures was appropriate. 22       41 59
II9. The underlying rationale of the techniques presented was clear. 22       32 68
II10. The relevance of the lectures was strong. 22     9 27 64

Items II1-II10 evaluated lectures given by the instructors.  Ratings are quite high, with all participants (100%) agreeing or strongly agreeing that the instructors were knowledgeable, explained the material well, provided real-world examples, prepared for the lectures, and coordinated their lectures.  All participants (100%) also agreed that the level of the lectures was appropriate and that the rationale of the techniques presented was clear, and majorities agreed that the lectures captured the essentials of the field (95%) and that the relevance of the lectures was strong (91%).

Lecture Comments

Participants were also given the opportunity to provide written comments about the lectures on day one and day two of the workshop. Here are some of the comments provided:

Day One Lecture:

  • "Excellent."
  • "Really good."

Day Two Lecture:

  • "Wonderful! Thanks!"
  • "Great lecture, answered questions & integrated to hands-on"

A further comment on both days approved of the lectures, but suggested more details on the steps followed in the hands-on session be provided.

III. Hands-on Practice

  N Strongly disagree
%
Disagree
%
Unsure
%
Agree
%
Strongly agree
%
III1. The hands-on sessions were important for the learning process in the Workshop. 18       17 84
III2. The concrete examples in the hands-on tutorials increased my understanding of the lectures. 18       33 67
III3. The hands-on sessions were long enough. 18   6 6 28 61
III4. The hands-on sessions were coordinated with the lectures. 18     11 22 67
III5. There were sufficient instructions to proceed with the hands-on assignments. 18     11 33 56
III6. The supplied computers were adequate for the exercises. 18     6 22 72

Survey items III1-III6 asked about the hands-on practice provided in the workshop, specifically the building of clusters using the computers and other hardware and software provided by the workshop.  All participants (100%) felt the hands-on sessions were important to the goals of the workshop and that examples in the lectures increased understanding, and majorities (89-94%) indicated that the hands-on sessions were long enough, coordinated with the lectures, included sufficient instructions on how to proceed and that the supplied computers were adequate for the exercises.

IV. General Organization

  N Strongly disagree
%
Disagree
%
Unsure
%
Agree
%
Strongly agree
%
IV1. The number of participants was reasonable. 18    1189
IV2. There were enough staff to help the participants. 18   3367
IV3. There was sufficient information provided about the workshop via the website and e-mails. 18  62867
IV4. Instructors were readily available for Q&A outside the lecture periods. 18   1783
IV5. The Workshop web site was informative before the workshop started. 17 6 2965

The general organization items, questions IV1-IV5 of the survey, ask about the basic structure and operation of the workshop.  All participants (100%), indicate that the number of participants in the workshop was reasonable, that there was enough staff to help participants, and that the instructors were available for questions outside of the lecture periods.  A majority also indicate that there was sufficient information about the workshop via the website and e-mails (95%), and that the website was informative before the workshop started (94%).


V. Overall Satisfaction

  N Strongly disagree
%
Disagree
%
Unsure
%
Agree
%
Strongly agree
%
V1. Overall technical support was good. 18    2278
V2. Overall general support was good. 18   2872
V3. The Workshop was well organized. 18    2872
V4. The balance between lectures and hands-on sessions was optimal. 18   5050
V5. The Workshop addressed my research needs. 17   125335
V6. Overall, the Workshop met my expectations. 18   3961

A final set of scale questions, items V1-V6, rated overall satisfaction.  All participants (100%) indicated overall technical support was good, that overall general support was good, that the workshop was well organized, that the balance between lectures and hands-on sessions was optimal, and that the workshop met expectations.  Most participants (88%) indicated that the workshop addressed their research needs.

VI. General Comments

At the end of the survey, participants were asked to write responses to two open questions, "What suggestions do you have for improving the workshop?" and "What suggestions do you have for similar workshops?".  A summary of these comments is provided below, raw comments are available by writing workshop+cluster@ks.uiuc.edu.  Suggestions from participants include the following:

  • use a more robust operating system
  • cut-and-paste option for commands to reduce typos
  • more info at website
  • more hands-on time
  • have one cluster already operating for comparison
  • don't hold workshop when Illinois is playing