Computational Biophysics Workshop - Urbana, Nov. 8-12, 2004
General Evaluation of the Theoretical and Computational Biophysics Workshop held at the University of Illinois, Urbana
November 8-12, 2004
Questionnaire: Gila Budescu, TCB
group, UIUC, and modified by David Brandon, TCB Group, UIUC
Analysis and report: Kathryn Walsh, UIUC, Brijeet Dhaliwal, TCB Group,
UIUC, and David
Brandon, TCB group, UIUC
The UIUC's Theoretical and Computational Biophysics Group (TCBG), an NIH Resource for Macromolecular Modeling and Bioinformatics is headed by Klaus Schulten, and CO-PIs Z. Luthey-Schulten, L. Kale, and R. Skeel.  As part of its outreach, the Resource offers workshops to introduce and transfer its programs and technological solutions to the biomedical community.  The Resource participated in a one-week (November 8-12) workshop sponsored by the Beckman Institute, and the National Center for Supercomputing Applications, at the University of Illinois Urbana, Illinois. The Beckman Institute provided facilities, internet access, and organization of the workshop, while the Resource provided instructors, lectures, tutorials, and 20 Apple G4 laptops loaded with needed software and tutorial files.
Workshop lectures were given by K. Schulten (UIUC), Z. Schulten (UIUC), and E. Tajkhorshid (UIUC).  Several Resource graduate students and one graduate student from the UIUC School of Chemical Sciences accompanied the lecturers to the workshop and provided instructional support on-site. Tutorials and preparation of the laptops was provided by the graduate students and by other Resource staff, and on-site the tutorial sessions were led by the graduate students. The program of the workshop consisted of lectures and hands-on sessions, with an emphasis on the latter.
At the end of the week, students leaving the
program were asked to complete a general evaluation questionnaire.  The
general evaluation form asks about topics such as outcomes of the
program, ratings of attributes of lectures and tutorials, organization
and communication, and so on; click here
to see the form used. Participation in the evaluation was
voluntary. A total of 17 general evaluation forms were returned,
providing an overall response rate of 81%. Demographically,
education levels of participants were high; the majority of attendents
were PhD candidates, three were doctorates, two were undergraduates,
and one was a non-professional academic.
The results of the general evaluation questionnaire are summarized below, within the following sections:
- I. Outcomes
- II. Lectures
- III. Hands-on tutorials
- IV. Environment & Technical Resources
- V. Communication & Dissemination
- VI. General Organization
- VII. Satisfaction
- VIII. Comments
All responses in the following tables are reported in percentages, rows adding up to 100%. Not all respondents answered to all items; the number of responses per item is presented next to each question.
Table I. Distribution of Outcome Ratings
N |
Strongly Disagree |
Disagree |
Unsure |
Agree |
Strongly Agree |
|
1.The workshop broadened my understanding of concepts and principles in the field of Computational and Theoretical Biophysics. |
17 |
|
5.9 |
17.6 |
17.6 |
58.8 |
2.The workshop improved my ability to carry out original research in the field of Computational and Theoretical Biophysics. |
15 |
|
|
13.3 |
20.0 |
66.7 |
3.The workshop improved significantly my computational skills. |
17 |
|
11.8 |
23.5 |
35.3 |
29.4 |
4.The workshop taught me techniques directly applicable to my career. |
17 |
|
5.9 |
|
29.4 |
64.7 |
5.The material presented in the workshop was relevant to my research. |
17 |
|
5.9 |
11.8 |
17.6 |
64.7 |
Items I.1-5 refer to desired outcomes. Most respondents agreed that the workshop broadened their understanding of the field (76.4%). Most respondents agreed that the workshop strengthened their research abilities (86.7%). The majority of respondents agreed that the school significantly improved their computational skills (64.7%), that they acquired useful techniques (94.1%) and that the material was relevant to their own research (82.3%).
Table II. Distribution of Lecture Ratings
|
N |
Strongly Disagree |
Disagree |
Unsure |
Agree |
Strongly Agree |
1.The instructor's knowledge of the subjects was good. |
17 |
|
|
|
23.5 |
76.5 |
2.The instructors explained the material well. |
17 |
|
|
5.9
|
52.9 |
41.2 |
3.The instructors provided real-world examples. |
17 |
|
|
|
23.5 |
76.5 |
4.The instructors were prepared for the lectures. |
17 |
|
5.9 |
|
41.2 |
52.9 |
5.The lectures were coordinated between instructors. |
17 |
|
|
11.8 |
35.3 |
52.9 |
6.Lectures incorporated recent developments in the field. |
17 |
|
|
|
17.6 |
82.4 |
7.The range of lectures captured the overall essentials of the field. |
17 |
|
|
23.5 |
29.4 |
47.1 |
8.The level of the lectures was appropriate. |
17 |
|
5.9 |
11.8 |
41.2 |
41.2 |
9.The underlying rationale of the techniques presented was clear. |
16 |
|
6.3 |
|
37.5 |
56.3 |
10.We were exposed to a well representative range of techniques. |
17 |
|
5.9 |
|
29.4 |
64.7 |
11.The instructors stimulated my intellectual curiosity. |
17 |
|
5.9 |
|
11.8 |
82.4 |
Items II.1-11 address the level, scope, and quality of the lectures. All respondents rated the speakers' knowledge good(100%), and also agreed that the instructors explained the material well (94.1%), provided real-world examples (100%), were prepared for lectures (94.1%), and coordinated the lectures with other instructors(88.2%). There was general agreement that the lectures incorporated recent developments(100%), and captured the field essentials (76.5%). Most respondents found the level of the lectures to be appropriate (82.4%). In addition, most respondents felt that the underlying rationale of the techniques were presented clearly (93.8%), and that the range of techniques presented was representative of the field (94.1%). The majority of respondents agreed that the instructors stimulated their intellectual curiosity (94.2%).
Table III. Distribution of Research Tutorial Ratings
|
N |
Strongly Disagree |
Disagree |
Unsure |
Agree |
Strongly Agree |
1.The hands-on sessions were important for the learning process in the workshop. |
17 |
|
|
|
17.6 |
82.4 |
2.The concrete examples in the hands-on tutorials increased my understanding of the lectures. |
17 |
|
|
5.9 |
17.6 |
76.5 |
3.The hands-on sessions were long enough. |
17 |
|
|
47.1 |
17.6 |
35.3 |
4.The hands-on sessions were coordinated with the lectures. |
16 |
|
|
6.3 |
31.3 |
62.5 |
5.TAs were well-prepared to answer questions. |
16 |
|
6.3 |
12.5 |
81.3 |
|
6.There were sufficient instructions to proceed with the hands-on assignments. |
16 |
|
6.3 |
|
6.3 |
87.5 |
Items III.1-6 deal with the level, quality, and scope of the hands-on tutorials. All respondents agreed that the hands-on sessions were important for the learning process during the workshop (100%). Most concurred that the concrete examples in the tutorials increased understanding of the lectures (94.1%), hands-on sessions were coordinated with lectures (93.8%), the TAs were well-prepared to answer questions (93.8%), and there were sufficient instructions to proceed with hands-on assignments (93.8%). About half of the respondents agreed that the sessions were long enough (52.9%).
Table IV. Distribution of Environment & Technical Resources Ratings
|
N |
Strongly Disagree |
Disagree |
Unsure |
Agree |
Strongly Agree |
1.The Apple Powerbook G4s were adequate for the exercises. |
17 |
|
|
|
23.5 |
76.5 |
2.The Apple Powerbook G4s ran smoothly. |
17 |
|
5.9 |
|
35.3 |
58.8 |
3.It was easy to learn how to use the Apple Powerbook G4s. |
17 |
|
|
17.6 |
29.4 |
52.9 |
4.The software used in the workshop ran well on the Apple Powerbook G4s. |
17 |
|
|
35.3 |
64.7 |
|
5.The lecture room was conducive to learning. |
17 |
|
|
5.9 |
17.6 |
76.5 |
6.Workshop access to the Internet was sufficient. |
17 |
|
|
5.9 |
23.5 |
70.6 |
7.The projection system was sufficient for the lectures. |
17 |
|
|
|
35.3 |
64.7 |
8.The Hampton Inn provided sufficient
accommodations. |
10 |
|
|
|
30.0 |
70.0 |
Items IV.1-8 address the
effectiveness of the physical environment and technical support during
the workshop. Most of these items refers to the Apple Powerbook
G4 laptop computers set up for the workshop. Most respondents
agreed that the laptops were adequate for the exercises (100%),
ran
smoothly (94.1%), and were easy to learn how to use (82.3%). All
respondents
agreed that the software used in the ran well on the
laptops (100%). Many agreed that the lecture room was conducive
to
learning(94.1%), access to the Internet was sifficient(94.1%), and the
projection system was sufficient for the lectures (100%).
Table V. Distribution of Communication & Dissemination Ratings
N |
Strongly Disagree |
Disagree |
Unsure |
Agree |
Strongly Agree |
|
1.Instructors were readily available for Q&A outside the lecture periods. |
17 |
|
|
5.9 |
23.5 |
70.6 |
2.The daily noon Q&A period was beneficial. |
17 |
|
|
5.9 |
29.4 |
64.7 |
3.The workshop web site was informative before the school period. |
17 |
|
5.9 |
5.9 |
23.5 |
64.7 |
4.The workshop web site was informative during the school period. |
16 |
|
6.3 |
12.5 |
31.3 |
50.0 |
5.The online information was up-to-date. |
16 |
|
6.3 |
|
50.0 |
43.8 |
6.The online material was organized. |
16 |
|
|
6.3 |
31.3 |
62.5 |
7.There was sufficient information about evening activities during the school. |
16 |
|
|
|
31.3 |
68.8 |
Items V.1-7 were designed to
assess the effectiveness of the workshop communication and
dissemination efforts and tools. Most respondents agreed that the
instructors were available outside the lecture periods (94.1%), and
agreed that the daily Q&A period was
beneficial (94.1%). The respondents were satisfied that the
workshop website was informative before (88.2%) and during (81.3%) the
workshop, and that the online information was
up-to-date (93.8%). Most agreed that the online material was well
organized (93.8%). All agreed that there was sufficient
information about evening
activities during the workshop (100%).
Table VI. Distribution of General Organization Ratings
|
N |
Strongly Disagree |
Disagree |
Unsure |
Agree |
Strongly Agree |
1.The number of participants was reasonable. |
17 |
|
|
|
29.4 |
70.6 |
2.There were enough instructors to help the participants. |
17 |
5.9 |
|
|
17.6 |
76.5 |
3.There were enough TAs and support staff to help the participants. |
17 |
|
|
|
17.6 |
82.4 |
4.The evening social activities enhanced the Workshop experience. |
17 |
|
|
11.8 |
23.5 |
64.7 |
Items VI.1-4 evaluate the
general organization of the school. All respondents agreed that
the numbers of participants was reasonable (100%), and the number of
TAs
and
support staff was sufficient (100%). Most agreed there were enough
instructors
to help the school participants (94.1%), and that the evening social
activities enhanced the Workshop (88.2%).
Table VII. Distribution of Satisfaction Ratings
|
N |
Strongly Disagree |
Disagree |
Unsure |
Agree |
Strongly Agree |
1.Overall technical support was good. |
17 |
|
|
|
11.8 |
88.2 |
2.Overall general support was good. |
17 |
|
|
|
23.5 |
76.5 |
3.The workshop was well organized. |
17 |
|
|
11.8 |
88.2 |
|
4.The balance between lectures and hands-on sessions was optimal. |
17 |
|
|
5.9 |
41.2 |
52.9 |
5.The workshop addressed my research needs. |
17 |
|
5.9 |
5.9 |
17.6 |
70.6 |
6.Overall, the workshop met my expectations. |
17 |
|
5.9 |
|
17.6 |
76.5 |
Items VII.1-6 were intended to measure the
attendees satisfaction
across the various workshop components. The results show that the
participants were mostly satisfied with the school. All respondents
agreed
that technical (100%) and general (100%) support was good, not to
mention
the
workshop was well organized(100%). There was also an optimal
balance of
lectures and hands-on sessions (94.1%). A majority (88.2%), indicated
the school addressed their research needs, and a majority also felt
that the workshop met their expectations at (94.1%).
VII. Comments
12 of the 17 respondents took the time to answer two open-ended items, 1) "What suggestions do you have for improving the workshop", and b) "What suggestions do you have for similar workshops?". Their comments, in their own words, can be requested from the organizers by emailing brandon@ks.uiuc.edu.
Specific comments from the daily lectures and tutorials, are available here.
While hard to reliably quantify and less than
systematic, open-ended
comments can provide insight into some important matters that may be
improved and others, not covered by the questionnaire and beyond the
control of the organizers. Generally, the comments touch on
participants
wanting more and/or earlier information about the summer school,
wanting
more or less information on particular topics, and compliments about
the
workshop. Overall, the many narrative responses do indicate that
the
participants cared enough about the school to take the time to write
them
down, which reflects a certain level of commitment and satisfaction on
their part, regardless whether the comments are positive or negative.
They
also shed more light on some participant expectations and provide new
ideas to consider for future training efforts.