Evaluation of the Theoretical and Computational Biophysics Online Workshop

February 22-27, 2007

Questionnaire: Gila Budescu, TCB Group, UIUC, and modified by David Brandon, TCB Group, UIUC
Analysis and report: David Brandon, TCB Group, UIUC

The UIUC's Theoretical and Computational Biophysics Group (TCBG), an NIH Resource for Macromolecular Modeling and Bioinformatics is headed by Klaus Schulten, and Co-PIs Z. Luthey-Schulten and L. Kale. As part of its outreach, the Resource offers workshops to introduce and transfer its programs and technological solutions to the biomedical community. The Resource organized a five-day (February 22-27) online workshop.

The program of the workshop consisted of lectures and hands-on sessions, with an emphasis on the latter. The workshop lecture was provided via a recorded streaming lecture of E. Tajkhorshid (UIUC). A graduate student provided instructional support for the tutorial via e-mail. Contact with the lecturer was provided via two scheduled conference call sessions.

The day after the completion of the workshop, participants were asked to complete an evaluation questionnaire. The evaluation form asks about topics such as outcomes of the workshop, perceptions of lectures and tutorials, organization and communication, and so on; click here to see the form used.  Participation in the evaluation was voluntary.  A total of 21 evaluation forms were returned, out of 26 participants, providing an overall response rate of 80%.  Demographically, education levels of participants were high, with a majority of participants indicating having obtained a PhD or participation in a doctoral program. Responses are detailed below by sections of the evaluation questionnaire: participation, outcomes, lecture, conference call, tutorial, organization, and satisfaction.

All responses in the following tables are reported in percentages, rows adding up to 100% (with the exception of the first table on participation). Not all respondents answered to all items; the number of responses per item is presented next to each question. References to 'agreement' among respondents is calculated by adding together the percentages for the 'agree' and 'strongly agree' responses.

I. Participation

  N I viewed the streaming lecture % I participated in the conference call % I worked through (even partially) the tutorial % None %
Which parts of the workshop did you utilize? Check all that apply. 21 95 33 71 0

The most utilized instructional feature of the workshop was the streaming lecture, with 95% of respondents indicating they viewed the lecture. A majority of respondents, 71%, also worked through the tutorial. A smaller number of respondents, 33%, indicate they participated in one of  the conference call sessions.


II. Outcomes

  N Strongly Disagree % Disagree % Unsure % Agree % Strongly Agree %
II.1. The workshop broadened my understanding of concepts and principles in the field of Computational and Theoretical Biophysics. 21   57 43
II.2. The workshop improved my ability to carry out original research in the field of Theoretical and Computational Biophysics. 21   19 48 33
II.3. The workshop improved significantly my computational skills. 21   10 33 43 14
II.4. The workshop taught me techniques directly applicable to my career. 21   14 57 29
II.5. The material presented in the workshop was relevant to my research. 21   5 62 33
 

Regarding outcomes associated with participation in the workshop, all respondents indicate that the workshop broadened their understanding of computational and theoretical biophysics. Majorities also indicate that the material presented was relevant to their research at 95%, applicable to their career at 86%, and that the workshop improved their ability to research at 81%. Just over half, 57%, indicate that the workshop significantly improved their computational skills.


III. Lecture

  N Strongly Disagree % Disagree % Unsure % Agree % Strongly Agree %
III.1. The lecture captured the overall essentials of the topic. 21 5 43 52
III.2. The level of the lecture was appropriate. 21 48 52
III.3. The lecture provided real-world examples. 21 5 52 43
III.4. The lecture incorporated recent development in the field. 20 5 50 45
III.5. The underlying rationale of the techniques presented was clear. 20 5 10 50 35
III.6. The lecture stimulated my intellectual curiosity. 20 5 50 45
III.7. The lecturer's knowledge of the subject was good. 21 43 57
III.8. The lecturer explained the material well. 20 45 55
III.9. The video quality of the the lecture was good. 21 14 5 43 38
III.10. The audio quality of the lecture was good. 21 5 5 48 43
III.11. The powerpoint slides were a useful accompaniment to the lecture. 20 5 5 30 60

A streaming video was used to deliver the lecture used in the workshop. All respondents agreed that the level of the lecture was appropriate, that the lecturer's knowledge of the subject was good, and that the lecturer explained the material well. At 95%, nearly all respondents agreed that the lecture captured the overall essentials of the topic, that the lecture provided real-world examples, that the lecture incorporated recent developments in the field, and that the lecture stimulated their intellectual curiosity. Slightly smaller majorities, in the 81-91 percent range, agreed that the underlying rationale of the techniques in the lecture was clear, that the video and audio quality of the lecture was good, and that the PowerPoint slides (same as those presented in the lecture) were a useful accompaniment to the lecture itself. Respondents were asked to skip these questions if they did not view the lecture.


IV. Conference Call

  N Strongly Disagree % Disagree % Unsure % Agree % Strongly Agree %
IV.1. The conference call was a valuable part of the workshop. 14     14 36 50
IV.2. The conference call enhanced my understanding of other course material. 14     21 43 36
IV.3. The number of participants involved in the conference call was practical. 14     21 21 57

Most participants responding indicated that the conference call was a valuable part of the workshop at 86%, that the call enhanced their understanding of other course material at 79%, and that the number of participants involved in the conference call wasn't impractical at 78%. While respondents were allowed to skip this question if they did not participate in the call, some participants who didn't call may have answered the questions as 'unsure' or gave their general opinion about the perceived value of a conference call to the workshop. Participants were asked to skip the questions about the conference call if they didn't participate in one of the conference call sessions.


V. Tutorial

  N Strongly Disagree % Disagree % Unsure % Agree % Strongly Agree %
V.1. The tutorial important for the learning process in the workshop. 19 5 37 58
V.2. The concrete examples in the tutorial increased my understanding of other workshop content. 18 11 50 39
V.3. There were sufficient instructions to proceed with the tutorial. 15 47 53
V.4. The Teaching Assistant was well-prepared to answer questions (skip if you had no contact with the TA). 10 10 20 70

All participants responding agreed that the tutorial provided sufficient instructions. A majority of respondents also indicate that the tutorial was important for the learning process in the workshop at 95%, that the Teaching Assistant was well-prepared to answer questions at 90%, and that the concrete examples in the tutorial enhanced understanding of the workshop content. Respondents were asked to skip the tutorial questions if they didn't work on the tutorial itself.


VI. Organization

  N Strongly Disagree % Disagree % Unsure % Agree % Strongly Agree %
VI.1. The workshop web site was informative. 21 14 38 48
VI.2. Emails about the workshop were informative. 21 5 5 52 38
VI.3. The workshop application and registration process was reasonable. 21 48 52

All respondents found the workshop application and registration process reasonable, and majorities found that the workshop e-mails were informative at 90%, and that the workshop web site was informative at 86%.


VII. The following questions concern your overall satisfaction with the workshop.

  N Strongly Disagree % Disagree % Unsure % Agree % Strongly Agree %
VII.1. The workshop addressed my research needs. 21 10 62 29
VII.2. I would participate in other online workshops like this one. 21 5 38 57
VII.3. Overall, the workshop met my expectations. 21 10 10 33 48

A majority of responding participants indicate that the workshop addressed their research needs at 91%, and that the workshop met their expectations at 81%. Further, 95% of respondents indicated that they would participate in similar future workshops.

VIII. Comments

Participants provided responses to two open questions, "What suggestions do you have for improving this workshop?" and "What suggestions do you have for similar workshops?". Responses from both questions are compiled below; comments include:
  • Cut up the streaming video into segments matching the different sections of the PowerPoint slides provided or into other logical pieces
  • Provided a downloadable version of the streaming lecture for those with problems streaming the video
  • Email sessions to answer questions in addition to the conference call, post answers to emailed questions, allow for participant e-mail exchanges
  • Provide information about other participants
  • Add more content - on biological membrane systems, more on the technical aspects of VMD and NAMD, add more content in general to fill the 5-day workshop period

 

Funded by a grant from
the National Institute of
General Medical Sciences
of the National Institutes
of Health