Computational Biophysics Workshop - Bethesda, Nov. 5-7, 2007

General Evaluation of the Computational Biophysics Workshop at Bethesda, Maryland

November 5-7, 2007

Questionnaire: Gila Budescu, TCB Group, UIUC, and modified by David Brandon, TCB Group, UIUC
Analysis and report: David Brandon, TCB Group, UIUC

The NIH Resource for Macromolecular Modeling and Bioinformatics is headed by Klaus Schulten, and Co-PIs Z. Luthey-Schulten, L. Kale, E. Tajkhorshid, and A. Aksimentiev. As part of its outreach, the Resource offers workshops to introduce and transfer its programs and technological solutions to the biomedical community. The Resource presented a three-day (November 5-7, 2007) workshop sponsored by Helix Systems, Center for Information Technology at the Bethesda, Maryland campus of the National Institutes of Health. Resource faculty provided morning lectures, and teaching assistants and faculty led afternoon hands-on sessions using Resource software, and tutorials consisting of text and computer files. Helix Systems organized the event on the Bethesda campus, including advertising and registration of participants, ongoing contact with participants, setting up the event room, formatting of laptops, tutorial printing, and overall support of the workshop.

Workshop lectures were provided by K. Schulten (UIUC) and E. Tajkhorshid (UIUC). Three Resource members (M. Sotomayor, J. Gumbart, and J. Hsin) accompanied the lecturers to the workshop and provided instructional support on-site. A special thanks to Dr. Rosemary Braun, a postdoctoral fellow with the NIH's National Cancer Institute (Division of Cancer Prevention) for help in testing the laptops provided for workshop attendees, and for assisting with tutorial sessions during the first two days of the workshop. Tutorials and preparation of the laptops was provided by the graduate students and by other Resource staff, and on-site the tutorial sessions were led by the graduate students.  The program of the workshop consisted of lectures and hands-on sessions, with an emphasis on the latter. On the last day of the workshop, participants were asked to complete a general evaluation form consisting of questions about the lectures, hands-on tutorial sessions, outcomes, and overall satisfaction. A copy of the form is available here, with results described below.

In the summary below, participant 'agreement' with a statement is calculated by adding together the 'agree' and 'strongly agree' percentages from the tables below, e.g., in the first item of the first table below agreement is assessed at 93% (40% agree + 53% strongly agree). Due to rounding, some cumulative percentages in the table exceed 100%. The number of participants responding to each question is indicated by the 'N' count in each table. Attendance varied during the workshop, with some attending lectures but not tutorials, and with some days more popular than others. At a maximum the workshop drew 30 participants, hence we will place the response rate for the general evaluation form at about 50%.

I. Outcomes

With regards to outcomes, all participants (100%) indicate that the workshop was relevant to their research, and a high majority (93% for each question) of participants also indicate that the workshop broadened their understanding of concepts and principles in computational and theoretical biophysics, their ability to carry out research in that field, and that the workshop taught them techniques directly applicable to their careers. Fewer participants (80%) indicated that the workshop significantly improved their computational skills.

 

N

 Strongly disagree Disagree Unsure Agree Strongly agree
1. The Workshop broadened my understanding of concepts and principles in the field of Computational and Theoretical Biophysics. 15     7% 40% 53%
2. The Workshop improved my ability to carry out original research in the field of Theoretical and Computational Biophysics. 15     7% 40% 53%
3. The Workshop improved significantly my computational skills 15 7%   13% 67% 13%
4. The Workshop taught me techniques directly applicable to my career. 15     7% 27% 67%
5. The material presented in the Workshop was relevant to my research. 15       40 % 60 %

II. Lectures

Participant ratings of lectures were very high for nearly all items, with all (100%) indicating that instructors had good subject knowledge, explained material well, provided real-world examples, coordinated their lectures with each other, incorporated recent developments in the field in their lectures, stimulated participants intellectual curiosity, and that instructors lectures were at the level of the workshop audience. Strong but slightly less support (93%) from participants was found for assessments of whether lectures captured the overall essentials of the field, and that the underlying rationale of the techniques presented was clear. Over three-fourths (79%) of participants thought that the daily question and answer period was beneficial to the workshop.

 

N

 Strongly disagree Disagree Unsure Agree Strongly agree
1. The instructors' knowledge of the subjects was good. 15         100%
2. The instructors explained the material well. 15       13% 87%
3. The instructors provided real-world examples. 15       13% 87%
4. The lectures were coordinated between instructors. 15       20% 80%
5. Lectures incorporated recent developments in the field. 15       13% 87%
6. The range of lectures captured the overall essentials of the field. 15     7% 13% 80%
7. The level of the lectures was appropriate. 15       27% 73%
8. The underlying rationale of the techniques presented was clear. 15     7% 33% 60%
9. The instructors stimulated my intellectual curiosity. 15       7% 93%
10. The daily noon Q&A period was beneficial. 14   7% 14% 36% 43%

III. Hands-on Sessions

The hands-on sessions refers to the afternoon tutorial sessions, during which participants completed work on their own or provided laptops. All participants agree that the hands-on sessions were important for the learning process of the workshop, that teaching assistants were well-prepared to answer questions, that there were enough teaching assistants and instructional staff to help participants, and that the software used in the workshop ran well on their own or the laboratory laptops. A strong majority (93%) also indicate examples in the hands-on sessions  increased understanding of lectures, that hands-on sessions were coordinated with lectures, and that there were sufficient instructions to proceed with tutorial assignments. While a small majority (67%) thought that the hands on sessions were long enough, a substantial portion (34%) did not indicate agreement with this question stem, suggesting that more time for hands-on sessions would have benefited some participants.

 

N

 Strongly disagree Disagree Unsure Agree Strongly agree
1. The hands-on sessions were important for the learning process in the Workshop. 15       13% 87%
2. The concrete examples in the hands-on tutorials increased my understanding of the lectures. 15     7% 13% 80%
3. The hands-on sessions were long enough. 15 7% 7% 20% 27% 40%
4. The hands-on sessions were coordinated with the lectures. 15     7% 33% 60%
5. TAs were well-prepared to answer questions. 15       7% 93%
6. There were sufficient instructions to proceed with the hands-on assignments. 15     7% 20% 73%
7. There were enough TAs / instructional staff to help the participants. 15       13% 87%
8. The software used in the Workshop ran well on (circle one) my laptop / the laboratory laptop. 15       7% 93%

IV. Overall Satisfaction

A final set of questions asked participants about their overall satisfaction with the workshop. All participants (100%) agree that the workshop met their expectations, and that the event was well organized. Majorities also indicated that the workshop addressed participant research needs (93%) and that the balance between lectures and hands-on sessions was optimal (86%).

 

N

 Strongly disagree Disagree Unsure Agree Strongly agree
1. The Workshop was well organized. 15       33% 67%
2. The balance between lectures and hands-on sessions was optimal. 15     13% 53% 33%
3. The Workshop addressed my research needs. 15     7% 47% 47%
4. Overall, the Workshop met my expectations. 15       13% 87%

V. Comments

Participants were solicited for comments in three areas, 1) suggestions for improving the workshop, 2) suggestions for similar workshops, and 3) other comments. Each area of comments in summarized below; a compilation of raw statements can be obtained by e-mailing workshop+bethesda@ks.uiuc.edu. If a comment was made in one area, it is not repeated again in another area below.

  • Suggestions for improving the workshop:
    • longer time frame - extend to more days, particularly the hands-on sessions needed more time
    • demonstrate use of VMD plugins and provide more information on this topic
    • allow participants to work through tutorials on their own in a coordinated fashion, then at workshop work together to solve common problems
    • less information on psf and pdb files, which most participants probably already know
  • Suggestions for similar workshops:
    • NAMD with AMBER force-field
    • (at the CIT site) NAMD on the BioWulf cluster
    • how to judge NAMD results - which parameters are crucial for the right simulation results
    • how NAMD compares with other software
    • larger room with more/better internet access to use for downloading needed files
  • Other comments:
    • tutorials could be completed independently, allowing more time for lectures
    • a follow-up on membrane proteins specifically