GPU Programming for Molecular Modeling Workshop - Urbana, Aug. 6-8, 2010

General Evaluation of the GPU Programming for Molecular Modeling Workshop held at the University of Illinois, Urbana

August 6-8, 2010

Questionnaire: Gila Budescu, TCB group, UIUC, and modified by David Brandon, TCB Group, UIUC
Analysis and report: David Brandon, TCB group, UIUC

The Theoretical and Computational Biophysics Group (TCBG), an NIH Resource for Macromolecular Modeling and Bioinformatics is headed by Klaus Schulten, and CO-PIs Z. Luthey-Schulten,  L. Kale, A. Aksimentiev, and E. Tajkhorshid.  As part of its outreach, the Resource offers workshops to introduce and transfer its programs and technological solutions to the biomedical community.  The Resource organized a 2.25 day (August 6-8, 2010) workshop held at the Beckman Institute for Advanced Science and Technology on the campus of the University of Illinois at Urbana-Champaign.

Workshop lectures were provided by TCBG development staff members John Stone and James Phillips, and postdoctoral associates David Hardy and Andrew Magis. Development staff member Kirby Vandivort assisted with providing consultations to workshop participants on their programming issues. The program of the workshop consisted of lectures, participant presentations, and a programming project laboratory. Participants worked on their own laptops during the workshop.

At the end of the workshop, participants were asked to complete a general evaluation questionnaire.  The general evaluation form asks about topics such as outcomes of the program, ratings of attributes of lectures and tutorials, organization and communication, and so on; click here to see the form used.  Participation in the evaluation was voluntary.  A total of 12 general evaluation forms were returned, providing an overall response rate of 86% (TCBG participants excluded).  All responses in the following tables are reported in percentages, rows adding up to 100%. Not all respondents answered to all items; the number of responses per item is presented next to each question.

In the summary below, participant 'agreement' with a statement is calculated by adding together the 'agree' and 'strongly agree' percentages from the tables below, e.g., in the first item of the first table below agreement is assessed at 100% (17% agree + 83% strongly agree). Due to rounding, some cumulative percentages in the table exceed 100%. The number of participants responding to each question is indicated by the 'N' count in each table.

I. Outcomes

All respondents (100%) agreed that the workshop broadened their understanding of the field of GPU programming for molecular modeling, and felt the material presented was relevant to their research/development goals.  Most respondents (75%) agreed that the workshop strengthened their research abilities in GPU programming, and taught them techniques applicable to their career.  A majority (67%) also found that the workshop significantly improved their computational skills.

  N Strongly Disagree Disagree Unsure Agree Strongly Agree
1. The workshop broadened my understanding of concepts and principles in the field of GPU programming for molecular modeling. 12 17% 83%
2. The workshop improved my ability to carry out original research in the field of GPU programming for molecular modeling 12 25% 25% 50%
3. The workshop improved significantly my computational skills. 12 8% 25% 50% 17%
4. The workshop taught me techniques directly applicable to my career. 12 25% 33% 42%
5. The material presented in the workshop was relevant to my research/development goals. 12 67% 33%











II. Lectures

All respondents (100%) rated the speakers' subject knowledge as good, agreed that the instructors explained the material well, found that instructors provided real-world examples, coordinated their lectures well, and that lectures incoporated recent developments in the field. A strong majority (92%) of participants agreed that lectures captured the essentials of the field, that the level of lectures was appropriate, that the rationale of the techniques presented was clear, and that instructors stimulated their intellectual curiosity.

  N Strongly Disagree Disagree Unsure Agree Strongly Agree
1. The instructors’ knowledge of the subjects was good. 12 100%
2. The instructors explained the material well. 12 25% 75%
3. The instructors provided real-world examples. 12 17% 83%
4. The lectures were coordinated between instructors. 12 17% 83%
5. Lectures incorporated recent developments in the field. 12 100%
6. The range of lectures captured the overall essentials of the field. 12 8% 58% 33%
7. The level of the lectures was appropriate. 12 8% 33% 58%
8. The underlying rationale of the techniques presented was clear. 12 8% 33% 58%
9. The instructors stimulated my intellectual curiosity. 12 8% 92%


III.  Participant Presentations, Group Discussion, Project Laboratory

All participants (100%) found that participant project presentations were a useful part of the workshop, that group discussion of projects was beneficial, that instructors were well-prepared to answer questions, and that there was enough instructional staff to help participants. A strong majority (92%) of participants felt they received valuable input on their projects from instructors. A majority of participants (82%) were uncertain if they programming project laboratory was beneficial. And, half (50%) of those participants responding felt that they received valuable input from other participants on their projects, with most of the rest (42%) unsure about this input.


  N Strongly Disagree Disagree Unsure Agree Strongly Agree
1. Participant project presentations were a useful part of the workshop. 12 33% 67%
2. The group discussion of participant projects was beneficial. 12 33% 67%
3. The programming project laboratory session (on Sunday) was beneficial. 11 82% 18% 0%
4. I received valuable input on my project from other workshop participants. 12 8% 42% 33% 17%
5. I received valuable input on my project from workshop instructors. 12 8% 50% 42%
6. Instructors were well-prepared to answer questions. 12 8% 92%
7. There was enough instructional staff to help the participants. 12 17% 83%


IV.  Environment and Technical Resources

All participants (100%) responding indicated that the internet connection supplied for the workshop was sufficient for workshop tasks, and that the hotel suggested for participants provided sufficient accommodations. Majorities indicated that the projections system was sufficient for the lectures (92%), and that the lecture room was conducive to learning (75%).


  N Strongly Disagree Disagree Unsure Agree Strongly Agree
1. The lecture room was conducive to learning. 12 25% 25% 50%
2. The projection system was sufficient for the lectures. 12 8% 25% 67%
3. The Internet connection provided was sufficient for workshop tasks. 12 8% 92%
4. The Hampton Inn provided sufficient accommodations (hotel guests only). 12 25% 75%

V.  Communication & Dissemination

All participants (100%) indicate that instructors were readily available for questions and answers outside of lecture sessions. Majorities found that the workshop web site was informative about the event (92%), and that emails about the workshop were helpful (83%).


  N Strongly Disagree Disagree Unsure Agree Strongly Agree
1. Instructors were readily available for Q&A outside the lecture periods. 11 100%
2. The workshop web site was informative about the event. 12 8% 50% 42%
3. Emails about the workshop were helpful. 12 8% 8% 25% 58%

VI.   Overall Satisfaction

All participants (100%) found that the workshop was well organized, that the workshop met their expectations, and that they would recommend the workshop to others. Majorities found that the balance between lectures and lab sessions was optimal (83%), and that the workshop addressed their research/development needs (75%). While many (41%) disagreed that the workshop had allowed them to make significant progress on a GPU version of their code, one-third (33%) were uncertain about this question, and one-fourth (25%) agreed that they had in fact made such progress.


  N Strongly Disagree Disagree Unsure Agree Strongly Agree
1. The workshop was well organized. 12 17% 83%
2. The balance between lectures and discussion/laboratory sessions was optimal. 12 17% 33% 50%
3. I was able to make significant progress on a GPU version of my code during the workshop. 12 8% 33% 33% 8% 17%
4. The workshop addressed my research/development needs. 12 25% 33% 42%
5. Overall, the workshop met my expectations. 12 42% 58%
6. I would recommend this workshop to others. 12 25% 75%

VII. Comments

Four open-ended questions were used to solicit input from participants: what would they do to improve the workshop;  what ideas do they have for similar workshops; what topics were most/least valuable, and what should be covered in future workshops; and, an 'other comments' section. The workshop recieved detailed and insightful responses from participants, for which we thank them. A number of comments are summarized below; a complete list of responses is available by emailing workshop+GPU_Aug2010@ks.uiuc.edu.

What suggestions do you have for improving the workshop?

  • 1) Provide all the lecture material before hand 2) The balance between lectures (morning) and student presentations and lab time (afternoon) was perfect for some already burnt out from one week of VSCSE workshop – don’t change that. If no VCSCE, then can add one lab assignment, depending on student GPU experience level!
  • Actually, I have only one suggestion. I would like to see more examples involving docking and quantum calculations.
  • The domain science problems were interesting. Perhaps a guest lecture from more people like Andrew, who directly apply GPUs to their problem.

What suggestions/ideas do you have for similar workshops?

  • Include a lecture or some time to utilizing the texture units as an option/approach towards using the GPU for modeling.
  • Workshop more geared toward development, with more devoted time to CUDA/kernel programming for the participants.
  • How about putting together a two part exercise where we could learn or implement a DCS method.

What topics were most valuable / least valuable to you? What topics do you think should be covered in future workshops?

  • I liked MPI the various concrete examples. I would like to see information on some of the bundled libraries/tools. E.g., FFT, BIAS, other APIs.
  • Valuable – hearing the thought process behind kernel design, especially the first lecture where the kernel was improved several times. This was a great example to see the considerations someone should keep in mind when programming for a GPU, the strength/weakness of each approach and to see performance tuning in action.
  • The snippets of code in the lectures was very useful. I would like to see less detail and more application.

Other Comments?

  • In some parts of lectures, a very brief rehash of material covered during the GPU workshop last week would have been helpful (ex. Binning). A list of supplemental material, texts, and future learning references would be great. Thank you all for a great workshop!
  • Lots of great stuff. Nitty gritty gems! Thanks guys!
  • Providing code snippets was useful. Please continue that.