GPU Programming for Molecular Modeling Workshop - Urbana, Aug. 2-4, 2013

General Evaluation of the GPU Programming for Molecular Modeling Workshop held at the University of Illinois, Urbana

August 2-4, 2013

Questionnaire: Gila Budescu, TCB group, UIUC, and modified by David Brandon, TCB Group, UIUC
Analysis and report: David Brandon, TCB group, UIUC

The Theoretical and Computational Biophysics Group (TCBG), an NIH Center for Macromolecular Modeling and Bioinformatics is headed by Klaus Schulten, and CO-PIs Z. Luthey-Schulten,  L. Kale, A. Aksimentiev, and E. Tajkhorshid. As part of its outreach, the Center offers workshops to introduce and transfer its programs and technological solutions to the biomedical community. The Center organized a 2.25 day (August 2-4, 2013) workshop held at the Beckman Institute for Advanced Science and Technology on the campus of the University of Illinois at Urbana-Champaign.

TCBG development staff members John Stone, James Phillips, and David Hardy provided lectures, along with Fernanda Foertter, from the Oak Ridge Leadership Computing Facility, and Ivan Ufimtsev, from Stanford University. The program of the workshop consisted of lectures, participant presentations, and a programming project laboratory. Participants worked on their own laptops during the workshop, but also accessed supercomputing resources.

At the end of the workshop, participants were asked to complete a general evaluation questionnaire. The general evaluation form asks about topics such as outcomes of the program, ratings of attributes of lectures and tutorials, organization and communication, and so on; click here to see the form used.  Participation in the evaluation was voluntary.  A total of 11 usable general evaluation forms were returned, providing an overall response rate of 100%.  All responses in the following tables are reported in percentages, rows adding up to 100%. Not all respondents answered to all items; the number of responses per item is presented next to each question.

In the summary below, participant 'agreement' with a statement is calculated by adding together the 'agree' and 'strongly agree' percentages from the tables below, e.g., in the first item of the first table below agreement is assessed at 100% (17% agree + 83% strongly agree). Due to rounding, some cumulative percentages in the table exceed 100%. The number of participants responding to each question is indicated by the 'N' count in each table.

I. Outcomes

All respondents (100%) agreed that the workshop broadened their understanding of the field of GPU programming for molecular modeling. A high majority (91%) indicated the workshop improved their ability to do original research using GPU programming, that the workshop taught techniques applicable to their career, and felt the material presented was relevant to their research/development goals. Most respondents (73%) found that the workshop significantly improved their computational skills.

  N Strongly Disagree Disagree Unsure Agree Strongly Agree
1. The workshop broadened my understanding of concepts and principles in the field of GPU programming for molecular modeling. 11       18% 82%
2. The workshop improved my ability to carry out original research in the field of GPU programming for molecular modeling 11     9% 36% 55%
3. The workshop improved significantly my computational skills. 11     27% 55% 18%
4. The workshop taught me techniques directly applicable to my career. 11     9% 27% 64%
5. The material presented in the workshop was relevant to my research/development goals. 11     9% 9% 82%

II. Lectures

All respondents (100%) rated the speakers' subject knowledge as good, agreed that the instructors explained the material well, found that instructors provided real-world examples, that lectures incoporated recent developments in the field,that the level of lectures was appropriate, that the rationale of the techniques presented was clear, and that instructors stimulated their intellectual curiosity. A high majority (91%) indicated that instructors coordinated their lectures well, and that lectures captured the essentials of the field.

  N Strongly Disagree Disagree Unsure Agree Strongly Agree
1. The instructors’ knowledge of the subjects was good. 11       9% 91%
2. The instructors explained the material well. 11       27% 73%
3. The instructors provided real-world examples. 11       18% 82%
4. The lectures were coordinated between instructors. 11   9%   27% 64%
5. Lectures incorporated recent developments in the field. 11       36% 64%
6. The range of lectures captured the overall essentials of the field. 11     9% 27% 64%
7. The level of the lectures was appropriate. 11       55% 45%
8. The underlying rationale of the techniques presented was clear. 11       36% 64%
9. The instructors stimulated my intellectual curiosity. 11       9% 91%

III.  Participant Presentations, Group Discussion, Project Laboratory

All participants (100%) found that that instructors were well-prepared to answer questions and that there were enough instructional staff to help participants. A high majority (90%-91%) of participants indicated that participant project presentations were a useful part of the workshop, that group discussion of projects was beneficial, and felt they received valuable input on their projects from instructors. Lower majorities of participants reported that programming project laboratory was beneficial (70%) and that they received valuable input from other participants on their projects (60%).

  N Strongly Disagree Disagree Unsure Agree Strongly Agree
1.     Participant project presentations were a useful part of the workshop. 11   9%   55% 36%
2.     The group discussion of participant projects was beneficial. 11   9%   9% 82%
3.     The programming project laboratory session (on Sunday) was beneficial. 10   10% 20% 40% 30%
4.     I received valuable input on my project from other workshop participants. 10 10% 10% 20% 50% 10%
5.     I received valuable input on my project from workshop instructors. 10     10% 60% 30%
6.     Instructors were well-prepared to answer questions. 11       27% 73%
7.     There was enough instructional staff to help the participants. 10       20% 80%

IV.  Environment and Technical Resources

All participants (100%) responding indicated that that the projections system was sufficient for the lectures. A high majority (90%-91%) reported that the internet connection supplied for the workshop was sufficient for workshop tasks, and that the lecture room was conducive to learning.

  N Strongly Disagree Disagree Unsure Agree Strongly Agree
1. The lecture room was conducive to learning. 11     9% 18% 73%
2. The projection system was sufficient for the lectures. 11       18% 82%
3. The Internet connection provided was sufficient for workshop tasks. 10     10% 10% 80%

V.  Communication & Dissemination

All participants (100%) indicate that instructors were readily available for questions and answers outside of lecture sessions. Majorities (82%) found that the workshop web site was informative about the event and that emails about the workshop were helpful.

  N Strongly Disagree Disagree Unsure Agree Strongly Agree
1. Instructors were readily available for Q&A outside the lecture periods. 11         100%
2. The workshop web site was informative about the event. 11     18% 18% 64%
3. Emails about the workshop were helpful. 11   9% 9% 18% 64%

VI.   Overall Satisfaction

All participants (100%) found that the workshop was well organized, that the balance between lectures and lab sessions was optimal, that the workshop met their expectations, and that they would recommend the workshop to others. A high majority (91%) of participants found that the workshop addressed their research/development needs, while half of participants (50%) indicated they were able to make significant progress on a GPU version of my code during the workshop.

  N Strongly Disagree Disagree Unsure Agree Strongly Agree
1. The workshop was well organized. 11       18% 82%
2. The balance between lectures and discussion/laboratory sessions was optimal. 10       40% 60%
3. I was able to make significant progress on a GPU version of my code during the workshop. 10 10% 10% 30% 30% 20%
4. The workshop addressed my research/development needs. 11       36% 55%
5. Overall, the workshop met my expectations. 11       18% 82%
6. I would recommend this workshop to others. 11       9% 91%

 

VII. Comments

Four open-ended questions were used to solicit input from participants: what would they do to improve the workshop;  what ideas do they have for similar workshops; what topics were most/least valuable, and what should be covered in future workshops; and, an 'other comments' section. The workshop recieved detailed and insightful responses from participants, for which we thank them. A number of comments are summarized below; a complete list of responses is available by emailing workshop+gpuaug2013@ks.uiuc.edu.

What suggestions do you have for improving the workshop?

  • More instructors’ lectures. Discussion about GPU codes and tricks in detail.
  • The workshop is great. It will be better if it can show more detail example and practice.
  • Overall, I was very impressed with the workshop. Organization was quite well and the format (informal) worked great for encouraging one-on-one discussion. Thanks!

What suggestions/ideas do you have for similar workshops?

  • Workshop to analyze common chunks of code between various simulation sources in a single publicly available API -> or trade tips on optimizing respective element if not feasible (sort of like this workshop, but less tutorial, more implementation lab).
  • More group integration, with task to perform and team competitions with rewards. Do a follow-up of the problems intendant to do by the participants. What were the things that they applied the most or needed the most.
  • None at the moment

What topics were most valuable / least valuable to you? What topics do you think should be covered in future workshops?

  • Most enjoyed code optimization technique discussion.
  • All are good, to see how different GPU programming skills can be used to speed up different simulation calculating.
  • My understanding was/is that (program’s) acceleration by GPUs are ongoing projects here. There seem to be many new findings and experiences in this institute. Maybe you can focus more on them.

Other comments?

  • Exceeded my expectations and met the goals I am hoping to meet.
  • Overall the workshop exceeded my expectations. John Stone and Jim Phillips esp. were able to give me great tips for optimizing our code – a. Big thank you to them and all the instructor team!
  • Great! Good organization and speakers.