Evaluation of the Summer School on Theoretical and Computational Biophysics - Questionnaire 2

Lectures and Tutorials Evaluation

Questionnaire, analysis, and report: David Brandon and Klaus Schulten, TCB group, UIUC

At the end of the summer school participants were asked to evaluate the summer school's lectures and tutorials.  Rankings of the relevance of the lectures and tutorials were solicited, as were open comments about each lecture and tutorial. Participation in the evaluation was voluntary.  A total of 36 evaluation forms were collected, of approximately 53 that were picked up, for a return rate of 67.9%. 

Click here for the SS03 Lectures & Tutorials Feedback Form, and here for its cover page.

Summaries for the lectures and tutorials are comprised of three elements, 1) the proportion rating the relevance of the lecture or tutorial as highly relevant (i.e. 'very good' + 'excellent' ratings; see Table 1: Summary of Relevance Statistics below), 2) select comments considered illustrative of respondent opinion, and 3) text summarizing the main points of the total body of comments for a lecture or tutorial.  As is frequently the case with surveys, not all respondents answered all questions; the number of responses for the relevance ratings (r=) and comments (c=) are listed next to the name of each lecture and tutorial summary, e.g. (N: r=34, c=8).

Some issues to consider when reading the comments: 

• Written comments, particularly when comments are extreme in one direction or the other, tend to stick in one's head more so than statistics that may present a more accurate summary of opinion.
• There aren't enough comments to provide a sample size that can be considered representative of the entire summer school population; e.g. for one tutorial there are three comments.  Further, those responding are self-selected, i.e. those who went through with completing the evaluation form may or may not be representative of a 'typical' attendee. 
• Attendees appear to have been somewhat heterogeneous in scientific background, training, interests, and to an extent language; so, for any lecture or tutorial there was likely always someone new to the topic who needed more time, help and explanation, and at the same time someone very experienced who wanted more breadth and/or depth on the topic.

Summaries are organized below by day, lecture, and tutorial, and can be located using the navigation table below or by scrolling down the page.

Day 1

Day 1 Lecture:  Molecular Graphics and Molecular Dynamics (N: r=35, c=11)

Nearly all respondents, 91.5%, rated the lecture as highly relevant.  Sample comments are:

• "A nice broad overview of what we would be learning, and the types of problems tackled by biophysicists."
• "Good lecture & good tutorial.  Our lab has VMD, but we normally use it just for graphics in papers, PowerPoint, etc.  I found out that VMD has many more applications that we can now use in our lab."

Overall, those commenting indicate the lecture provided a good overview, and also helped frame how VMD could be used for MD.  There was some desire expressed in comments for additional technical details or resources, e.g. TCL scripts, equilibration details.

Day 1 Tutorial:  VMD Tutorial (N: r=35, c=11)

As was the case for the day one lecture, 91.4% found the tutorial content highly relevant.  Sample comments are:

• "I have used VMD extensively but still learned a lot.  I came here to learn about NAMD & its capabilities & feel that thru the tutorials, I accomplished this.  The TAs were great!"
• "Only slightly too long for the available time.  Very clear and very well thought out."

Comments are positive regarding the overall content and organization of the tutorial.  There are some suggestions that how VMD interacts with other programs be discussed, that some features could use more discussion, and that the tutorial ran long. 

Day 2

Day 2 Lecture:  Nonequilibrium Analysis of Molecular Dynamics (N: r=35, c=8)

A clear majority, 82.9%, found the lecture content to be highly relevant.  Sample comments are:

• "Also a very nice lecture - I especially enjoyed the temperature quench echoes."
• "Effects reminding us that thermodynamics is not the god of all microscopic dynamics - this was great! I love seeing kinetics and coherence."

Overall, the comments are positive, complimenting both the scope and details of the lecture, with some interest in more details on 'temperature echo'.

Day 2 Tutorial:  NAMD Tutorial (N: r=35, c=13)

Almost all those responding, 91.5%, rated the tutorial relevance as very high.  Sample comments are:

• "This was possibly the best tutorial in the SS - informative, well-organized, and easy to follow."
• "It was great to have such an in depth NAMD tutorial.  The TAs were excellent - a huge help, very knowledgeable, & eager to help."

Comments are uniformly positive regarding the organization and writing of the tutorial; there was some desire for more time, and a few comments seeking more information on selected technical details. 

Day 3

Day 3 Lecture:  Steered Molecular Dynamics  (N: r=35, c=10)

A clear majority, 82.9%, rated the relevance of the lecture as very high.  Sample comments are:

• "Three good ones in a row!  I especially enjoyed the biology of mechanical proteins."
• "Nicely motivated and historically informative presentation."

The majority of comments compliment the lecture.  A few comments suggest that more detail on umbrella sampling be included.  

Day 3 Tutorial:  Stretching Deca-Alanine (N: r=34, c=7)

The relevance of the lecture was high, with 88.3% rating relevance as very good to excellent.  Sample comments are:

• "From this tutorial I learnt a lot!"
• "This was cool - I understood a lot more by doing.'

As was the case with the day three lecture, comments are positive about the tutorial.  One comment suggests more detail on equations, and another comment suggests listeners be given more details on calculating free radicals.

Day 4

Day 4 Lecture:  Simulating Membrane Channels  (N: r=35, c=12)

A very high proportion, 91.4%, rate the lecture as highly relevant.  Sample comments are:

• "I liked how this exemplified replacing complex systems with simpler ones in order to elucidate the fundamental mechanisms of a process."
• "The lecture was really good. I learned a lot from this lecture, and the speaker made the material understandable for people that are not familiar w/membrane channels."

Overall, comments are complimentary regarding the content and presentation of the lecture; a single comment suggests fewer examples be included. 

Day 4 Tutorial:  Simulating Nanotubes  (N: r=33, c=9)

With 90.9% rating relevance as very good to excellent, the majority of respondents found this topic of interest.  Sample comments are:

• "Very intuitive tutorial, especially the change of water behavior after changing non-bond parameters."
• "Another great tutorial that complimented the lecture very well.  As usual, the TAs were a great asset."

Across responses, the tutorial is well-regarded, though because the topic was very new to some respondents, they weren't sure what to do with their newfound knowledge.  

Day 5

Day 5 Lecture:  Overview of Quantum Chemistry  (N: r=34, c=19)

Of those responding, 41.1% rated the overview of quantum chemistry lecture as highly relevant to their interests.  Sample comments are:

• "Very nice overview.  I would have liked very much to see each method described more rigorously, but this is clearly impossible with the time constraint given to Prof. Martinez."
• "Excellent overview!"

While several comments indicate the lecture was too long, and too detailed, there are also comments indicating that for this particular subject it is difficult to cover all the important topics within the given time constraints.  Other comments suggest more basic concepts for those unfamiliar with the topic, with less in-depth content on other topics, and a few other comments suggest a different presentation of the material.   

Day 5 Tutorial:  Computing Proton Affinities  (N: r=33, c=6)

A majority, 63.6%, of respondents found this tutorial highly relevant. Sample comments are:

• "This definitely made me appreciate QM methods more!"
• "The tutorial was well thought out and very clear.  However, it was very long if one tried to set up the input files instead of using the sample files provided.  Overall, I worked (around) 8 hours on this tutorial and I didn’t finish."

Comments also indicate that participants found the tutorial long, but challenging as well as interesting. 

Day 6

Day 6 Lecture:  Determining Classical Force Fields  (N: r=34, c=9)

At 67.6%, a majority of respondents found the lecture to be highly relevant.  Sample comments are:

• "Very clear"
• "Nice presentation."

The majority of comments indicate respondents found the lecture content to be helpful, with a few comments suggesting a different presentation of the material. 

Day 6 Tutorial:  CHARMM Parameters for non-standard residues  (N: r=33, c=10)

Similar to the lecture on the same day, 66.7% found the content of the tutorial to be highly relevant.  Sample comments are:

• "Great help from Markus.  Thank you very much!"
• "Very useful for people who may have to parameterize a force field."

Comments indicate participants appreciated the content of the tutorial; some comments suggesting more applied guidelines or practice would be an asset. 

Day 7

Day 7 Lecture:  Bioinformatics  (N: r=34, c=14)

A majority, 55.8%, found the bioinformatics lecture highly relevant. Sample comments are:

• "A very good overview of a field about which I knew very little."
• "This extremely exciting field should have been given more attention."

Several comments indicate the lecture was useful, though for some the topic was too new or distant from their own background to be informative.  A few comments indicate interest in more detail on software applicable to bioinformatics.

Day 7 Tutorial:  Sequence and Structure Alignment  (N: r=34, c=12)

A small majority, 52.9%, found this tutorial highly relevant.  Sample comments are:

• "Very interesting.  Working out the matrix by hand was a little tedious but it helped greatly in improving my understanding.  Possibly as good as the NAMD tutorial."
• "This was new material to me and it was nice to put in my own protein and examine homologous structures.  I even found a protein very similar to my own that I had not seen in my primary literature."

Nearly all comments are appreciative or compliment the tutorial to some degree.  Other comments address applicable software, or some specifics of the tutorial.  

Day 8

Day 8 Lecture:  Simulating Membranes  (N: r=34, c=8)

A strong majority, 88.3%, found this lecture highly relevant.  Sample comments are:

• "Fascinating.  Very pertinent to my own work."
• "It was good to have a guest speaker.  He brought a different flavour to the kinds of problems addressed and methods used to deal with them."

Comments indicate the lecture was interesting, clear, and well organized.  There were some suggestions that more be added about addressing applied aspects of the topic.  

Day 8 Lecture:  Parallel Computing (N: r=34, c=11)

A high proportion, 82.3%, of those responding found this topic highly relevant.  Same comments are:

• "Entertaining and informative lecture.  Very nice."
• "The lecture was very useful.  The lecture gave me a better picture of the advantages & problems w/parallelization."

The majority of comments are complimentary regarding the lecture. For some, the topic was familiar territory, a few others suggest changes to the presentation.   

Day 8 Tutorial:  Parallel Computing  (N: r=33, c=13)

Participants split evenly in their ratings of the relevance of this tutorial, with 50.0% indicating the tutorial was highly relevant.  Sample comments are:

• "Illustrative."
• "This was a useful tutorial, but since so many people were submitting their jobs, I couldn't finish the whole thing.  (I guess that really can't be helped, though)."

As the sample comment illustrates, there were problems in submitting jobs to the NCSA queue, that in turn may color how respondents rate the overall content of the tutorial.  Many comments indicate the topic of the tutorial was familiar territory for participants, with other comments indicating the tutorial or a simpler presentation of the same material could be good for beginners.   

Day 9

Day 9 Lecture:  Force Evaluation, Integrators, & Propagators (N: r=33, c=9)

Over two-thirds of respondents, 69.7%, found this lecture to be highly relevant.  Sample comments are:

• "Nice to see things I've read editorialized and see the formalism."
• "Very good linking lecture material to NAMD2 code & parameters."

Comments are a mixture of those who would like to have more detail in the lecture, and those who would like to see more of the basics of this topic covered.   

Day 9 Lecture:  Optimization and Monte Carlo Techniques (N: r=34, c=12)

A slight majority, 55.9%, found this lecture highly relevant.  Sample comments are:

• "Good talk."
• "Interesting."

A majority of the comments indicate those answering would have liked more advanced content in the lecture. 

Day 9 Tutorial:  Numerical Analysis  (N: r=30, c=12)

At 43.4%, less than half of those responding found this tutorial to be highly relevant.  Sample comments are:

• "It was cool to see the capabilities of mathematica (I usually use maple &/or matlab).  Also, the tutorial was well set-up to get good & specific lessons to the students.  I learned a lot."
• "This tutorial was interesting.  I didn't really understand initially what I was supposed to be seeing w/the exercises, but the TAs gave very good explanations, so I was finally able to understand by the end."

Many comments indicate participants would have liked more details included in the tutorial, and a different organization of the material.   

Day 10

Day 10 Lecture: Modeling the Photosynthetic Unit  (N: r=33, c=9)

Nearly all of those responding, 97.0%, found this lecture to be highly relevant.  Sample comments are:

• "This was an amazing lecture emphasizing a growing interest in biology and science in general - putting things into the big picture.  Models to guide intuition and calculate large systems were combined with molecular dynamics, quantum chemistry (computational HFIDFT and perturbation theory - very nice!), and biological function."
• "Klaus is a riot."

Comments indicate an appreciation of the enthusiasm and delivery skills demonstrated by the speaker, as well as finding the content interesting. 

Day 10 Tutorial: Electron and Excitation Transfer  (N: r=22, c=3)

A high percentage, 90.9%, found this tutorial to be highly relevant.  Sample comments are:

• "Excellent."
• "I liked having the option of skipping to the parts in the tutorial.  Ex:  the note on p. 5."

Comments were complimentary of the tutorial. 

Cluster Building Overview Lecture  (N: r=31, c=11)

A high proportion, 87.1%, found the lecture to be highly relevant.  Sample comments are:

• "Fast-paced and not too dry.  Disabused me of some misconceptions about clusters."
• "Very good lecture.  The speakers gave us a good overview of what to consider when building a cluster, problems that arise, etc."

The majority of comments are complimentary; a few comments suggest access to more detail be provided. 

Cluster Building Hands-on Sessions  (N: r=29, c=12)

At 96.6%, nearly all those responding  indicated this tutorial was highly relevant.  Sample comments are:

• "Another fun part of the whole summer school."
• "I really liked this & had fun with the session.  I don't know if I could completely build my own cluster yet, but I got a good idea of what is involved & what I would need to do."

All comments are complimentary, and several of the comments indicating participants gained a sense of confidence regarding their own ability to put together a cluster. 

The complete set of comments is available to TCB group members via this link:  SS03 Lecture & Tutorials Raw Data.  Others should contact sumschool03@ks.uiuc.edu to request the comments.  

 

Table 1: Summary of Relevance Statistics