The default method is to color by the atom name. The way it works is that there is a color category called `Name' which contains a list of all the atom names (e.g., CA, N, O5', and H) that have been loaded into VMD. Each name is assigned one of the 16 main colors (e.g., cyan, blue, red, and white). When the drawing representation needs a color for a specific atom, it looks in the appropriate color category and finds that CA is colored cyan, N is blue, and so on.
Most of the coloring methods are based on color categories, so coloring by `ResName' colors each residue name differently, `SegName' colors each segment differently, and so on. The mapping between a given item in a color category and a color can be changed using the Color window. This allows users to make atoms with the name CA be black and the residue CYS be yellow. Some attention was given to making the colors reasonable, so that oxygens are red, nitrogens blue, sulphur and cysteines yellow, etc.