Re: Memory keep increasing.

From: Ming (ebullience_at_emails.bjut.edu.cn)
Date: Thu Nov 12 2009 - 23:51:38 CST

Hi Axel,
Thanks for your reply. I've been keeping moitering the memory changes. I found the memory still kept increasing while the job is runing. Below is the output of the "free -m -s 30". I'm sure nobody else uses this SGI Altix station. Is this increase in memory normal? Thanks!

P.S.By the way, I found the used memory increased by 6G within 12 hours.

             total used free shared buffers cached
Mem: 61521 11543 49978 0 0 166
-/+ buffers/cache: 11377 50144
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11553 49968 0 0 171
-/+ buffers/cache: 11382 50139
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11555 49965 0 0 171
-/+ buffers/cache: 11384 50137
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11558 49963 0 0 172
-/+ buffers/cache: 11385 50135
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11561 49960 0 0 173
-/+ buffers/cache: 11387 50134
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11563 49957 0 0 174
-/+ buffers/cache: 11389 50132
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11565 49956 0 0 175
-/+ buffers/cache: 11390 50131
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11569 49952 0 0 176
-/+ buffers/cache: 11393 50128
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11571 49950 0 0 177
-/+ buffers/cache: 11394 50127
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11570 49950 0 0 177
-/+ buffers/cache: 11393 50128
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11573 49948 0 0 178
-/+ buffers/cache: 11394 50127
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11575 49946 0 0 179
-/+ buffers/cache: 11395 50125
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11577 49944 0 0 180
-/+ buffers/cache: 11396 50124
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11578 49943 0 0 181
-/+ buffers/cache: 11397 50124
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11582 49938 0 0 182
-/+ buffers/cache: 11399 50121
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11584 49937 0 0 183
-/+ buffers/cache: 11400 50121
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11585 49935 0 0 184
-/+ buffers/cache: 11401 50120
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11587 49934 0 0 184
-/+ buffers/cache: 11402 50119
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11588 49932 0 0 185
-/+ buffers/cache: 11403 50118
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11588 49932 0 0 186
-/+ buffers/cache: 11402 50119
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11592 49928 0 0 187
-/+ buffers/cache: 11405 50116
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11594 49927 0 0 188
-/+ buffers/cache: 11405 50115
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11595 49926 0 0 189
-/+ buffers/cache: 11406 50115
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11597 49924 0 0 190
-/+ buffers/cache: 11407 50114
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11598 49922 0 0 190
-/+ buffers/cache: 11407 50113
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11600 49921 0 0 191
-/+ buffers/cache: 11408 50113
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11602 49919 0 0 193
-/+ buffers/cache: 11408 50112
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11603 49918 0 0 193
-/+ buffers/cache: 11409 50112
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11604 49916 0 0 194
-/+ buffers/cache: 11410 50111
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11604 49917 0 0 195
-/+ buffers/cache: 11409 50112
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11605 49915 0 0 196
-/+ buffers/cache: 11409 50112
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11607 49914 0 0 197
-/+ buffers/cache: 11410 50111
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11609 49912 0 0 198
-/+ buffers/cache: 11410 50110
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11611 49909 0 0 199
-/+ buffers/cache: 11412 50109
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11613 49907 0 0 200
-/+ buffers/cache: 11413 50108
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11615 49906 0 0 201
-/+ buffers/cache: 11414 50107
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11617 49903 0 0 202
-/+ buffers/cache: 11415 50105
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11619 49902 0 0 202
-/+ buffers/cache: 11416 50105
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11621 49900 0 0 204
-/+ buffers/cache: 11416 50105
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11615 49905 0 0 205
-/+ buffers/cache: 11410 50111
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11616 49904 0 0 206
-/+ buffers/cache: 11410 50111
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11618 49903 0 0 207
-/+ buffers/cache: 11410 50110
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11619 49902 0 0 208
-/+ buffers/cache: 11411 50110
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11620 49901 0 0 208
-/+ buffers/cache: 11411 50110
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11622 49899 0 0 210
-/+ buffers/cache: 11411 50110
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11625 49896 0 0 211
-/+ buffers/cache: 11413 50108
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11625 49896 0 0 212
-/+ buffers/cache: 11412 50108
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11627 49894 0 0 213
-/+ buffers/cache: 11413 50108
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11628 49893 0 0 214
-/+ buffers/cache: 11414 50107
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11629 49892 0 0 214
-/+ buffers/cache: 11414 50106
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11631 49890 0 0 216
-/+ buffers/cache: 11414 50107
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11633 49888 0 0 217
-/+ buffers/cache: 11415 50106
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11634 49887 0 0 218
-/+ buffers/cache: 11416 50105
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11636 49885 0 0 218
-/+ buffers/cache: 11417 50103
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11638 49883 0 0 219
-/+ buffers/cache: 11418 50102
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11640 49880 0 0 220
-/+ buffers/cache: 11420 50101
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11643 49877 0 0 222
-/+ buffers/cache: 11421 50100
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11645 49876 0 0 223
-/+ buffers/cache: 11422 50099
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11647 49874 0 0 223
-/+ buffers/cache: 11423 50098
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11647 49874 0 0 224
-/+ buffers/cache: 11422 50098
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11648 49872 0 0 224
-/+ buffers/cache: 11423 50097
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11650 49871 0 0 225
-/+ buffers/cache: 11424 50096
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11653 49868 0 0 227
-/+ buffers/cache: 11425 50095
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11654 49867 0 0 227
-/+ buffers/cache: 11426 50095
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11655 49866 0 0 228
-/+ buffers/cache: 11426 50094
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11656 49864 0 0 228
-/+ buffers/cache: 11427 50093
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11658 49863 0 0 229
-/+ buffers/cache: 11428 50092
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11660 49861 0 0 230
-/+ buffers/cache: 11429 50092
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11661 49859 0 0 231
-/+ buffers/cache: 11430 50091
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11666 49855 0 0 233
-/+ buffers/cache: 11432 50088
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11666 49854 0 0 234
-/+ buffers/cache: 11432 50088
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11667 49854 0 0 234
-/+ buffers/cache: 11432 50089
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11670 49851 0 0 235
-/+ buffers/cache: 11435 50086
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11670 49851 0 0 236
-/+ buffers/cache: 11434 50087
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11677 49844 0 0 239
-/+ buffers/cache: 11437 50084
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11678 49843 0 0 241
-/+ buffers/cache: 11437 50084
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11679 49841 0 0 241
-/+ buffers/cache: 11437 50083
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11680 49840 0 0 242
-/+ buffers/cache: 11438 50083
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11682 49839 0 0 242
-/+ buffers/cache: 11439 50082
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11683 49838 0 0 243
-/+ buffers/cache: 11439 50081
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11683 49838 0 0 244
-/+ buffers/cache: 11439 50082
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11686 49834 0 0 245
-/+ buffers/cache: 11441 50080
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11688 49833 0 0 245
-/+ buffers/cache: 11442 50079
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11689 49832 0 0 246
-/+ buffers/cache: 11443 50078
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11690 49830 0 0 246
-/+ buffers/cache: 11444 50077
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11714 49807 0 0 268
-/+ buffers/cache: 11445 50075
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11790 49730 0 0 287
-/+ buffers/cache: 11503 50018
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11810 49711 0 0 289
-/+ buffers/cache: 11520 50001
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11811 49710 0 0 290
-/+ buffers/cache: 11520 50001
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11818 49703 0 0 290
-/+ buffers/cache: 11527 49994
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11856 49665 0 0 293
-/+ buffers/cache: 11563 49958
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11872 49648 0 0 295
-/+ buffers/cache: 11577 49944
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11874 49647 0 0 295
-/+ buffers/cache: 11578 49943
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11877 49643 0 0 296
-/+ buffers/cache: 11580 49940
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11879 49642 0 0 297
-/+ buffers/cache: 11581 49939
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11881 49640 0 0 298
-/+ buffers/cache: 11583 49938
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11883 49637 0 0 298
-/+ buffers/cache: 11585 49936
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11884 49637 0 0 299
-/+ buffers/cache: 11584 49936
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11893 49628 0 0 301
-/+ buffers/cache: 11591 49929
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11904 49617 0 0 303
-/+ buffers/cache: 11600 49921
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11901 49619 0 0 304
-/+ buffers/cache: 11597 49924
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11904 49617 0 0 305
-/+ buffers/cache: 11599 49922
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11905 49616 0 0 305
-/+ buffers/cache: 11599 49922
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11905 49615 0 0 306
-/+ buffers/cache: 11599 49921
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11907 49614 0 0 306
-/+ buffers/cache: 11600 49920
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11909 49611 0 0 307
-/+ buffers/cache: 11601 49919
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11910 49610 0 0 308
-/+ buffers/cache: 11602 49919
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11912 49608 0 0 309
-/+ buffers/cache: 11603 49917
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11914 49607 0 0 309
-/+ buffers/cache: 11604 49916
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11917 49604 0 0 309
-/+ buffers/cache: 11607 49914
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11924 49597 0 0 310
-/+ buffers/cache: 11613 49907
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11939 49582 0 0 313
-/+ buffers/cache: 11625 49895
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11951 49569 0 0 314
-/+ buffers/cache: 11637 49884
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11969 49552 0 0 314
-/+ buffers/cache: 11654 49866
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 11995 49526 0 0 314
-/+ buffers/cache: 11680 49841
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12013 49508 0 0 315
-/+ buffers/cache: 11697 49823
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12032 49489 0 0 316
-/+ buffers/cache: 11715 49806
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12048 49472 0 0 318
-/+ buffers/cache: 11730 49790
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12061 49459 0 0 318
-/+ buffers/cache: 11743 49778
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12083 49438 0 0 322
-/+ buffers/cache: 11760 49761
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12093 49428 0 0 323
-/+ buffers/cache: 11769 49751
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12108 49413 0 0 326
-/+ buffers/cache: 11781 49739
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12133 49388 0 0 331
-/+ buffers/cache: 11802 49719
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12157 49364 0 0 340
-/+ buffers/cache: 11817 49704
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12169 49351 0 0 340
-/+ buffers/cache: 11829 49692
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12189 49332 0 0 343
-/+ buffers/cache: 11845 49675
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12196 49325 0 0 343
-/+ buffers/cache: 11852 49669
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12207 49314 0 0 344
-/+ buffers/cache: 11862 49658
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12219 49301 0 0 345
-/+ buffers/cache: 11874 49647
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12236 49285 0 0 346
-/+ buffers/cache: 11889 49632
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12253 49268 0 0 347
-/+ buffers/cache: 11905 49615
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12259 49262 0 0 347
-/+ buffers/cache: 11911 49609
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12263 49258 0 0 347
-/+ buffers/cache: 11915 49606
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12274 49247 0 0 348
-/+ buffers/cache: 11926 49595
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12281 49240 0 0 348
-/+ buffers/cache: 11932 49588
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12284 49237 0 0 349
-/+ buffers/cache: 11935 49586
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12285 49235 0 0 349
-/+ buffers/cache: 11936 49585
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12292 49229 0 0 350
-/+ buffers/cache: 11942 49579
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12309 49212 0 0 350
-/+ buffers/cache: 11958 49563
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12315 49206 0 0 351
-/+ buffers/cache: 11963 49558
Swap: 9999 0 9999

             total used free shared buffers cached
Mem: 61521 12322 49199 0 0 352
-/+ buffers/cache: 11970 49551
Swap: 9999 0 9999

>From: Axel Kohlmeyer <akohlmey_at_gmail.com>
>Reply-To:
>To: Ming <ebullience_at_emails.bjut.edu.cn>
>Subject: Re: namd-l: Memory keep increasing.
>Date:Thu, 12 Nov 2009 08:36:00 -0500
>
>2009/11/12 Ming
:
>> Dear all,
>>
>> I am using the NAMD_2.6_Linux-ia64-MPT version on a SGI Altix 64 cpu
>> workstation. I've set the MPI_DSM_DISTRIBUTE=yes and MPI_MEMMAP_OFF=1 in my
>> .bashrc. The job can ban successfully now but I found the memory keep
>> increasing when the job runing. Below is part of the output from the "free
>> -m -s 10" commond. Is this normal? Any suggestions? Thanks in advance!
>
>whether you have a memory leak or not is very hard to tell.
>please note, that if you are equilibrating a system, particularly
>and input fresh out of "solvate", then your neighbor lists may
>be growing for a while. you have to monitor for a long time, since
>most codes only increase their internal memory pools and never
>shrink them. at some point the memory use should level out.
>
>you also have to check what else is running on the machine.
>in any case, you still have plenty of memory left. so why worry?
>
>cheers,
> axel.
>
>>
>>
>> Mem: 61521 6114 55407 0 0 461
>> -/+ buffers/cache: 5652 55869
>> Swap: 9999 0 9999
>>
>> total used free shared buffers cached
>> Mem: 61521 6151 55369 0 0 462
>> -/+ buffers/cache: 5689 55831
>> Swap: 9999 0 9999
>>
>> total used free shared buffers cached
>> Mem: 61521 6184 55337 0 0 462
>> -/+ buffers/cache: 5721 55799
>> Swap: 9999 0 9999
>>
>> total used free shared buffers cached
>> Mem: 61521 6188 55333 0 0 462
>> -/+ buffers/cache: 5725 55795
>> Swap: 9999 0 9999
>>
>> total used free shared buffers cached
>> Mem: 61521 6191 55330 0 0 463
>> -/+ buffers/cache: 5728 55793
>> Swap: 9999 0 9999
>>
>> total used free shared buffers cached
>> Mem: 61521 6200 55321 0 0 463
>> -/+ buffers/cache: 5736 55784
>> Swap: 9999 0 9999
>>
>>
>
>
>
>--
>Dr. Axel Kohlmeyer akohlmey_at_gmail.com
>Institute for Computational Molecular Science
>College of Science and Technology
>Temple University, Philadelphia PA, USA.
>

This archive was generated by hypermail 2.1.6 : Wed Feb 29 2012 - 15:51:41 CST