VMD-L Mailing List

From: Robin Betz (robin_at_robinbetz.com)
Date: Tue Jan 28 2025 - 00:30:47 CST

Hi Stipe,

This code defines a new python type, atomsel, by instantiating the
PyTypeObject struct directly. The fields in that struct change with every
minor version, and there were changes in Python 3.11 and 3.12 afaik, so
copying the file from the repo is unlikely to fix the compilation errors
you're seeing.

I updated vmd-python to use the more future-proof way of instantiating the
type from a specification object, but it required some changes to other
files. I'll see if I can get a patch up for "regular" VMD to do this in the
next few days, or you can look at the vmd-python repo for this commit that
should mostly apply:
https://urldefense.com/v3/__https://github.com/Eigenstate/vmd-python/commit/7753e8cd7bc6574540ae63d8272f5b732b823317__;!!DZ3fjg!-gFBA79s1kX1vJZmnYmLt6oa_cekk9k3d7QZ0ipmJqC26DybZJrqXoFRg8biL-XK52xJ794xX5lEJ4fBEw$

Bset,
Robin

On Mon, Jan 27, 2025 at 3:09 AM Stipe Mustać <stipe.mustac_at_pharma.unizg.hr>
wrote:

> Hi Robin,
>
> First of all I want to apologize for answering a bit later, I was really
> away for the weekend. I really didn't expect to get the answer from the
> creator himself, thank you very much for finding some time to answer.
>
> When I use tcl interpreter i am also able to write some tcl scripts which
> I can after that use inside the vmd to manipulate my system (writing script
> and then sourcing with vmd tcl interpreter). I believe in some cases that i
> also used some features or packages which weren't inherently built into
> vmd. Can't I do the same with python interpreter? I mean write some script
> which uses some other python package (e.g. numpy) and run that script using
> vmd python interpreter?
>
> As for the error, I knew I had to change the same things as I did with
> regular build. But some parts of code still aren't the same which causes
> new errors to arise. I will open up an issue on github, but despite of that
> i will also write it here once again in case someone with the same issue
> looking here for the answer finds it.
>
> ---error:---
>
> py_atomsel.C:2138:24: error: invalid conversion from ‘PyTypeObject*’ {aka
> ‘_typeobject*’} to ‘Py_ssize_t’ {aka ‘long int’} [-fpermissive]
> 2138 | PyObject_HEAD_INIT(&PyType_Type)
> /usr/include/python3.13/object.h:138:10: note: in definition of macro
> ‘PyObject_HEAD_INIT’
> 138 | (type) \
> | ^~~~
> In file included from PluginMgr.h:31,
> from VMDApp.h:23,
> from py_atomsel.C:21:
> ResizeArray.h: In instantiation of ‘void ResizeArray<T>::append(const T&)
> [with T = Matrix4]’:
> BaseMolecule.h:460:55: required from here
> 460 | void add_instance(Matrix4 & inst) { instances.append(inst); }
> | ~~~~~~~~~~~~~~~~^~~~~~
> ResizeArray.h:108:13: warning: ‘void* memcpy(void*, const void*, size_t)’
> writing to an object of type ‘class Matrix4’ with no trivial
> copy-assignment; use copy-assignment or copy-initialization instead
> [-Wclass-memaccess]
> 108 | memcpy(newdata, data, currSize * sizeof(T));
> | ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> In file included from Timestep.h:29,
> from BaseMolecule.h:36,
> from DrawMolecule.h:26,
> from Molecule.h:27,
> from MoleculeList.h:26,
> from py_atomsel.C:22:
> Matrix4.h:26:7: note: ‘class Matrix4’ declared here
> 26 | class Matrix4 {
> | ^~~~~~~
> make: *** [Makefile:536: py_atomsel.o] Error 1
> Traceback (most recent call last):
>
> ---code downloaded from your website:---
>
> #if PY_MAJOR_VERSION >= 3
> PyTypeObject itertype = {
> PyObject_HEAD_INIT(&PyType_Type)
> "atomsel.iterator",
> sizeof(atomsel_iterobject), 0, // basic, item size
> (destructor)iter_dealloc, // dealloc
> 0, //tp_print
> 0, 0, // tp get and setattr
> 0, // tp_as_async
> 0, // tp_repr
> 0, 0, 0, // as number, sequence, mapping
> 0, 0, 0, // hash, call, str
> PyObject_GenericGetAttr, 0, // getattro, setattro
> 0, // tp_as_buffer
> Py_TPFLAGS_DEFAULT, // flags
> 0, // docstring
> 0, 0, 0, // traverse, clear, richcompare
> 0, // tp_weaklistoffset
> PyObject_SelfIter, // tp_iter
> (iternextfunc)iter_next, // tp_iternext
> iter_methods, // tp_methods
> 0, 0, 0, // members, getset, base
> };
>
> PyTypeObject Atomsel_Type = {
> PyObject_HEAD_INIT(0)
> "atomsel",
> sizeof(PyAtomSelObject), 0, // basic, item size
> (destructor)atomsel_dealloc, //dealloc
> 0, // tp_print
> 0, 0, // tp get and set attr
> 0, // tp_as_async
> (reprfunc)atomsel_repr, // tp_repr
> 0, 0, &atomsel_mapping, // as number, sequence, mapping
> 0, 0, (reprfunc)atomsel_str, // hash, call, str
> (getattrofunc) atomsel_getattro, // getattro
> (setattrofunc) atomsel_setattro, // setattro
> 0, // tp_as_buffer
> Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, // flags
> atomsel_doc, // docstring
> 0, 0, 0, // traverse, clear, richcompare
> 0, // tp_weaklistoffset
> atomsel_iter, // tp_iter
> 0, // tp_iternext
> atomselection_methods, // tp_methods,
> 0, atomsel_getset, 0, // members, getset, base
> 0, 0, 0, // tp_dict, descr_get, descr_set
> 0, 0, // dictoffset, init
> PyType_GenericAlloc, // tp_alloc
> atomsel_new, // tp_new
> PyObject_Del, // tp_free
> };
>
> ---source code downloaded from the official site:---
>
> #if PY_MAJOR_VERSION >= 3
> PyTypeObject itertype = {
> // XXX Clang++ doesn't like this initializer, see PEP 3123:
> // https://urldefense.com/v3/__https://www.python.org/dev/peps/pep-3123/__;!!DZ3fjg!-gFBA79s1kX1vJZmnYmLt6oa_cekk9k3d7QZ0ipmJqC26DybZJrqXoFRg8biL-XK52xJ794xX5lMv_pCoA$
> // See also:
> //
> https://urldefense.com/v3/__http://python3porting.com/cextensions.html*module-initialization__;Iw!!DZ3fjg!-gFBA79s1kX1vJZmnYmLt6oa_cekk9k3d7QZ0ipmJqC26DybZJrqXoFRg8biL-XK52xJ794xX5ldC6T5TQ$
> // https://urldefense.com/v3/__https://docs.python.org/3.7/extending/newtypes.html__;!!DZ3fjg!-gFBA79s1kX1vJZmnYmLt6oa_cekk9k3d7QZ0ipmJqC26DybZJrqXoFRg8biL-XK52xJ794xX5m46zUutg$
> // PyObject_HEAD_INIT(&PyType_Type)
> PyVarObject_HEAD_INIT(NULL, 0)
> "atomsel.iterator",
> sizeof(atomsel_iterobject), 0, // basic, item size
> (destructor)iter_dealloc, // dealloc
> 0, //tp_print
> 0, 0, // tp get and setattr
> 0, // tp_as_async
> 0, // tp_repr
> 0, 0, 0, // as number, sequence, mapping
> 0, 0, 0, // hash, call, str
> PyObject_GenericGetAttr, 0, // getattro, setattro
> 0, // tp_as_buffer
> Py_TPFLAGS_DEFAULT, // flags
> 0, // docstringHi Robin,
> 0, 0, 0, // traverse, clear, richcompare
> 0, // tp_weaklistoffset
> PyObject_SelfIter, // tp_iter
> (iternextfunc)iter_next, // tp_iternext
> iter_methods, // tp_methods
> 0, 0, 0, // members, getset, base
> };
>
> PyTypeObject Atomsel_Type = {
> // XXX Clang++ doesn't like this initializer:
> // https://urldefense.com/v3/__https://www.python.org/dev/peps/pep-3123/__;!!DZ3fjg!-gFBA79s1kX1vJZmnYmLt6oa_cekk9k3d7QZ0ipmJqC26DybZJrqXoFRg8biL-XK52xJ794xX5lMv_pCoA$
> // See also:
> //
> https://urldefense.com/v3/__http://python3porting.com/cextensions.html*module-initialization__;Iw!!DZ3fjg!-gFBA79s1kX1vJZmnYmLt6oa_cekk9k3d7QZ0ipmJqC26DybZJrqXoFRg8biL-XK52xJ794xX5ldC6T5TQ$
> // https://urldefense.com/v3/__https://docs.python.org/3.7/extending/newtypes.html__;!!DZ3fjg!-gFBA79s1kX1vJZmnYmLt6oa_cekk9k3d7QZ0ipmJqC26DybZJrqXoFRg8biL-XK52xJ794xX5m46zUutg$
> // PyObject_HEAD_INIT(0)
> PyVarObject_HEAD_INIT(NULL, 0)
> "atomsel",
> sizeof(PyAtomSelObject), 0, // basic, item size
> (destructor)atomsel_dealloc, //dealloc
> 0, // tp_print
> 0, 0, // tp get and set attr
> 0, // tp_as_async
> (reprfunc)atomsel_repr, // tp_repr
> 0, 0, &atomsel_mapping, // as number, sequence, mapping
> 0, 0, (reprfunc)atomsel_str, // hash, call, str
> (getattrofunc) atomsel_getattro, // getattro
> (setattrofunc) atomsel_setattro, // setattro
> 0, // tp_as_buffer
> Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE, // flags
> atomsel_doc, // docstring
> 0, 0, 0, // traverse, clear, richcompare
> 0, // tp_weaklistoffset
> atomsel_iter, // tp_iter
> 0, // tp_iternext
> atomselection_methods, // tp_methods,
> 0, atomsel_getset, 0, // members, getset, base
> 0, 0, 0, // tp_dict, descr_get, descr_set
> 0, 0, // dictoffset, init
> PyType_GenericAlloc, // tp_alloc
> atomsel_new, // tp_new
> PyObject_Del, // tp_free
> };
>
> Is it safe to replace this code chunk by the official one to solve this
> error?
>
> Best,
> Stipe
> On 1/23/25 5:33 AM, Robin Betz wrote:
>
> Hi Stipe,
>
> vmd-python is an importable Python module that is used from other scripts
> that run in the Python interpreter. It's useful for running analyses, etc,
> using Python scripts that may use other packages, and doesn't have the
> visualization window. "Regular" VMD is a standalone executable that embeds
> an interpreter (either Python or TCL) that can be used to interact with the
> program. I use "regular" VMD for visualizing and exploring MD systems and
> trajectories, and then write scripts that use vmd-python to perform
> analyses and generate plots.
>
> You'll get the same compilation errors you had for Python 3.13 + VMD
> trying to build vmd-python. If you open up an issue on the github (
> https://urldefense.com/v3/__https://github.com/Eigenstate/vmd-python__;!!DZ3fjg!-gFBA79s1kX1vJZmnYmLt6oa_cekk9k3d7QZ0ipmJqC26DybZJrqXoFRg8biL-XK52xJ794xX5mESw3qFg$
> <https://urldefense.com/v3/__https://github.com/Eigenstate/vmd-python__;!!DZ3fjg!5l7DtkIA3Lqg2MdXtD8UY3aLIy28zM-7rOJ848KJ80dXS_OJK0A15h2_C0NIVhN-chZtPN9sR-23dBJ9UA$>)
> I'll try to get to it. There's conda builds available but only through
> Python 3.1-- that's also something I've been meaning to fix.
>
> Best,
> Robin
>
> On Wed, Jan 22, 2025 at 4:20 PM Stipe Mustać <stipe.mustac_at_pharma.unizg.hr>
> wrote:
>
>> Dear Josh,
>>
>> Thanks for your help (and after some additional struggles) I finally
>> managed to compile the vmd with python interpreter. I didn't install any
>> programs listed in the documentation as dependencies (surf, stride,
>> tachyon) but the software seems to run normally?
>>
>> I tried to compile vmd-python by Robin Betz, but i have some other
>> compilation errors too which for now I can't solve. Should I open a new
>> topic or can I go on here with that compilation error here? I also don't
>> understand what are the advantages of vmd-python over just using vmd with
>> python interpreter regularly?
>>
>> Thank you very much once again, I would never be able to solve this
>> struggle without your assistance.
>> Sincerely,
>> Stipe
>> On 1/21/25 8:04 PM, Josh Vermaas wrote:
>>
>> Hi Stipe,
>>
>> Lets try to keep this in VMD-L, so that future googlers can figure out
>> what the problem was. I think this is more of the same. Looks like you are
>> using python 3.13, while my Ubuntu machine uses 3.12. The key line from
>> your logs are these:
>>
>> PythonTextInterp.C:105:24: error: ‘PyEval_CallObject’ was not declared in
>> this scope; did you mean ‘PyObject_CallObject’?
>> PythonTextInterp.C:177:3: error: ‘PySys_AddWarnOption’ was not declared
>> in this scope; did you mean ‘PySys_ResetWarnOptions’?
>>
>> For what I am sure are good reasons, occasionally python deprecates parts
>> of their API. In this case, VMD is depending on PyEval_CallObject and
>> PySys_AddWarnOption. PyEval_CallObject was deprecated in 3.9, and was
>> removed as of 3.13, which is why you are running into issues compiling it.
>> Based on a bit of googling, it looks like you just use PyObject_CallObject
>> instead.
>>
>> For AddWarnOption it looks like the API changed pretty drastically. I'm
>> not actually sure what happens if we have no warn options, so you may want
>> to consider just exempting python 3.13 and higher from AddWarnOption, as we
>> just add the defaults anyway. So something like this:
>>
>> #if PY_MAJOR_VERSION >= 3 && PY_MINOR_VERSION <= 12
>> PySys_AddWarnOption(L"default");
>> #elif PY_MAJOR_VERSION < 3
>> PySys_AddWarnOption((char*) "default");
>> #endif
>>
>> I know Diego has been working quite a bit on a "VMD 2.0" that hopefully
>> has all these deprecations taken care of.
>>
>> -Josh
>>
>>
>> On 1/21/25 12:39 PM, Stipe Mustać wrote:
>>
>> Hi Josh,
>>
>> I changed the mentioned lines and the compilation finished without
>> reporting any errors. Then I went on to compile vmd, and got the following:
>>
>> Compiling PythonTextInterp.C --> PythonTextInterp.o ...
>> PythonTextInterp.C: In function ‘void call_callbacks(const char*,
>> PyObject*)’:
>> PythonTextInterp.C:105:24: error: ‘PyEval_CallObject’ was not declared in
>> this scope; did you mean ‘PyObject_CallObject’?
>> 105 | PyObject *result = PyEval_CallObject(obj, arglist);
>> | ^~~~~~~~~~~~~~~~~
>> | PyObject_CallObject
>> PythonTextInterp.C: In constructor
>> ‘PythonTextInterp::PythonTextInterp(VMDApp*)’:
>> PythonTextInterp.C:177:3: error: ‘PySys_AddWarnOption’ was not declared
>> in this scope; did you mean ‘PySys_ResetWarnOptions’?
>> 177 | PySys_AddWarnOption(L"default");
>> | ^~~~~~~~~~~~~~~~~~~
>> | PySys_ResetWarnOptions
>> PythonTextInterp.C:199:16: warning: ‘void PySys_SetArgv(int, wchar_t**)’
>> is deprecated [-Wdeprecated-declarations]
>> 199 | PySys_SetArgv(argc, wargv);
>> | ~~~~~~~~~~~~~^~~~~~~~~~~~~
>> In file included from /usr/include/python3.13/Python.h:125,
>> from py_commands.h:24,
>> from PythonTextInterp.C:21:
>> /usr/include/python3.13/sysmodule.h:10:38: note: declared here
>> 10 | Py_DEPRECATED(3.11) PyAPI_FUNC(void) PySys_SetArgv(int, wchar_t
>> **);
>> | ^~~~~~~~~~~~~
>> make: *** [Makefile:621: PythonTextInterp.o] Error 1
>>
>> I am not sure if i provided the right directories as libraries or is it
>> really some python syntax problem? Log file again in attachment.
>>
>> Thank you very much once again for your assistance.
>>
>> Stipe
>> On 1/21/25 5:53 PM, Josh Vermaas wrote:
>>
>> Hi Stipe,
>>
>> Based on the log you attached, you want to look for compiler errors. I
>> see this in the log you attached:
>>
>> src/qcschemaplugin.c: In function ‘open_qcschema_read’:
>> src/qcschemaplugin.c:162:29: error: assignment to ‘int’ from ‘struct
>> _json_value *’ makes integer from pointer without a cast [-Wint-conversion]
>> 162 | data->totalcharge = aux_value->u.object.values[j].value;
>> | ^
>> src/qcschemaplugin.c:165:30: error: assignment to ‘int’ from ‘struct
>> _json_value *’ makes integer from pointer without a cast [-Wint-conversion]
>> 165 | data->multiplicity =
>> aux_value->u.object.values[j].value;
>>
>> To me, this looks like your compiler is pickier than mine, and is
>> treating something that used to be a warning into a full-blown error. See
>> https://urldefense.com/v3/__https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106416__;!!DZ3fjg!-gFBA79s1kX1vJZmnYmLt6oa_cekk9k3d7QZ0ipmJqC26DybZJrqXoFRg8biL-XK52xJ794xX5lFM_dxRQ$
>> <https://urldefense.com/v3/__https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106416__;!!HXCxUKc!0gXBBlCKD1JmI3G9yQRw9h2LyV4GBDdHs2sBsXyx7GpOMLrJpvzrRPhrE04oA2ru5H0Y-ZlIirL-qdqS7TgRr7NCaBz5NQ$>
>> which makes this look like the intended behavior nowadays, which is
>> unfortunate since this particular plugin hasn't been changed in YEARS. I
>> think you could make the compiler happy by adding in the cast there:
>>
>> data->totalcharge = (int) aux_value->u.object.values[j].value;
>> data->multiplicity = (int) aux_value->u.object.values[j].value;
>>
>> Or alternatively changing the compiler flags to make that error a warning
>> again.
>>
>> -Josh
>>
>>
>> On 1/21/25 11:18 AM, Stipe Mustać wrote:
>>
>> Hello Josh!
>>
>> Thank you very much for the links you provided. I used them all and I can
>> compile nothing. Neither vmd nor vmd-python. I actually realized later i
>> have to compile vmd from source and that is what i am trying to do the
>> entire day today. I was mostly referring to the guide posted by Robin Betz.
>> Unfortunately I can‘t even compile the plugins. The error I get when
>> compiling plugins is:
>>
>> make[2]: *** [Makefile:556:
>> ../compile/lib_LINUXAMD64/molfile/qcschemaplugin.o] Error 1
>> make[2]: Leaving directory
>> '/home/stipe/downloads/vmdpackaging/plugins/molfile_plugin'
>> make[1]: *** [Makefile:164: molfilelibs] Error 1
>> make[1]: Leaving directory '/home/stipe/downloads/vmdpackaging/plugins'
>> make: *** [Make-arch:338: LINUXAMD64] Error 2
>>
>> I applied that sed command you provided in your guide to switch from tcl
>> 8.5 to 8.6. I am sending the entire make output in the attachment. I
>> haven’t still installed surf, stride or tachyon. My os is arch linux and it
>> uses python 3.13.
>>
>> Thank you once again.
>> Sincerely,
>> Stipe Mustac
>>
>>
>> > Am 21.01.2025 um 16:23 schrieb Josh Vermaas <vermaasj_at_msu.edu>
>> <vermaasj_at_msu.edu>:
>> >
>> > Hi Stipe,
>> >
>> > How did you install VMD? The default VMD installations that you can
>> download from the ks.uiuc.edu website don't come with a python
>> interpreter built-in. Instead, if you want python and graphics, you end up
>> needing to compile VMD yourself.
>> https://urldefense.com/v3/__https://robinbetz.com/blog/2015/01/08/compiling-vmd-with-python-support/__;!!DZ3fjg!5Cp0UHc8gqyWY_z16AWli_FiYQKIyw3BouVGyNUkt9XAZL4zSZdjwo-vEOH5HAHnuSGU33rJAEKLeSGKjP8$
>> has a blogpost describing the process, and she's also got a way of
>> installing VMD as a python package (
>> https://urldefense.com/v3/__https://vmd.robinbetz.com/__;!!DZ3fjg!5Cp0UHc8gqyWY_z16AWli_FiYQKIyw3BouVGyNUkt9XAZL4zSZdjwo-vEOH5HAHnuSGU33rJAEKLw_8HEtI$
>> ). I've got packaging instructions for alpha versions of VMD that I use
>> within my lab as well.
>> https://urldefense.com/v3/__https://github.com/jvermaas/vmd-packaging-instructions__;!!DZ3fjg!5Cp0UHc8gqyWY_z16AWli_FiYQKIyw3BouVGyNUkt9XAZL4zSZdjwo-vEOH5HAHnuSGU33rJAEKL8qbMb2s$
>> > Hope this helps!
>> >
>> > -Josh
>> >
>> >
>> >> On 1/20/25 2:18 PM, Stipe Mustać wrote:
>> >> Hello! I am trying to use python interpreter in vmd. The error message
>> I keep getting is:
>> >>
>> >> Info) Starting Python...
>> >> 'import site' failed; use -v for traceback
>> >> rlwrap: warning: vmd crashed, killed by SIGSEGV (core dumped).
>> >> rlwrap itself has not crashed, but for transparency,
>> >> it will now kill itself with the same signal
>> >>
>> >> warnings can be silenced by the --no-warnings (-n) option
>> >> Segmentation fault (core dumped)
>> >>
>> >> I saw a person who posted in this mailing list having the same issue
>> in 2023, tried to reproduce all the tutorials and guides, but without any
>> success. I would appreciate any help.
>> >>
>> >> Sincerely,
>> >> Stipe Mustac
>> >>
>> >
>> > --
>> > Josh Vermaas
>> >
>> > vermaasj_at_msu.edu
>> > Assistant Professor, Plant Research Laboratory and Biochemistry and
>> Molecular Biology
>> > Michigan State University
>> > vermaaslab.github.io
>> <https://urldefense.com/v3/__http://vermaaslab.github.io__;!!DZ3fjg!5l7DtkIA3Lqg2MdXtD8UY3aLIy28zM-7rOJ848KJ80dXS_OJK0A15h2_C0NIVhN-chZtPN9sR-3j4Yzeaw$>
>> >
>> >
>>
>>
>> --
>> Josh Vermaas
>> vermaasj_at_msu.edu
>> Assistant Professor, Plant Research Laboratory and Biochemistry and Molecular Biology
>> Michigan State Universityvermaaslab.github.io <https://urldefense.com/v3/__http://vermaaslab.github.io__;!!DZ3fjg!5l7DtkIA3Lqg2MdXtD8UY3aLIy28zM-7rOJ848KJ80dXS_OJK0A15h2_C0NIVhN-chZtPN9sR-3j4Yzeaw$>
>>
>>
>> --
>> Josh Vermaas
>> vermaasj_at_msu.edu
>> Assistant Professor, Plant Research Laboratory and Biochemistry and Molecular Biology
>> Michigan State Universityvermaaslab.github.io <https://urldefense.com/v3/__http://vermaaslab.github.io__;!!DZ3fjg!5l7DtkIA3Lqg2MdXtD8UY3aLIy28zM-7rOJ848KJ80dXS_OJK0A15h2_C0NIVhN-chZtPN9sR-3j4Yzeaw$>
>>
>>