Skip to content

SimStore: Support for OpenMM snapshots - #949

Merged
dwhswenson merged 15 commits into
openpathsampling:masterfrom
dwhswenson:simstore-openmm
Dec 23, 2020
Merged

SimStore: Support for OpenMM snapshots#949
dwhswenson merged 15 commits into
openpathsampling:masterfrom
dwhswenson:simstore-openmm

Conversation

@dwhswenson

Copy link
Copy Markdown
Member

This will make it possible to store OpenMM snapshots in SimStore.

Note: this changes OpenMM snapshots so that the KineticContainer and StaticContainer objects, which are wrappers around the velocities and coordinates/box vectors, respectively, now also carry information about the engine of their associated snapshot. This should be fully backward compatible with the old storage. I've done some minor testing of that, but it would be nice if someone with older files (maybe @sroet, @arjunwadhawan, or @hejung?) could double-check that they can open/analyze older netcdfplus files when they check out this branch.

This also involves significant refactoring under the hood of SimStore, making it more flexible with regards to adding user-defined types.

I'm going to add some unit tests and docstrings before declaring it ready for review, but it's already at a point where I'm hoping to get some help with checking backward compatibility.

@sroet

sroet commented Dec 7, 2020

Copy link
Copy Markdown
Member

@dwhswenson I don't have any old netcdf available that I would like to load up (all of them used to take a long time to even open up).
If needed, I could spin up a small simulation in an old OPS version and see if I can load it with this branch. Just ping me if you want me to do that 😉

@dwhswenson

Copy link
Copy Markdown
Member Author

@sroet : The main thing I want to check is that any existing analysis (especially anything that requires snapshot coord/vels, like calculating a new CV on existing data or even simulation restarts) still works. So if you have any scripts along those lines that will work on a small dataset, then please do! (Obviously, some analyses, like TIS, only work with a lot of data.)

I've checked a few basic analysis scripts that I have, and it seemed to work. In principle, everything should work -- older files will just create snapshots with, e.g., snap.kinetics.engine is None. There might be some problems with loading such snapshots from netcdfplus and re-saving as SimStore, but switching storages is going to be hard anyway.

@dwhswenson
dwhswenson marked this pull request as ready for review December 20, 2020 06:01
@dwhswenson

Copy link
Copy Markdown
Member Author

This is ready for review and comment. I will leave it open for at least 48 hours, merging no earlier than Tue 22 Dec 07:00 GMT (08:00 local).

@sroet

sroet commented Dec 22, 2020

Copy link
Copy Markdown
Member

@sroet : The main thing I want to check is that any existing analysis (especially anything that requires snapshot coord/vels, like calculating a new CV on existing data or even simulation restarts) still works. So if you have any scripts along those lines that will work on a small dataset, then please do! (Obviously, some analyses, like TIS, only work with a lot of data.)

The .nc file that I managed to run

  • could load up an old trajectory (Think > 2.5 years old)
  • load state definitions (from a different file, saved by OPS master), and use them for scheme.initial_conditions_from_trajectories(traj) and save these initial conditions on this branch
  • load state definitions (from a different file, saved by this branch) and use them for scheme.initial_conditions_from_trajectories(traj) and save these initial conditions on this branch

This seems to me that at least continuing simulations (into new storgaes) is possible and is as far as I have time/hardware to spend on this.
(As getting the state stores to work was already a pain, as I could not get a working python2 env up and running to load the old marshall code back in)

@dwhswenson

Copy link
Copy Markdown
Member Author

Thanks @sroet : If this can load a trajectory that's that old, including recalculating things (as would be needed from initial_conditions_from_trajectories if you don't have CVs cached, which sounds impossible in your case), then this should be fine. Merging!

@dwhswenson
dwhswenson merged commit 9495b54 into openpathsampling:master Dec 23, 2020
@dwhswenson
dwhswenson deleted the simstore-openmm branch December 23, 2020 02:39
@dwhswenson dwhswenson mentioned this pull request Dec 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants