MAST: Read product function for MastMissions#3593
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3593 +/- ##
==========================================
+ Coverage 73.17% 73.19% +0.02%
==========================================
Files 227 227
Lines 21078 21118 +40
==========================================
+ Hits 15423 15457 +34
- Misses 5655 5661 +6 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Until we have a more mature version ready, can we please ensure documentation / tutorials call out that for FITS files I presume this can be avoided once there's a "get_s3_uris()" endpoint in MissionMAST for all the supported missions beyond Roman? |
617201a to
cdda4a2
Compare
7808985 to
e1ea60e
Compare
e1ea60e to
a5214b2
Compare
bsipocz
left a comment
There was a problem hiding this comment.
code changes look good, but I have problem with the proposed changes in pyproject.toml, IMO we should avoid workaround for [all] and should not include optional dependencies to be installed for all the test jobs.
| "sphinx-astropy>=1.5", | ||
| "scipy", | ||
| "fsspec[http]", | ||
| "asdf" |
| "fsspec[http]", | ||
| "fsspec[http,s3]", | ||
| "moto[s3]", | ||
| "s3fs", | ||
| "asdf" |
There was a problem hiding this comment.
I don't think that optional dependencies should be installed for all test runs. The fact that we have fsspec and s3fs listed here is a problem of its own and should be cleaned up. So, I would not think we should add asdf.
A function in the
MastMissionsclass that can read products into memory.For FITS files, we use
astropy.io.fits.open. The product files are downloaded and cached by Astropy, and then read into aHDUListobject.For ASDF files, we use
fsspecandasdf. Our downloads for Roman data are redirected to S3, so these files are not downloaded and are streamed directly intoasdf.AsdfFileobjects.