Codecov Report

Merging #2261 (a82fc2f) into main (08e57d7) will decrease coverage by 0.00%.
The diff coverage is 0.00%.

❗ Current head a82fc2f differs from pull request most recent head 8711b84. Consider uploading reports for the commit 8711b84 to get more accurate results

@@            Coverage Diff             @@
##             main    #2261      +/-   ##
==========================================
- Coverage   62.75%   62.74%   -0.01%     
==========================================
  Files         130      130              
  Lines       16835    16837       +2     
==========================================
  Hits        10564    10564              
- Misses       6271     6273       +2     

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 08e57d7...8711b84. Read the comment docs.

result = Observations.download_file(uri, cloud_only=True)
ERROR: Error pulling from S3 bucket: 'productFilename' [astroquery.mast.observations]
WARNING: Skipping file... [astroquery.mast.observations]

Preventing file skip shows that the issue is in get_cloud_uri:

astroquery/astroquery/mast/cloud.py in get_cloud_uri(self, data_product, include_bucket, full_url)
    130                 raise
    131 
--> 132         warnings.warn("Unable to locate file {}.".format(data_product['productFilename']), NoResultsWarning)
    133         return None
    134 

KeyError: 'productFilename'

And yeah, it just looks like the data isn't there yet:

from astroquery.mast import cloud

ca = cloud.CloudAccess()
ca.get_cloud_uri(galex_products[1])

WARNING: NoResultsWarning: Unable to locate file [Jenny edit: CENSORED FOR SECURITY]. [astroquery.mast.cloud]

The test below shows galex will be supported for the cloud with the changes in this PR, so I think all that's left now is to way for the GALEX data to be uploaded into the cloud and I will check back on this PR to re-run tests and troubleshoot if needed:

ca.is_supported(data_product=galex_products[0])

Out[27]: True

A separate issue is how the data_product for supported missions has been working with get_cloud_uri all this time when get_cloud_uri only takes Astropy tables but data_product has always been a python dict.

data_product is created here, with data_uri being the keyword value:

Then data_product is fed into here:

Which feeds it into get_cloud_uri in the same form it came in (a python dict):

But get_cloud_uri takes only Astropy rows:

Which is why I'm getting the productFilename key error because it's trying to get a column out of my python dict. Need to backtrack and watch things run with a Tess or hst input and see if I missed a stepped where that conversion happens.

It's not much help for you for this PR, but ultimately I think we need to make these download functions a bit more generic/ or rewrite them and put into utils, so eventually the same approach will work for all three archives.

I think this is ready to go. Based on convos with the team, it sounds like this is a high priority one, so should take precedent over #2279 . Thanks!

Thanks @jaymedina!