Skip to content

OAC csv parsing error #1401

Description

@bsipocz

The cause of this issue is the HTML tag in the alias filed, that doesn't parse properly when using the csv data format.

Suggested workaround: remote data_format kwarg as it shouldn't matter for the end user what server output format we parse internally.

Cross-reference: astrocatalogs/OACAPI#11

==============

There are a few new(ish) test failures with OAC: https://travis-ci.org/astropy/astroquery/jobs/512564435#L4631

______________________ TestOACClass.test_query_object_csv ______________________
self = <astroquery.oac.tests.test_oac_remote.TestOACClass object at 0x7f8ef4fdfb00>
    def test_query_object_csv(self):
>       phot = OAC.query_object(event='SN2014J')
astroquery/oac/tests/test_oac_remote.py:32: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
astroquery/utils/class_or_instance.py:25: in f
    return self.fn(obj, *args, **kwds)
astroquery/utils/process_asyncs.py:29: in newmethod
    result = self._parse_result(response, verbose=verbose)
astroquery/oac/core.py:487: in _parse_result
    output_response = self._format_output(raw_output)
astroquery/oac/core.py:451: in _format_output
    output = json.loads(raw_output)
/home/travis/miniconda/envs/test/lib/python3.7/json/__init__.py:348: in loads
    return _default_decoder.decode(s)
/home/travis/miniconda/envs/test/lib/python3.7/json/decoder.py:337: in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <json.decoder.JSONDecoder object at 0x7f8f03188fd0>
s = 'event,escapevelocity,stellarclass,lumdist,ra,references,maxappmag,hostdec,host,instruments,sources,redshift,hostra,ma...y et al.,Steve J. Fossey",,2014/01/21,300,,,"+69:40:25.9,+69:40:26.00,+69:40:26.0",,e,-18.4,,0.1358,SN2014J,"Ia,Ia-HV"'
idx = 0
    def raw_decode(self, s, idx=0):
        """Decode a JSON document from ``s`` (a ``str`` beginning with
        a JSON document) and return a 2-tuple of the Python
        representation and the index in ``s`` where the document ended.
    
        This can be used to decode a JSON document from a string that may
        have extraneous data at the end.
    
        """
        try:
            obj, end = self.scan_once(s, idx)
        except StopIteration as err:
>           raise JSONDecodeError("Expecting value", s, err.value) from None
E           json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
/home/travis/miniconda/envs/test/lib/python3.7/json/decoder.py:355: JSONDecodeError
----------------------------- Captured stdout call -----------------------------
The API did not return a valid CSV output! 
Outputing JSON-compliant dictionary instead.
____________________ TestOACClass.test_query_region_box_csv ____________________
self = <astroquery.oac.tests.test_oac_remote.TestOACClass object at 0x7f8ef400cb38>
    def test_query_region_box_csv(self):
        phot = OAC.query_region(coordinates=self.test_coords,
                                width=self.test_width,
>                               height=self.test_height)
astroquery/oac/tests/test_oac_remote.py:53: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
astroquery/utils/class_or_instance.py:25: in f
    return self.fn(obj, *args, **kwds)
astroquery/utils/process_asyncs.py:29: in newmethod
    result = self._parse_result(response, verbose=verbose)
astroquery/oac/core.py:487: in _parse_result
    output_response = self._format_output(raw_output)
astroquery/oac/core.py:451: in _format_output
    output = json.loads(raw_output)
/home/travis/miniconda/envs/test/lib/python3.7/json/__init__.py:348: in loads
    return _default_decoder.decode(s)
/home/travis/miniconda/envs/test/lib/python3.7/json/decoder.py:337: in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <json.decoder.JSONDecoder object at 0x7f8f03188fd0>
s = 'event,escapevelocity,stellarclass,lumdist,ra,references,maxappmag,hostdec,host,instruments,sources,redshift,hostra,ma...y et al.,Steve J. Fossey",,2014/01/21,300,,,"+69:40:25.9,+69:40:26.00,+69:40:26.0",,e,-18.4,,0.1358,SN2014J,"Ia,Ia-HV"'
idx = 0
    def raw_decode(self, s, idx=0):
        """Decode a JSON document from ``s`` (a ``str`` beginning with
        a JSON document) and return a 2-tuple of the Python
        representation and the index in ``s`` where the document ended.
    
        This can be used to decode a JSON document from a string that may
        have extraneous data at the end.
    
        """
        try:
            obj, end = self.scan_once(s, idx)
        except StopIteration as err:
>           raise JSONDecodeError("Expecting value", s, err.value) from None
E           json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
/home/travis/miniconda/envs/test/lib/python3.7/json/decoder.py:355: JSONDecodeError
----------------------------- Captured stdout call -----------------------------
The API did not return a valid CSV output! 
Outputing JSON-compliant dictionary instead.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions