Skip to content

New module for visualizing pipeline data in a table#594

Open
kkhchung wants to merge 1 commit into
python-microscopy:masterfrom
kkhchung:pipeline-table
Open

New module for visualizing pipeline data in a table#594
kkhchung wants to merge 1 commit into
python-microscopy:masterfrom
kkhchung:pipeline-table

Conversation

@kkhchung

@kkhchung kkhchung commented Nov 7, 2020

Copy link
Copy Markdown
Contributor

Addresses issue #584.

Is this a bugfix or an enhancement?
enhancement

Proposed changes:
Module for viewing pipeline data in a table.
Start by calling pipelineTable.Plug(MainWindow) in VisGUI shell.
Support cell highlighting and sorting.
Support basic hist/scatter plots.

The pipeline is copied to a recarray for fast data access. Needs to be refreshed when the pipeline changes.

Checklist:

  • Tested with numpy=1.14
    1.16.6 and 1.19.1
  • Tested on python 2.7 and 3.6
    2.7 and 3.7
  • Tested with wx=3.x and wx=4.x [if UI code]
    4.04
  • Does the PR avoid variable renaming in existing code, whitespace changes, and other forms of tidying? [There is a place for code tidying, but it makes reviewing
    much simpler if this is kept separate from functional changes]

If an enhancement (or non-trivial bugfix):

  • Has this been discussed in advance (feature request, PR proposal, email, or direct conversation)?
  • Does this change how users interact with the software? How will these changes be communicated?
  • Does this maintain backwards compatibility with old data?
  • Does this change the required dependencies?
  • Are there any other side effects of the change?

Support sorting and highlighting.
Include basic graphing support.
@codecov

codecov Bot commented Nov 7, 2020

Copy link
Copy Markdown

Codecov Report

Merging #594 (f9b5dbc) into master (f865908) will decrease coverage by 0.02%.
The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master    #594      +/-   ##
=========================================
- Coverage    8.30%   8.27%   -0.03%     
=========================================
  Files         848     849       +1     
  Lines       97595   97911     +316     
=========================================
+ Hits         8102    8105       +3     
- Misses      89493   89806     +313     
Impacted Files Coverage Δ
PYME/DSView/modules/pipelineTable.py 0.00% <0.00%> (ø)
PYME/localization/FitFactories/InterpFitR.py 80.90% <0.00%> (+2.72%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f865908...f9b5dbc. Read the comment docs.

@David-Baddeley

David-Baddeley commented Nov 8, 2020

Copy link
Copy Markdown
Contributor

Thanks! In the medium term I'd be interested in either adapting this as a generic viewer for tabular data (both the VisGUI pipeline and, e.g., the output of measurements on pixel based images), or creating a new tabular viewer that is more or less a hybrid of this and the existing recArrayView - copying some features but potentially omitting others, e.g. sort*. A few quick thoughts/questions to help me better understand the code below:

  • the use of global variables to store references to the pipeline (along with other stuff) has the potential to be problematic as it means that there can only ever be one table viewer (i.e. you can't have e.g. one viewer for the main pipeline and one for the results of some analysis on a generated image).
  • file position in hierarchy seems a bit off - this would seem to be a VisGUI plugin, but is in the dh5view plugins folder (this is something that I can easily change on merging, just wanted to clarify intent)
  • how necessary for performance is the caching? In general access to the pipeline output should be pretty fast and it would be nice to avoid a copy if we can. I can see an argument that the caching might be important when displaying sorted output (I can think of a more memory-efficient way of doing this but want to know I'm on the right track before suggesting it).

(*) the reason why I might want to omit sort in a generic tabular viewer is that it would be nice if what you saw in the viewer was the same as what you got when you saved a .csv or similar when running a recipe automatically. I'm not yet decided on how important that is.

@kkhchung

Copy link
Copy Markdown
Contributor Author
  • I was just using this to view the pipeline so I wasn't worried about using global variables. It probably does need quite a bit of rewrite for use as a generic tabular viewer.
  • I have no opinion on the file hierarchy since I'm not familiar with it at all. Sorry if I put it in the wrong place.
  • Without caching there is a delay with updating the table when scrolling. Not unbearable depending on how you use it. I left the code in if you want to comment it in for testing. Link
  • The sort function is performed by the wx grid class in the GUI. I don't think the underlying object is actually sorted. Right now if you right click on the column heading it cycles between ascend, descend and unsorted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants