Skip to content

Spark 3.3/3.4: support read of partition metadata column when table is over 1k#10641

Merged
szehon-ho merged 2 commits into
apache:mainfrom
dramaticlly:spark3334partitionsWith1kColu
Jul 12, 2024
Merged

Spark 3.3/3.4: support read of partition metadata column when table is over 1k#10641
szehon-ho merged 2 commits into
apache:mainfrom
dramaticlly:spark3334partitionsWith1kColu

Conversation

@dramaticlly

@dramaticlly dramaticlly commented Jul 5, 2024

Copy link
Copy Markdown
Contributor

Backport of pull request #10547 but looks like junit5 and assertJ is not there so keep the original unit test style. Also skip the vectorized read of ORC in Spark 3.3 unit tests

@github-actions github-actions Bot added the spark label Jul 5, 2024
@dramaticlly dramaticlly closed this Jul 5, 2024
@dramaticlly dramaticlly reopened this Jul 5, 2024
@dramaticlly

Copy link
Copy Markdown
Contributor Author

@szehon-ho or @RussellSpitzer if you want to take a look at this back port

@dramaticlly

Copy link
Copy Markdown
Contributor Author

@szehon-ho

@szehon-ho szehon-ho merged commit 9724aa2 into apache:main Jul 12, 2024
@szehon-ho

Copy link
Copy Markdown
Member

Merged, thanks @dramaticlly

jasonf20 pushed a commit to jasonf20/iceberg that referenced this pull request Aug 4, 2024
zachdisc pushed a commit to zachdisc/iceberg that referenced this pull request Dec 23, 2024
@dramaticlly dramaticlly deleted the spark3334partitionsWith1kColu branch January 26, 2025 04:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants