Skip to content

feat: redesign python API #713

Description

@SemyonSinchenko

Is your feature request related to a problem? Please describe.
It is really hard to maintain due to two implementations.

Describe the solution you would like

  • All the logic is in the top-level graphframes.py instead of being duplicated in connect / classic
  • Naming convention should follow Python, not Scala (snake_case)
  • Pregel and AM should be top-level classes without details of the implementation
  • Connect / Classic should not contain logic, docs, etc., but should be just simple functions classes to proxy the py4j or protobuf

Component

  • Scala Core Internal
  • Scala API
  • Spark Connect Plugin
  • Infrastructure
  • PySpark Classic
  • PySpark Connect

Additional context
BREAKING CHANGES

I see it as a part of the planning 1.0 release.

Are you planning on creating a PR?

  • I'm willing to make a pull-request

Metadata

Metadata

Assignees

No one assigned

    Fields

    No fields configured for Feature.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions