ProjectBranchManager.list() return type correct? #2556

bmmcwhirt · 2023-04-26T15:01:14Z

bmmcwhirt
Apr 26, 2023

I am working with a ProjectBranchManager and getting a list of branches has the return type (RESTObjectList | List[RESTObject])

I would expect (List[ProjectBranch] | None).

I'm clearly not understanding the purpose of the ProjectBranchManager. What I do know is that:

branches = gl.projects.get(id).branches.list()

Does not work in any IDE auto completion I have worked with, and even using:

project = gl.projects.get(id)
branches = project.branches.list()

Also has issues with auto complete.

project: Gitlab.v4.objects.Project = gl.projects.get(id)
branches: List[Gitlab.v4.objects.ProjectBranch] = project.branches.list()

Auto complete likes this but the linters go crazy because the return types do not match.

I very much like being able to hit tab or return to autocomplete as it makes coding faster. However my code has to pass linters, snyk, sonaqube and Prisma Cloud scans to be merged and at present it's impossible with the way I am doing things.

So the question is what am I doing wrong and how to I do it correctly?

Answered by nejch

Apr 26, 2023

@bmmcwhirt thanks for the write-up! You're not doing anything wrong, there are simply some limitations with our ListMixin return types (and a few others).

The return type depends on the method signature, e.g. whether you use iterator=False (default) for listing, which returns a list. If you use iterator=True, it returns an iterator (RESTObjectList) for more efficient processing (https://python-gitlab.readthedocs.io/en/stable/api-usage.html#pagination). We just haven't taken the time to narrow this down with overloads or some other approach.

You can do some type narrowing on your own though, this should also already be documented here:
https://python-gitlab.readthedocs.io/en/stable/api-usa…

View full answer

nejch · 2023-04-26T15:34:06Z

nejch
Apr 26, 2023
Maintainer

@bmmcwhirt thanks for the write-up! You're not doing anything wrong, there are simply some limitations with our ListMixin return types (and a few others).

The return type depends on the method signature, e.g. whether you use iterator=False (default) for listing, which returns a list. If you use iterator=True, it returns an iterator (RESTObjectList) for more efficient processing (https://python-gitlab.readthedocs.io/en/stable/api-usage.html#pagination). We just haven't taken the time to narrow this down with overloads or some other approach.

You can do some type narrowing on your own though, this should also already be documented here:
https://python-gitlab.readthedocs.io/en/stable/api-usage-advanced.html#typing

See typing for other issues related to typing.

0 replies

bmmcwhirt · 2023-04-26T16:26:09Z

bmmcwhirt
Apr 26, 2023
Author

Thanks. If I come up with a good well formatted solution I'll do a fork/pull request for it to be reviewed.

2 replies

bmmcwhirt Apr 26, 2023
Author

If anyone comes across this, here is my inelegant solution. There seems to be no GitlabProjectBranchesList so I do have to return GitlabList. There does need to be some try/excepts added. I'm also using a custom log manager so you can ignore that part.

import sys
from gitlab import GitlabGetError, GitlabList
from gitlab import cast
from gitlab.exceptions import GitlabAuthenticationError, GitlabListError
from gitlab.v4.objects import Project, ProjectBranch, ProjectBranchManager
from gitlab.client import Gitlab

from GitlabPyUtils import log_manager


class BranchUtils:
    """ Internal Gitlabe branch utilities.

    :return: Self instance.
    :rtype: BranchUtils
    """    
    proj_branch_mgr: ProjectBranchManager

    def __init__(self, gl_conn: Gitlab, project: Project) -> None:
        """__init__ Initialize class.

        :param gl_conn: Gitlab client object instance.
        :type gl_conn: Gitlab
        :param project: Gitlab Project object.
        :type project: Project
        """
        try:
            self.proj_branch_mgr = ProjectBranchManager(
                gl=gl_conn, parent=project
            )
        except GitlabAuthenticationError as auth_error:
            log_manager.err_log.critical(auth_error.error_message)
            sys.exit(1)
        except GitlabGetError as get_error:
            log_manager.err_log.critical(get_error.error_message)
            sys.exit(1)

    def get_branches_list(
        self
    ) -> list[ProjectBranch] | GitlabList | None:
        """Get branch list with propper typecast.

        :return: Itterable list or list of ProjectBranc, or None.
        :rtype: list[ProjectBranch] | GitlabList | None
        """
        intermin_branches = self.proj_branch_mgr.list()
        branches: list[ProjectBranch] = list[ProjectBranch]()
        if isinstance(intermin_branches, type(branches)):
            branches = cast(type(branches), intermin_branches)
        elif isinstance(intermin_branches, GitlabList):
            branches.__class__ = GitlabList
            branches.__dict__ = intermin_branches.__dict__
        else:
            return None
        return branches

bmmcwhirt Apr 27, 2023
Author

I made some corrections the the above work around. I have also added comments and suggestions.

I copied my entire code, but again I am using a custom logger so the get_branch method is just a wrapper for that purpose.

""" In house Gitlab branch utilities."""


import sys

from gitlab import GitlabGetError, GitlabList, cast
from gitlab.base import RESTObjectList, RESTObject
from gitlab.client import Gitlab
from gitlab.exceptions import GitlabAuthenticationError, GitlabListError
from gitlab.v4.objects import Project, ProjectBranch, ProjectBranchManager

from GitlabPyUtils import log_manager


class BranchUtils:
    """Internal Gitlabe branch utilities.

    :return: Self instance.
    :rtype: BranchUtils
    """

    proj_branch_mgr: ProjectBranchManager

    def __init__(self, gl_conn: Gitlab, project: Project) -> None:
        """__init__ Initialize class.

        :param gl_conn: Gitlab client object instance.
        :type gl_conn: Gitlab
        :param project: Gitlab Project object.
        :type project: Project
        """
        try:
            self.proj_branch_mgr = ProjectBranchManager(
                gl=gl_conn,
                parent=project
            )
        except GitlabAuthenticationError as auth_error:
            log_manager.err_log.critical(auth_error.error_message)
            sys.exit(1)
        except GitlabGetError as get_error:
            log_manager.err_log.critical(get_error.error_message)
            sys.exit(1)

    def get_branch(self, get_branch: str) -> ProjectBranch:
        """get_branch _summary_

        :param get_branch: _description_
        :type get_branch: str
        :return: _description_
        :rtype: ProjectBranch
        """
        try:
            branch: ProjectBranch = self.proj_branch_mgr.get(get_branch)
            return branch
        except GitlabAuthenticationError as auth_error:
            log_manager.err_log.critical(auth_error.error_message)
            sys.exit(1)
        except GitlabListError as list_error:
            log_manager.err_log.critical(list_error.error_message)
            sys.exit(1)

    # See below inside this method. If GitlabBranchList
    # is crated then the return type GitlabList should
    # be changed to GitlabBranchList
    def get_branches_list(
        self,
        get_itter: bool = True
    ) -> list[ProjectBranch] | GitlabList | None:
        """Get branch list with proper typecast.

        :return: Iterable list or list of ProjectBranc, or None.
        :rtype: list[ProjectBranch] | GitlabList | None
        """
        intermin_branches = self.proj_branch_mgr.list(iterator=get_itter)
        branches: list[ProjectBranch] | GitlabList = list[ProjectBranch]()
        if isinstance(intermin_branches, RESTObject):
            if get_itter:
                raise GitlabListError(
                    error_message='\
                    List[ProjectBranch] was requested. \
                    Invalid type was received.\
                    ',
                    response_code=-1
                )
            try:
                branches = cast(type(branches), intermin_branches)
                return branches
            except TypeError:
                log_manager.err_log.error(
                    msg='\
                    Received type could not be converted to \
                    List[ProjectBranch]\
                    '
                )
                return None

        elif isinstance(intermin_branches, RESTObjectList):
            if not get_itter:
                raise GitlabListError(
                    error_message='\
                    GitlabList was requested. Invalid type was received. \
                    ',
                    response_code=-1
                )
            try:
                # Upstream a class needs to be made, probably called
                # GitlabBranchList inheriting RESTObjectList.
                # If that happens this line should be replaced with:
                #  branches.__class__ = GitlabBranchList
                branches.__class__ = GitlabList
                branches.__dict__ = intermin_branches.__dict__
               return branches
            except TypeError:
                # If the above GitlabBranchList is created then this
                # error message should be adjusted. Suggest:
                #  msg='Received type could not be converted to \
                # GitlabBranchList.'
                log_manager.err_log.error(
                    msg='Received type could not be converted to GitlabList.'
                )
        else:
            return None

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ProjectBranchManager.list() return type correct? #2556

Uh oh!

{{title}}

Uh oh!

Replies: 2 comments 2 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

ProjectBranchManager.list() return type correct? #2556

Uh oh!

bmmcwhirt Apr 26, 2023

Replies: 2 comments · 2 replies

Uh oh!

nejch Apr 26, 2023 Maintainer

Uh oh!

bmmcwhirt Apr 26, 2023 Author

Uh oh!

bmmcwhirt Apr 26, 2023 Author

Uh oh!

bmmcwhirt Apr 27, 2023 Author

bmmcwhirt
Apr 26, 2023

Replies: 2 comments 2 replies

nejch
Apr 26, 2023
Maintainer

bmmcwhirt
Apr 26, 2023
Author

bmmcwhirt Apr 26, 2023
Author

bmmcwhirt Apr 27, 2023
Author