Skip to content

refacator discrete tokenizer #2592

Merged
mravanelli merged 22 commits into
speechbrain:developfrom
poonehmousavi:discrete_token_fix
Jul 17, 2024
Merged

refacator discrete tokenizer #2592
mravanelli merged 22 commits into
speechbrain:developfrom
poonehmousavi:discrete_token_fix

Conversation

@poonehmousavi

@poonehmousavi poonehmousavi commented Jul 5, 2024

Copy link
Copy Markdown
Collaborator

What does this PR do?

  • Move all discrete tokenizers to discrete_tokenizer folder
  • update LJSpeech/quantization with new quantization HF repo and merge all hubert,wavlm, and wab2bev2 into one file
  • update discrete_ssl with new quantization HF repo
Before submitting
  • Did you read the contributor guideline?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Does your code adhere to project-specific code style and conventions?

PR review

Reviewer checklist
  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified
  • Confirm that the changes adhere to compatibility requirements (e.g., Python version, platform)
  • Review the self-review checklist to ensure the code is ready for review

@poonehmousavi poonehmousavi requested a review from mravanelli July 5, 2024 00:00
@poonehmousavi poonehmousavi self-assigned this Jul 5, 2024
@poonehmousavi poonehmousavi added the enhancement New feature or request label Jul 5, 2024
@poonehmousavi poonehmousavi requested review from mravanelli and removed request for mravanelli July 5, 2024 00:47
@poonehmousavi poonehmousavi mentioned this pull request Jul 5, 2024
Merged
12 tasks
@poonehmousavi poonehmousavi requested a review from TParcollet July 5, 2024 15:17
@poonehmousavi

poonehmousavi commented Jul 5, 2024

Copy link
Copy Markdown
Collaborator Author

@TParcollet this is the PR for the discrete tokenizer... even the other models are dependant on other libraries..so they are not exactly SB homemade model

@poonehmousavi poonehmousavi changed the title change discrete tokenizer folder refacator discrete tokenizer Jul 16, 2024
@mravanelli

mravanelli commented Jul 17, 2024

Copy link
Copy Markdown
Collaborator

Thank you @poonehmousavi

@mravanelli mravanelli merged commit 8552c8b into speechbrain:develop Jul 17, 2024
@poonehmousavi poonehmousavi deleted the discrete_token_fix branch July 29, 2024 16:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mravanelli mravanelli mravanelli approved these changes
@TParcollet TParcollet Awaiting requested review from TParcollet

Assignees

@poonehmousavi poonehmousavi

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@poonehmousavi @mravanelli