Skip to content

Add file check on all recipe tests #2126

Merged
Adel-Moumen merged 37 commits into
speechbrain:developfrom
mravanelli:morechecks
Aug 16, 2023
Merged

Add file check on all recipe tests #2126
Adel-Moumen merged 37 commits into
speechbrain:developfrom
mravanelli:morechecks

Conversation

@mravanelli

@mravanelli mravanelli commented Aug 14, 2023

Copy link
Copy Markdown
Collaborator

This PR extends our recipe tests to perform file checks on all the existing recipes.

@mravanelli mravanelli added the enhancement New feature or request label Aug 14, 2023
@mravanelli mravanelli requested a review from Adel-Moumen August 14, 2023 04:28
@mravanelli mravanelli self-assigned this Aug 14, 2023
Adel-Moumen
Adel-Moumen reviewed Aug 14, 2023
View reviewed changes
Comment thread tests/recipes/DVoice.csv

@Adel-Moumen Adel-Moumen Aug 14, 2023

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why removing wer_test from file_exists ? Why is it empty now?

@Adel-Moumen

Adel-Moumen commented Aug 15, 2023 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Note: Errors with DVoice, fluent-speech-commands

       ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/wer_test_real.txt
        ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/train_log.txt
        ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/predictions.jsonl
        ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/log.txt
        ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/env.log
        ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/train.py
        ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/hyperparams.yaml
        ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/save/SLURM_tokenizer/tokenizer.ckpt
(182/192) Running test for SLURP_row_4...
        ... 71.11s
        ...checking files & performance...
(183/192) Running test for SLURP_row_5...
        ... 5.98s
        ...checking files & performance...
(184/192) Running test for fluent-speech-commands_row_2...
        ... 6.81s
        ERROR: Error in fluent-speech-commands_row_2 (recipes/fluent-speech-commands/direct/hparams/train.yaml). Check tests/tmp/fluent-speech-commands_row_2/stderr.txt and tests/tmp/fluent-speech-commands_row_2/stdout.txt for more info.
        ...checking files & performance...
        ERROR: The recipe fluent-speech-commands_row_2 does not contain the expected file tests/tmp/fluent-speech-commands_row_2/train_log.txt
        ERROR: The recipe fluent-speech-commands_row_2 does not contain the expected file tests/tmp/fluent-speech-commands_row_2/log.txt
        ERROR: The recipe fluent-speech-commands_row_2 does not contain the expected file tests/tmp/fluent-speech-commands_row_2/wer_test.txt
        ERROR: The recipe fluent-speech-commands_row_2 does not contain the expected file tests/tmp/fluent-speech-commands_row_2/env.log
        ERROR: The recipe fluent-speech-commands_row_2 does not contain the expected file tests/tmp/fluent-speech-commands_row_2/train.py
        ERROR: The recipe fluent-speech-commands_row_2 does not contain the expected file tests/tmp/fluent-speech-commands_row_2/hyperparams.yaml
        ERROR: The recipe fluent-speech-commands_row_2 does not contain the expected file tests/tmp/fluent-speech-commands_row_2/save/FSC_tokenizer/tokenizer.ckpt
(185/192) Running test for fluent-speech-commands_row_3...

@Adel-Moumen

Adel-Moumen commented Aug 15, 2023

Copy link
Copy Markdown
Collaborator

Note: I will rerun everything on a different cluster with a different environment. I think I have some troubles with my current cluster regarding recipe testing.

Adel-Moumen
Adel-Moumen approved these changes Aug 16, 2023
View reviewed changes

@Adel-Moumen Adel-Moumen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Tests are passing. Thanks Mirco :)

@Adel-Moumen Adel-Moumen merged commit 5565073 into speechbrain:develop Aug 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Adel-Moumen Adel-Moumen Adel-Moumen approved these changes

Assignees

@mravanelli mravanelli

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@mravanelli @Adel-Moumen