Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add mixed support for narration_id and uid #86

Open
willprice opened this issue Jul 20, 2020 · 3 comments
Open

Add mixed support for narration_id and uid #86

willprice opened this issue Jul 20, 2020 · 3 comments

Comments

@willprice
Copy link
Member

Such that both EK-55 and EK-100 can be used throughout the lib

@BCJuan
Copy link

BCJuan commented Oct 21, 2020

Hi,

I am interested in using this library for EK-100. I do not know if those changes have been implemented but it appears not.

I have cloned the repo and have been able to modify it to gulp and read from it (using of course the EK-100 annotations and the EK-100 dataset).

I have followed the structure detailed inside the epic-kitchens-55-lib/epic_kitchens/gulp/adapter.py:

                    frame-segments/
                    ├── P01
                    │   ├── P01_01
                    │   |   ├── P01_01_0_open-door
                    │   |   |   ├── frame_0000000008.jpg
                    │   |   |   ...
                    │   |   |   ├── frame_0000000202.jpg
                    │   |   ...
                    │   |   ├── P01_01_329_put-down-plate
                    │   |   |   ├── frame_0000098424.jpg
                    │   |   |   ...
                    │   |   |   ├── frame_0000098501.jpg
                    │   ...

For RGB frames and for only gulping and reading, the main change has been the UID_COL to 'narration_idand inadapter.py, modify the method _segment_metadata_to_clip_id to:

    def _segment_metadata_to_clip_id(self, meta):
        clip_id = "{uid}_{narration}".format(
            uid=meta[UID_COL],
            narration=meta[NARRATION_COL].lower().strip().replace(" ", "-"))
        else:
        )
        return clip_id

I am thinking that extending this library (only, and for the moment, to read and gulp) could imply the following:

  1. Add argument to EpicDatasetAdapter, like:
    def __init__(
        self,
        video_segment_dir: str,
        annotations_df: pd.DataFrame,
        frame_size: int = -1,
        extension: str = "jpg",
        labelled: bool = True,
        version: str = "55"
    ) -> None:
  1. Then, changing the method _segment_metadata_to_clip_id to:
    def _segment_metadata_to_clip_id(self, meta):
        clip_id = "{uid}_{narration}".format(
            uid=meta[UID_COL],
            narration=meta[NARRATION_COL].lower().strip().replace(" ", "-"))
        else:
        )
        return clip_id
  1. and modifying the main.py csv reading to:
    labels = pd.read_csv(labels_file)
    labels.set_index("narration_id", drop=False, inplace=True)

@willprice
Copy link
Member Author

I rewrote the gulping adapter in the TSM/TSN/TRN repo: https://github.com/epic-kitchens/C1-Action-Recognition-TSN-TRN-TSM/blob/master/src/gulp_data.py

@BCJuan
Copy link

BCJuan commented Oct 21, 2020

Great then, haven't noticed that. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants