Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not finetuned generalised checkpoint #7

Open
kajalsanklecha opened this issue Oct 29, 2024 · 2 comments
Open

Not finetuned generalised checkpoint #7

kajalsanklecha opened this issue Oct 29, 2024 · 2 comments

Comments

@kajalsanklecha
Copy link

Hi Authors,

There are checkpoints available for models finetuned for the generation of Chairs, tables, lamps, and benches.

Can you please share the checkpoint file of the generalized "not finetuned" model for the 3 variations of the number of parameters?

@kajalsanklecha kajalsanklecha changed the title Non finetuned generalised checkpoint Not finetuned generalised checkpoint Oct 29, 2024
@kajalsanklecha
Copy link
Author

What does the "testing the model from scratch" prompt mean in the code. Does it mean it is taking the non fine-tuned model for inference?

@ch3cook-fdu
Copy link
Collaborator

We provide pre-trained weights for different sizes of models (i.e. 125m, 350m, and 1.3b) on huggingface🤗. Download the pre-trained weights from the links below to replace the pytorch_model.bin files in the corresponding folders under the ./mesh-xl/ folder. The model details are shown below:

Model Size #Layers #Heads $d_\text{model}$ $d_\text{FFN}$ GPU Hours Download Link
125M 12 12 768 3072 1944 download link
350M 24 16 1024 4096 6000 download link
1.3B 24 32 2048 8192 23232 download link

The "testing the model from scratch" is only used for debug propose.

For more information, please refer to the readme.md file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants