How to embed content from Jupyter Notebook into a separated Markdown file?

[lgonzalezsa]

Exploring Myst Markdown and found https://mystmd.org/guide/reuse-jupyter-outputs which is something I would like to do with my data analysis.

1. Do the data collection and exploration in a Jupyter Notebook (using vscode by the way)
2. Create an isolated Markdown file to write my article, conclusions, summary of results but embedding the cell output of the other Jupyter Notebooks created for the solution

I was exploring the documentation but in addition to the link above about reuse jupyter outputs, did not find an example to follow.

I was also exploring Quarto https://quarto.org/docs/authoring/notebook-embed#code-cell-options which seems to support this method, however, I would like to try Myst.

Any pointers to examples I really appreciate.

[fwkoch]

Hey @lgonzalezsa - I think you are on the right track. There is another mention in the docs here: https://mystmd.org/guide/cross-references#targeting-cells - but I don't think it adds much to the other page you found. All you should need is:

• notebook and markdown files in the same directory or a subdirectory, relative to your myst.yml file (created with myst init)
• notebook cell with the first line as #| label: my-label
• reference to this cell in the markdown file from a figure

  :::{figure} #my-label
  :::
  

A rich example that includes this feature can be found here: https://github.com/Notebooks-Now/submission-myst-full

In this example there is a labelled cell in notebooks/visualization-figure-creation-seaborn.ipynb and a reference to that label in the "results" section of article.md.

Hopefully that gets you a little further? Please let me know how it goes and if you run into more trouble!

[lgonzalezsa]

Thank you very much for your response and pointers.
I cloned the repository submission-myst-full and instantiate it locally in my laptop using podman container.

First thing I noticed was during the myst build --all, I receive a "Cannot find image" for the notebooks, however, I can see the images if I open the notebook. See the following image, the article does not render the image but the cell output exists in the notebook.

Now, I am experimenting with the embed directive because i need to use the output of a cell but not as a figure but got an error, see the video attached.

myst_embed_issue.mp4

[lgonzalezsa]

With regard to including variables in my markdown standalone article from another notebook, I continued searching for options and found the myst_nb.glue() https://jupyterbook.org/en/stable/content/executable/output-insert.html#glue-gluing

Now failing due to unknown directive: glue

Am I in the right track?

[rowanc1]

I tested the figure directive, and works fine in my repository 💯 rendering the figure properly

Wonderful! @fwkoch also tracked down the errors that you saw and has fixed them in jupyter-book/mystmd#540 we will do a new release of mystmd later today. Sorry about that, made some pretty large changes recently and missed this path.

If you have a notebook, you can use the {eval} directive, currently requires you to use the jupyterlab-myst extension.
See docs on inline expressions:
https://mystmd.org/guide/quickstart-jupyter-lab-myst#inline-expressions

We don't yet support glue in mystmd (some other things we don't support yet are here if you are looking at various docs!).
Because we are not yet evaluating the notebooks, there isn't a way to use glue, which is in memory only (AFAIK).

If you evaluate using jupyterlab-myst, and then embed the cell, I think it should work -- let me get back to you on that. We also do not yet have a way to expose executed variables outside of a notebook, I would love to have an issue on this to track that work and any initial thoughts you might have about how it would work!!

We will fix up the # | label: parsing to support the extra space, will aim to do that shortly as well:
jupyter-book/mystmd#541

[rowanc1]

Lots of the fixes from this thread are now available in mystmd v1.1.10! As well as a whole lot of improvements to the affiliations and authors by @fwkoch!!

[lgonzalezsa]

Awesome guys! way fast!

I was testing with IPython and Markdown to embed a cell output for what I need and seems reasonable alternative to include placeholder to fill with the computation notebook.

#| label: test-embed
from IPython.display import HTML, Markdown, display
df_ign['DateTime'] = pd.to_datetime(df_ign['Date'] + ' ' + df_ign['Time'])
display(Markdown(f"testing {df_ign['DateTime'][0]} with Markdown"))

[rowanc1]

Makes me think that we should have an embed role, perhaps, as well as a directive, which would be styled differently. As well, it would be good to support markdown as a type a bit easier!

One of the other ideas that I have been kicking around with @stevejpurves is to have a json display output somewhere in your notebook, and then being able to refer to those values in your document.

My study had {var}`#notebook-cell.participants` participants with a p-value of {var}`#notebook-cell.p_value`.

Not quite sure about the syntax, regardless would love to have some input! The benefit of this is that it encourages you to expose data from a computation/notebook in a structured way. There could also be a mini library in python that encodes an output as well if we want, maybe it just gets us back to glue? I will open an issue on this and tag you for more discussion.