.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto/integrations/flytekit_plugins/papermilltasks/simple.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_integrations_flytekit_plugins_papermilltasks_simple.py: Jupyter Notebook Tasks ----------------------- In this example, we will show how to create a flyte task that runs a simple notebook, accepts one input variable, transforms it, and produces one output. This can be generalized to multiple inputs and outputs. .. GENERATED FROM PYTHON SOURCE LINES 8-15 .. code-block:: default import math import os import pathlib from flytekit import kwtypes, task, workflow from flytekitplugins.papermill import NotebookTask .. GENERATED FROM PYTHON SOURCE LINES 16-31 How to specify inputs and outputs ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1. After you are satisfied with the notebook, ensure that the first cell only has the input variables for the notebook. Now add the tag ``parameters`` for the first cell. .. image:: https://raw.githubusercontent.com/flyteorg/static-resources/main/flytesnacks/user_guide/parameters.png :alt: Example of "parameters tag" added to the cell with input variables 2. Typically at the last cell of the notebook (which does not need to be the last cell), add a tag ``outputs`` for the intended cell. .. image:: https://raw.githubusercontent.com/flyteorg/static-resources/main/flytesnacks/user_guide/outputs.png :alt: Example of "parameters tag" added to the cell with input variables 3. In a python file, create a new task at the ``module`` level. An example task is shown below: .. GENERATED FROM PYTHON SOURCE LINES 31-40 .. code-block:: default nb = NotebookTask( name="simple-nb", notebook_path=os.path.join( pathlib.Path(__file__).parent.absolute(), "nb-simple.ipynb" ), render_deck=True, inputs=kwtypes(v=float), outputs=kwtypes(square=float), ) .. GENERATED FROM PYTHON SOURCE LINES 41-46 .. note:: - Note the notebook_path. This is the absolute path to the actual notebook. - Note the inputs and outputs. The variable names match the variable names in the jupyter notebook. - You can see the notebook on Flyte deck if ``render_deck`` is set to true. .. GENERATED FROM PYTHON SOURCE LINES 48-55 .. figure:: https://i.imgur.com/ogfVpr2.png :alt: Notebook :class: with-shadow Other tasks ^^^^^^^^^^^^^^^ You can definitely declare other tasks and seamlessly work with notebook tasks. The example below shows how to declare a task that accepts the squared value from the notebook and provides a sqrt: .. GENERATED FROM PYTHON SOURCE LINES 55-60 .. code-block:: default @task def square_root_task(f: float) -> float: return math.sqrt(f) .. GENERATED FROM PYTHON SOURCE LINES 61-62 Now treat the notebook task as a regular task: .. GENERATED FROM PYTHON SOURCE LINES 62-68 .. code-block:: default @workflow def nb_to_python_wf(f: float) -> float: out = nb(v=f) return square_root_task(f=out.square) .. GENERATED FROM PYTHON SOURCE LINES 69-70 And execute the task locally as well: .. GENERATED FROM PYTHON SOURCE LINES 70-73 .. code-block:: default if __name__ == "__main__": print(nb_to_python_wf(f=3.14)) .. GENERATED FROM PYTHON SOURCE LINES 74-80 Why Are There 3 Outputs? ^^^^^^^^^^^^^^^^^^^^^^^^ On executing, you should see 3 outputs instead of the expected one, because this task generates 2 implicit outputs. One of them is the executed notebook (captured) and a rendered (HTML) of the executed notebook. In this case they are called ``nb-simple-out.ipynb`` and ``nb-simple-out.html``, respectively. .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 0.000 seconds) .. _sphx_glr_download_auto_integrations_flytekit_plugins_papermilltasks_simple.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: simple.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: simple.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_