.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto/deployment/configure_use_gpus.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_auto_deployment_configure_use_gpus.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_deployment_configure_use_gpus.py:


.. _configure-gpus:

Configuring Flyte to Access GPUs
--------------------------------

Along with the simpler resources like CPU/Memory, you may want to configure and access GPU resources. Flyte
allows you to configure the GPU access poilcy for your cluster. GPUs are expensive and it would not be ideal to
treat machines with GPUs and machines with CPUs equally. You may want to reserve machines with GPUs for tasks
that explicitly request GPUs. To achieve this, Flyte uses the Kubernetes concept of `taints and tolerations <https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/>`__.

You can configure Flyte backend to automatically schedule your task onto a node with GPUs by tolerating specific taints.
This configuration is controlled under generic k8s plugin configuration as can be found `here <https://github.com/flyteorg/flyteplugins/blob/5a00b19d88b93f9636410a41f81a73356a711482/go/tasks/pluginmachinery/flytek8s/config/config.go#L120>`__.

The idea of this configuration is that whenever a task that can execute on Kubernetes requests for GPUs, it automatically
adds the matching toleration for that resource (in this case, ``gpu``) to the generated PodSpec.
As it follows here, you can configure it to access specific resources using the tolerations for all resources supported by
Kubernetes.

Here's an example configuration:

.. code-block:: yaml

    plugins:
      k8s:
        resource-tolerations:
          - nvidia.com/gpu:
            - key: "key1"
              operator: "Equal"
              value: "value1"
              effect: "NoSchedule"

Getting this configuration into your deployment will depend on how Flyte is deployed on your cluster. If you use the default Opta/Helm route, you'll need to amend your Helm chart values (`example <https://github.com/flyteorg/flyte/blob/cc127265aec490ad9537d29bd7baff828043c6f5/charts/flyte-core/values.yaml#L629>`__) so that they end up `here <https://github.com/flyteorg/flyte/blob/3d265f166fcdd8e20b07ff82b494c0a7f6b7b108/deployment/eks/flyte_helm_generated.yaml#L521>`__.


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  0.000 seconds)


.. _sphx_glr_download_auto_deployment_configure_use_gpus.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: configure_use_gpus.py <configure_use_gpus.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: configure_use_gpus.ipynb <configure_use_gpus.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_