↓Skip to main content
IT Blog IT Blog
  1. All Posts/

data-science-stack

Tools Shell

NVIDIA Data Science Stack

NVIDIA Data Science Stack is a tool to make it easy to setup a machine and
manage the software stacks for GPU accelerated Data Science.
This includes laptops, desktops, workstations, and cloud virtual machines.
Users can work with containers, or in a local environment.

  • Code repo: https://github.com/NVIDIA/data-science-stack
  • Releases: https://github.com/NVIDIA/data-science-stack/releases
  • Report issues: https://github.com/NVIDIA/data-science-stack/issues
  • Release planning: https://github.com/NVIDIA/data-science-stack/projects
  • Subscribe to release notifications – watch the
    https://github.com/NVIDIA/data-science-stack repository.
    We suggest “Releases Only”, if you haven’t subscribed before check the
    help for watching and unwatching repositories.

Contents

Quick Start

For usage and command documentation: ./data-science-stack help at any time.
_Note: The script is designed to run as the user, and ask for sudo password
when needed. Do not run it with sudo ...
On Ubuntu 18.04, 20.04, or Red Hat Enterprise Linux (RHEL) 8.x:

git clone https://github.com/NVIDIA/data-science-stack
cd data-science-stack
./data-science-stack setup-system

On RHEL Workstation 7.x:

git clone https://github.com/NVIDIA/data-science-stack
cd data-science-stack
./data-science-stack setup-system
# script will stop, manually install driver ... (instructions below)
./data-science-stack setup-system

On Windows Subsystem for Linux (WSL):
Note: This functionality is alpha only (and containers only) until WSL v2 becomes production ready
Follow the install instructions to install WSL v2 with CUDA support.
Then, create a a Ubuntu or RHEL VM, open a terminal, and follow OS-specific instructions above.
Next, users have a choice to use containers or a local Conda environment:

Option 1 – In a Container (Recommended for container users) 

./data-science-stack list
./data-science-stack build-container
./data-science-stack run-container

This creates and runs Jupyter in the container. Users can then connect
with the Jupyter notebook running at http://localhost:8888/
Control-C to exit.
To mount data or code into your container, see
How do I mount data into containers?
below.
The reverse of build-container is purge-container.
For information about Docker refer to https://docs.docker.com/

Option 2 – In a Local Conda Environment (Recommended for initial development work) 

./data-science-stack list
./data-science-stack build-conda-env
./data-science-stack run-jupyter

This creates the local environment and runs Jupyter. Users can then connect
with the Jupyter notebook at the address and token output by Jupyter.
Control-C to exit.
The reverse of build-conda-env is purge-conda-env.
For information about Conda environments refer to
https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html

Multiple Users

To setup multiple users on the machine, they will need to get access to
Docker and setup Conda in the account

# As the additional user
./data-science-stack setup-user
# ... use container or conda commands above

Upgrading

The script is designed to detect old versions of dependencies and upgrade
them, and create new environments/containers.
To upgrade automatically:

./data-science-stack upgrade

If a newer version of data science stack is available, the script will retrieve it and perform the upgrade.
To upgrade manually, get the new version of the script and environment configs with
git pull or with a new release .zip, and run the install steps again –
most likely setup-system and one of the build-... commands.
New environments and containers will be tagged with the version of the
script, so the old ones will not be modified.
Environments and containers are large, to clean up old ones use:

  • Containers – docker images and docker rmi ...
  • Local Conda environments – conda env list and conda env remove ...

Testing

Once Jupyter is up and running (with run-container or run-jupyter)
navigate in the left panel to any of the sample notebooks and run them.
The sample notebooks come from the RAPIDS notebooks repo
https://github.com/rapidsai/notebooks
From the command line in your environment, or inside the container, the
run-notebook <notebook-file> command can also be used. Expect warnings
since the notebooks can depend on functions only available when using
Jupyter’s web UI.

Local Tools

Version 2.7.0 introduced the install-tools command (paired with purge-tools), which extends the functionality of the stack. Currently, the list includes:

  • jupyter-repo2docker Point it to a github repository and it will create a docker container, and launch a jupyter notebook inside it
  • Nvidia GPU Cloud CLI This is perhaps the easiest way to interact with Nvidia assets
  • Kaggle CLI Allows users to sync up and manage Kaggle kernels, datasets, etc. locally
  • AWS CLI Allows users to remotely manage resources in AWS. The stack supports it via docker, so make sure you have docker installed.

Creating Custom Stacks

Creating custom environments is covered in the
Custom Data Science Stack Environments README.

Minimum Hardware and Software

  • NVIDIA GPU – Pascal, Volta, or Turing family GPU(s) including:
    • Quadro P, GV, and RTX series
    • Tesla P, V and T series
    • GeForce 10xx and 20xx
  • Operating System:
    • Ubuntu 18.04 or 20.04
    • Red Hat Enterprise Linux Workstation 7.5+ or 8.0+ (requires license)
    • Other Linux distributions are NOT supported, but may work as long as
      the driver and Docker work.

Operating System Setup

Disable “Secure Boot” in the system BIOS/UEFI before installing Linux.

Ubuntu

The Data Science stacks are supported on Ubuntu LTS 18.04.1+ or 20.04
with the 4.15+ kernel. Ubuntu can be downloaded from
https://www.ubuntu.com/download/desktop

Red Hat Enterprise Linux Workstation (RHEL)

The Data Science stacks are supported on Red Hat Enterprise Linux Workstation(RHEL) version 7.5+ or 8.x.
The RHEL ISO image can be downloaded with the instructions on:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/installation_guide/chap-download-red-hat-enterprise-linux

Red Hat Subscriptions

A Red Hat subscription will be needed to install and use Red Hat Enterprise
Linux. A subscription also lets the system obtain update packages and
additional packages for Red Hat Enterprise Linux. Either purchase a
subscription or obtain a free evaluation subscription from the
Red Hat Software & Download Center –
https://access.redhat.com/downloads
Register the system with the Red Hat Customer Portal to complete the initial
setup. See the How to Register and Subscribe a system to the Red Hat
Customer Portal using Red Hat Subscription-Manager for further information –
https://access.redhat.com/solutions/253273

Windows Subsystem for Linux (WSL v2)

Note: This functionality is alpha only (and containers only) until WSL v2 becomes production ready
Follow the install instructions for WSL v2 with CUDA support.
Then, create a Ubuntu or RHEL VM, open a terminal, and follow OS-specific instructions above.
_Note: WSL v2 currently requires CUDA 11.0, while data science stack 2.9.0 is based on CUDA 11.2. Therefore, WSL v2 is supported via containers only.

Installing the NVIDIA GPU Driver

It is important that updated NVIDIA drivers are installed on the system.
The minimum version of the NVIDIA driver supported is 460.39.
More recent drivers may be available, but may not have been tested with the
data science stacks.

Ubuntu or RHEL v8.x Driver Install

Driver install for Ubuntu is handled by data-science-stack setup-system
so no manual install should be required.
If the driver if too old or the script is having problems, the driver can
be removed (this may have side effects, read the warnings) and reinstalled:

./data-science-stack purge-driver
# reboot
./data-science-stack setup-system
#...


↑