LOG IN
START YOUR FREE TRIAL

Search

Setup Jupyter on AWS in 3 Easy Steps

Before getting started I will warn you. This is like putting together your car. It’s a nice side project but long-term not very practical. It’s complicated, time-consuming, and requires lots of knowledge. Maintenance over time will be a nightmare. Oh, and, the result might not have the best user experience.

What’s the upside? You’ll get Jupyter running in the cloud! Be able to do ALL the data science! Oh, and bragging rights.

Until something breaks… 💥

Anyway, let’s continue. 

HINT: There’s a much faster, more reliable, easier solution to Jupyter in the cloud. It's called Callisto. Details at the end.

The overall process is setting up an AWS account, launching a server, installing software, configuring network/security, and of course, finally, running Jupyter Notebooks. This article is for demonstration purposes. It will lay out the general process with moderate detail to be informative.

  1. Create and activate an AWS account 
    • Create an Account using email
    • Verify email 
    • Create password
    • Add contact information – with caveats for not using a personal number because it’s forever tied to your “root” account, making it insecure.
    • Add Payment methods (required)
    • Verify the phone number (required)
    • Choose Support plan (required)
    • Wait for Account Activation (a few minutes to 24 hours) 
    • Troubleshoot delays in activation if necessary
  2. Launch an EC2 Instance
    • Configure all the necessary info
      • Name and tags, for Name,
      • AMI (choose Ubuntu or AWS Deep Learning AMIs – if you need popular ML frameworks, NVIDIA drivers, and CUDA preinstalled). Going forward, we’ll assume you chose either of these two.
      • Instance Type. For Machine Learning, you may want GPU-based instances. See Amazon EC2 Instance Types – Amazon Web Services
      • Key pair (login)
      • Network settings and security group
      • Start and wait for it to boot up
  3. Connect to your instance  
  4. Once connected, install dependencies you need for your workflow. This could be GPU, CUDA, drivers, etc. If you have the AWS Deep Learning AMI, this will be preinstalled.
  5. Install miniconda
  6. Configure Jupyter notebook: jupyter notebook --generate-config 
  7. Set up password and copy hash by running:
    • from notebook.auth import passwd; passwd()
  8. Exit python and create certificates
    • mkdir certs
    • cd certs
    • openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mykey.key -out mycert.pem
    • This is not a trusted certificate and you will need to circumvent your browser’s protections against untrusted certificates to access your jupyter notebook server this way. If you want a trusted certificate, you’ll need to use something like Let’s Encrypt. See: Running a notebook server — Jupyter Notebook
    • sudo vim /home/ec2-user/.jupyter/jupyter_notebook_config.py
    • Set everything as follows:
      • c.NotebookApp.password=”  //(hashed-password)
      • c.NotebookApp.ip=’0.0.0.0′
      • c.NotebookApp.open_browser=False
      • c.NotebookApp.port=8888
      • c.NotebookApp.certfile=” //(the directory for mycert-pem)
  9. Run jupyter notebook while on ssh connection with ec2
    • run: jupyter notebook
    • or to keep the server running when you close the terminal: nohup jupyter notebook &
  10. We’re almost ready to use our server. Before moving forward, a note about $$$. Servers are charged hourly. Storage is charged based on the amount and lifetime of the storage. When not using the server, make sure to turn it off! (*whisper* Callisto does this automatically)
  11. Finally! The server can be accessed by going to https://(yourAWS public dns):8888/
    • The AWS public DNS might change every time you start it so make sure you get the latest one from the AWS portal.
  12. Optional but recommended step: Port forward and use SSL cert. Forward server port through ssh so we can connect from localhost in the browser. Otherwise, the browser will complain about https://(yourAWS public dns):8888/ not having the right certs and will think it’s “unsafe”. If you do this, make sure to use it with SSL certificate. Test by Logging in to the Jupyter notebook server – Deep Learning AMI 

Whew!

That was a lot! I honestly don’t expect anyone to do it by themselves. It’s quite complicated with many moving parts. At Callisto, we have dedicated professionals to keep our cloud systems up and running 24/7.

Callisto: Simple solution for Cloud Jupyter Notebooks!

(for macOS and iOS)

Why are we putting so much effort into creating an easy, fun, user-friendly data science platform? It’s so that you can spend more time researching and discovering! With Callisto, it’s easy. We promise!

3 steps to start your Callisto cloud server:

  1. Sign up at www.callistoapp.com – New Users get FREE credits
  2. Download and Login to Callisto Mac App 
  3. Add and Configure Your Server:
    • Click Cloud button -> Add Server -> Add Callisto Server
    • Give your server a Name, select CPU/GPU and Storage Capacity (GB)

Voila! Callisto starts up your new server…

As you might have guessed, Callisto will take care of everything that was mentioned in the EC2 steps. 

Callisto Cloud Servers will handle

  • Network Setup
  • Certificates
  • Security
  • Installing all the popular data science software, drivers, packages, and libraries.
  • Provide hardware that fits every CPU/GPU/Memory hardware needs
  • Intuitive notebook UI that’s fun to use (in Callisto macOS/iOS app)
  • UI to manage Python and R packages for Jupyter Notebooks
  • Automatically hibernate servers when not in use – no extra charges for servers that were left active overnight! When you’re ready the next day, you can pick up where you left off.
  • Oh, and we have a dedicated support team to help you get the most out of your data science journey

That’s quite the list! But I’m sure you’ll find more things you like with Callisto.

Start your cloud computing workflow at www.callistoapp.com