3/30/2023
Articles on setting up python environment
4/01/2023
- Quote from aseifert’s article: “Pip, venv, virtualenv, pyenv, pipenv, micropipenv, pip-tools, conda, miniconda, mamba, micromamba, poetry, hatch, pdm, pyflow 🤯 These days even the most senior Python developer is confused about all the options to manage environments.”
- After successful install of mambaforge from github (see ON notes), i verified that both conda info and mamba info work. Eventually, I should read this page from the mamba docs site.
- Took multiple steps, but using this YouTube videoI was able to: (1) install mamba, (2) install python 3.10 within mamba, (3) install jupyter lab, (4) activate the jupyter mamba ‘space’ or whatever it’s called, and (5) launch the jupyter server and connect to it from my local web client.
- Out of curiosity, I wondered how well PyTorch works with Apple Silicon and was led by this reddit comment to this PyTorch GitHub issue
- Should go through the basic PyTorch tutorial and download the Jupyter notebook to run locally or can run through Google Colab.
4/02/2023
Jupyter Notes
- Mamba + Jupyter setup video How to install JupyterLab using Mamba
- In the proper mamba environment, run ‘jupyter notebook’ and it should start the server in the terminal and automatically open a web browser client. Success!
PyTorch Notes
- Installing pytorch using conda / mamba. Gated Medium Towards Data Science article
- Successfully installed stable 2.0.0 version for Apple Silicon using ‘mamba install pytorch torchvision -cpytorch’
Mamba / Conda Notes
- To see list of conda environments created, use this command ‘conda info –envs’
- Steps:
- install conda by navigating to mamba-forge at github. Download the “Mambaforge-MacOSX-arm64.sh”, navigate to the directory, and then execute the shell script by typing ‘Mambaforge-MacOSX-arm64.sh’ from the command line.
- create an environment for pytorch. E.g., I want to call my personal pytorch environment pytorch-jh, so i type ‘mamba create -n pytorch-jh -c conda-forge python=3.10’
- ‘create -n pytorch-jh’ creates the environment called pytorch-jh
- ‘create -c conda-forge’ indicates i need to download python from the conda-forge channel
- ‘python=3.10’ pins the version of python as 3.10 for the pytorch-jh environment
- Download jupyter packages into pytorch-jh with this command ‘mamba install -n pytorch-jh -c conda-forge jupyterlab’
- Download PyTorch packages into pytorch-jh with this command ‘mamba install -n pytorch-jh pytorch torchvision -cpytorch’
- ‘install -n pytorch-jh’ means install into the pytorch-jh environmnet
- the list of packages per PyTorch.org are: pytorch, torchvision,
- i’m not exactly sure what ‘-cpytorch’ is doing but it somehow knew to download the mac-os-arm64 version so I think this is related
- Verified that pytorch is working and using the M2 GPU by running 1jh_test.py and 2jh_test.py adapted from the above Towards Data Science article
4/06/2023
- Used ChatGPT to find latest in Python books and courses
- Thoughts and notes on lists (ordered) vs. dictionaries (unorderd key-value pairs, allowing duplicate values) vs. sets (like dicts but unique with no distinct values)
- examples of iterables. All of the 3 above plus strings. See
- zip() function. strip() / lstrip() / rstrip() to get rid of whitespace characters. difference between pop() and delete().
- using python interpreter, up to ‘Organizing a List’ on p. 42
- when in doubt about using an array versus a standard python list, use the better, more mathematically useful numpy.array()
4/07/2023
- Simple explanation for ‘+=’ operator. Initialize ‘a = 10’. To set a to 17, you can then type ‘a += 7’. Then, print(a) will return ‘17’.
4/12/2023
- Found via phind.com’s answer about tensors vs. multi-dimensional array. This article a pretty good simple way of getting familiar with PyTorch tensors. Requires import of torch, numpy, and pandas
4/13/2023
- Comprehensions are useful syntatic sugar for quickly populating and looping through both lists and dictionaries.
- Concept of “collections” useful noun as explained in this YouTube about list comprehensions (80 seconds in)
4/27/2023
- Began reading Bruce Eckel’s Thinking in Python old PDF.
7/08/2024
- Bunch of info on Python environment/version/package managers on Hacker News today.
- This article by Larry Du summarizes state of the art as of July 2024. HN thread.
- Introduction to Rye, Intro YouTube video, and HN thread.
- In 2024, the core developers behind mamba have also started working on uv a new Python packager written in Rust. Per the Larry Du article above, “uv is by far the most promising package management tool in the Python ecosystem as of the writing of this post. This project actually aims to be a drop-in replacement for pip on top of being a Cargo for Python. The API is currently in no ways stable (as of 2024), but the benchmarks are incredibly promsing. Most notably, the development is backed by Astral.sh, a company formed by Charlie Marsh and the creators of the ruff linter, a widely beloved tool that virtually supplanted all incumbants overnight when it was released in 2022.”